Cloud February 2022

cloud@lists.wikimedia.org

10 participants
5 discussions

[Cloud-announce] Eliminating Debian Stretch in Cloud VPS
by Andrew Bogott 13 Oct '22

13 Oct '22

Debian Stretch's security support ends in mid 2022, and the Foundation's OS policy already discourages use of existing Stretch machines. That means that it's time for all project admins to start rebuilding your VMs with Bullseye (or, if you must, Buster.) Any webservices running in Kubernetes created in the last year or two are most likely using Buster images already, so there's no action needed for those. Older kubernetes jobs should be refreshed to use more modern images whenever possible. If you are still using the grid engine for webservices, we strongly encourage you to migrate your jobs to Kubernetes. For other grid uses, watch this space for future announcements about grid engine migration; we don't yet have a solution prepared for that. Details about the what and why for this process can be found here: https://wikitech.wikimedia.org/wiki/News/Stretch_deprecation Here is the deprecation timeline: March 2021: Stretch VM creation disabled in most projects July 6, 2021: Active support of Stretch ends, Stretch moves into LTS <- You are Here -> January 1st, 2022: Stretch VM creation disabled in all projects, deprecation nagging begins in earnest. Stretch alternatives will be available for tool migration in Toolforge May 1, 2022: All active Stretch VMs will be shut down (but not deleted) by WMCS admins. This includes Toolforge grid exec nodes. June 30, 2022: LTS support for Debian Stretch ends, all Stretch VMs will be deleted by WMCS admins _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 1

[Cloud-announce] [IMPORTANT] Announcing Toolforge Debian Stretch Grid Engine deprecation
by Seyram Komla Sapaty 10 Mar '22

10 Mar '22

Hello, all! This email contains valuable information about the Toolforge service. Starting today, we're initiating a process to migrate away from Debian Stretch to Debian Buster for all of Toolforge servers, and the most affected piece is the Grid Engine backend in particular. Debian Stretch was released in June 2017, and long term support for it (including security updates) will cease in June 2022. We need to shut down all Stretch hosts before the end of support date to ensure that Toolforge remains a secure platform. This migration will take several months because many people still use the Stretch hosts and our users are working on tools in their spare time. You should be aware that our ultimate goal is to deprecate Grid Engine entirely and replace it with Kubernetes. Read below for more information on this. == Initial timeline == Subject to change, see Wikitech[1] for living timeline. * 2022-02-15: Availability of Debian Buster grid announced to community * 2022-03-21: Weekly reminders via email to tool maintainers for tools still running on Stretch * Week of 2022-04-21: ** Daily reminders via email to tool maintainers for tools still running on Stretch ** Switch login.toolforge.org to point to Buster bastion * Week of 2022-05-02: Evaluate migration status and formulate plan for final shutdown of Stretch grid * Week of 2022-05-21: Shut down Stretch grid == What is changing? == * New bastion hosts running Debian Buster with connectivity to the new job grid * New versions of PHP, Python3, and other language runtimes * New versions of various support libraries == What should I do? == You should migrate your Toolforge tool to a newer environment. You have two options: * migrate from Toolforge Stretch Grid Engine to Toolforge Kubernetes[3]. * migrate from Toolforge Stretch Grid Engine to Toolforge Buster Grid Engine. The Cloud Services team has created the Toolforge Stretch deprecation[0] page on wikitech.wikimedia.org to document basic steps needed to move web services, cron jobs, and continuous jobs from the old Stretch grid to the new Buster grid. That page also provides more details on the language runtime and library version changes and will provide answers to common problems people encounter as we find them. If the answer to your problem isn't on the wiki, ask for help using any of our communication channels[2]. We encourage you to move to Kubernetes today if you can, see below for more details. For those who can't migrate to Kubernetes, the Debian Buster grid should be adopted within the next three months. == A note on the future of Toolforge, the Grid and Kubernetes == As of today, Toolforge is powered by both Grid Engine and Kubernetes. For a number of reasons, we have decided to deprecate Grid Engine and replace all of its functions with Kubernetes. We're not yet ready to offer all grid-like features on Kubernetes, but we're working on it. As soon as we are able, we will begin the process of migrating the workloads and shutting down the grid. This is something we hope to do between 2022 and 2023. We share this information to encourage you to evaluate migrating your tool away from Grid Engine to Kubernetes. One of the most prominent missing features on Kubernetes was a friendly command line interface to schedule jobs (like jsub). We've been working on that, and have a beta-level interface that you can try today: the Toolforge jobs framework [4]. [0]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation [1]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Time… [2]: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/About_Toolforge#Commun… [3]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes [4]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs-Framework Thanks. -- Seyram Komla Sapaty Developer Advocate Wikimedia Cloud Services _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

7 7

Learn how to develop bots in small wiki toolkit workshops
by Srishti Sethi 23 Feb '22

23 Feb '22

Hi all, TLDR; Want to learn new technical skills to help improve your wiki? Sign up for workshops on scripts and bot development, organized by the small wiki toolkits initiative: < https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops> [1]. Small wiki toolkits (SWT) is an initiative to build technical capacity in smaller language wikis by developing toolkits, conducting workshops, and providing technical support < https://meta.wikimedia.org/wiki/Small_wiki_toolkits> [2]. The first set of workshops for 2022 will teach individuals how to automate work in their wiki community by developing, maintaining, and hosting bots. These trainings build upon the lessons learned from two regional initiatives in 2021: <https://meta.wikimedia.org/wiki/Small_wiki_toolkits#Past_events> [3]. To sign up for a workshop, add your signature under a specific workshop section on this page < https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops> [4]. The first workshop is scheduled for February 25th, 16:00 UTC. The organizing team will share more details on mailing lists and on your talk page closer to the date of the workshop. The SWT Organizing Team is also exploring issuing digital certificates and badges to attendees of these workshops. More information on that will be available in the coming weeks. We look forward to your participation! Cheers, Srishti (On behalf of the SWT Workshops Organization team) [1] https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops [2] https://meta.wikimedia.org/wiki/Small_wiki_toolkits [3] https://meta.wikimedia.org/wiki/Small_wiki_toolkits#Past_events [4] https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops *Srishti Sethi* Senior Developer Advocate Wikimedia Foundation <https://wikimediafoundation.org/>

1 1

16 Feb '22

Hello. Yesterday, I started scheduling my cron jobs on the grid with the `--release buster` option (previously was defaulting to Stretch). I have gotten dozens of "Someone (probably you) recently logged in to your account from a new device" notifications since then. Can I do anything to stop this other than disable those notifications altogether? Thanks, PB

4 3

django with oauth?
by Roy Smith 16 Feb '22

16 Feb '22

I'm running Django 3.1 and social-auth-app-django 3.1.0 so I can authenticate against MediaWiki. Everything works, but want to start moving toward more modern releases. If I upgrade social-auth-app-django to anything newer, I get an exception in the authentication flow: Direct assignment to the forward side of a many-to-many set is prohibited. Use groups.set() instead. Has anybody else seen this problem? It looks like this was first reported about 18 months ago <https://github.com/python-social-auth/social-app-django/issues/256>, with no response to the issue. I fear that the whole python-social-auth project is close to abandoned. There's been a total of 4 commits in the last 6 months. Unfortunately, I'm not aware of any viable alternative.

2 1

2024

2023

2022

2021

2020

2019

2018

2017

Cloud February 2022