As of 2024-03-14T11:02 UTC the Toolforge Grid Engine service has been shutdown.[0][1]
This shutdown is the culmination of a final migration process from Grid Engine to Kubernetes that started in in late 2022.[2] Arturo wrote a blog post in 2022 that gives a detailed explanation of why we chose to take on the final shutdown project at that time.[3] The roots of this change go back much further however to at least August of 2015 when Yuvi Panda posted to the labs-l list about looking for more modern alternatives to the Grid Engine platform.[4]
Some tools have been lost and a few technical volunteers have been upset as many of us have striven to meet a vision of a more secure, performant, and maintainable platform for running the many critical tools hosted by the Toolforge project. I am deeply sorry to each of you who have been frustrated by this change, but today I stand to celebrate the collective work and accomplishment of the many humans who have helped imagine, design, implement, test, document, maintain, and use the Kubernetes deployment and support systems in Toolforge.
Thank you to the past and present members of the Wikimedia Cloud Services team. Thank you to the past and present technical volunteers acting as Toolforge admins. Thank you to the many, many Toolforge tool maintainers who use the platform, ask for new capabilities, and help each other make ever better software for the Wikimedia movement. Thank you to the folks who who will keep moving the Toolforge project and other technical spaces in the Wikimedia movement forward for many, many years to come.
[0]: https://sal.toolforge.org/log/DrOgPI4BGiVuUzOd9I1b [1]: https://wikitech.wikimedia.org/wiki/Obsolete:Toolforge/Grid [2]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation#T... [3]: https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/ [4]: https://lists.wikimedia.org/pipermail/labs-l/2015-August/003955.html
Bryan, on behalf of the Toolforge administrators
As someone who has been migrating a lot of tools, and who has been at times upset^W frustrated at some of the proverbial devils of the details, I want to thank Bryan, and everyone involved, for the sustained effort to keep toolforge going into the future, and congratulations on a job well done.
On Thu, Mar 14, 2024 at 11:04 PM Bryan Davis bd808@wikimedia.org wrote:
As of 2024-03-14T11:02 UTC the Toolforge Grid Engine service has been shutdown.[0][1]
This shutdown is the culmination of a final migration process from Grid Engine to Kubernetes that started in in late 2022.[2] Arturo wrote a blog post in 2022 that gives a detailed explanation of why we chose to take on the final shutdown project at that time.[3] The roots of this change go back much further however to at least August of 2015 when Yuvi Panda posted to the labs-l list about looking for more modern alternatives to the Grid Engine platform.[4]
Some tools have been lost and a few technical volunteers have been upset as many of us have striven to meet a vision of a more secure, performant, and maintainable platform for running the many critical tools hosted by the Toolforge project. I am deeply sorry to each of you who have been frustrated by this change, but today I stand to celebrate the collective work and accomplishment of the many humans who have helped imagine, design, implement, test, document, maintain, and use the Kubernetes deployment and support systems in Toolforge.
Thank you to the past and present members of the Wikimedia Cloud Services team. Thank you to the past and present technical volunteers acting as Toolforge admins. Thank you to the many, many Toolforge tool maintainers who use the platform, ask for new capabilities, and help each other make ever better software for the Wikimedia movement. Thank you to the folks who who will keep moving the Toolforge project and other technical spaces in the Wikimedia movement forward for many, many years to come.
Bryan, on behalf of the Toolforge administrators
Bryan Davis Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808 _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
On Fri, Mar 15, 2024 at 2:40 AM Magnus Manske via Wikitech-l wikitech-l@lists.wikimedia.org wrote:
As someone who has been migrating a lot of tools, and who has been at times upset^W frustrated at some of the proverbial devils of the details, I want to thank Bryan, and everyone involved, for the sustained effort to keep toolforge going into the future, and congratulations on a job well done.
Thank you very much Magnus for those words of support. As you very well know yourself, the folks who build things to help others are more likely to hear about frustrations when their efforts have fallen short than they are to hear praise when they are succeeding. Hearing from you and others that we are on the correct path goes a long way towards keeping up the energy to continue.
In that spirit, I would like to name a few folks specifically who have chopped more wood and carried more water in the final push to the deadline than even I had hoped they would. David Caro, Taavi Väänänen, and Seyram Komla Sapaty all put in great amounts of time and effort over the last few months to make this project as successful as possible.
David went above and beyond in to try and ensure that Toolforge had the technical capabilities needed to allow even less widely used runtime languages to migrate. Go stalk him a bit on Phabricator to see the patience and attention he used to help specific tools make it across, and be sure to check out the many "My first Buildpack" tutorials that he helped publish.
Taavi carried on a Toolforge admin tradition of finding ways to use Toolforge itself to provide new features for others by making a reusable custom image for running Pywikibot scripts using Build Service. He also spent quite a lot of extra time helping folks 1 on 1 with questions via IRC and Phabricator.
Komla used every means he could think of to try and contact folks who's tools were in need of attention--Phabricator tickets, Talk page messages, direct emails, tracking down SUL accounts for Developer accounts that were not responsive. He also kept the rest of us on the Toolforge admin team informed of counts of tools remaining and trends in feedback to consider. He should have had the honor of sending the final shutdown announcement yesterday, but all 3 of his different ISPs were disrupted by undersea cable cuts![0]
I would actually like to name one more person, Nicholas Skaggs, as having been critical to the final steps in converting from Grid Engine to Kubernetes. Nicholas was the manager of the Wikimedia Cloud Services team from June 2020 through February 2024. Nicholas had extreme faith in the abilities of the team and the larger technical community. He urged us to complete the difficult work needed for the migration rather than putting it off yet again with "just one more" upgrade of the Grid exec and control nodes. I and others will miss his leadership at the Foundation and wish him well in the next phase of his career.
[0]: https://blog.cloudflare.com/undersea-cable-failures-cause-internet-disruptio...
Bryan -- Bryan Davis Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808
wikitech-l@lists.wikimedia.org