December 2023 - Cloud - lists.wikimedia.org

Changes in schema of pagelinks tables
by Amir Sarabadani 15 Apr '24

15 Apr '24

(If you don’t work with pagelinks table, feel free to ignore this message) Hello, Here is an update and reminder on the previous announcement <https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…> regarding normalization of links tables that was sent around a year ago. As part of that work, soon the pl_namespace and pl_title columns of pagelinks table will be dropped and you will need to use pl_target_id joining with the linktarget table instead. This is basically identical to the templatelinks normalization that happened a year ago. Currently, MediaWiki writes to both data schemes of pagelinks for new rows in all wikis except English Wikipedia and Wikimedia Commons (we will start writing to these two wikis next week). We have started to backfill the data with the new schema but it will take weeks to finish in large wikis. So if you query this table directly or your tools do, You will need to update them accordingly. I will write a reminder before dropping the old columns once the data has been fully backfilled. You can keep track of the general long-term work in T300222 <https://phabricator.wikimedia.org/T300222> and the specific work for pagelinks in T299947 <https://phabricator.wikimedia.org/T299947>. You can also read more on the reasoning in T222224 <https://phabricator.wikimedia.org/T222224> or the previous announcement <https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…> . Thank you, -- *Amir Sarabadani (he/him)* Staff Database Architect Wikimedia Foundation <https://wikimediafoundation.org/>

3 5

Expired certificate?
by Roy Smith 20 Dec '23

20 Dec '23

Poking around on my debian bookworm instance, I found /usr/local/share/ca-certificates/wmf_ca_2017_2020.crt, which looks like an expired SSL certificate: > Certificate: > Data: > Version: 3 (0x2) > Serial Number: > 9f:14:76:9e:ea:f4:18:c3 > Signature Algorithm: sha256WithRSAEncryption > Issuer: C = US, ST = California, L = San Francisco, O = Wikimedia Foundation, OU = Operations, CN = WMF CA 2017-2020 > Validity > Not Before: Jul 19 20:43:26 2017 GMT > Not After : Jul 18 20:43:26 2020 GMT > Subject: C = US, ST = California, L = San Francisco, O = Wikimedia Foundation, OU = Operations, CN = WMF CA 2017-2020 Does this do anything useful? Does it do any harm?

4 3

Fwd: Installing elasticsearch on a VPS instance?
by Roy Smith 19 Dec '23

19 Dec '23

Trying this again without the screenshot, which got stuck in the filters. > Begin forwarded message: > > From: Roy Smith <roy(a)panix.com> > Subject: Installing elasticsearch on a VPS instance? > Date: December 17, 2023 at 4:00:08 PM EST > To: Wikimedia Cloud Services general discussion and support <cloud(a)lists.wikimedia.org> > > I want to install elasticsearch on a VPS instance. I’m guessing this involves puppet, but can’t figure out the details. The instructions at https://wikitech.wikimedia.org/wiki/Help:Puppet don’t match what I’m actually seeing in Horizon. I go to the Puppet Configuration tab for my instance and there’s no project/common/all sub-pages as described under Apply a puppet role to or change hiera config of an individual instance. > > https://horizon.wikimedia.org/project/instances/51660b10-1e33-4f49-a68f-e51…

2 5

[Cloud-announce] partial Toolforge outage
by Andrew Bogott 13 Dec '23

13 Dec '23

Toolforge just now suffered a partial grid-engine outage. All grid services should be back to normal as of this email; some k8s services may misbehave for the next hour or two. NFS misbehavior resulted in grid control mechanisms timing out, which meant that no new jobs could be scheduled for the last 90 minutes or so. We've rebooted the NFS server which has resolved the primary issues; however, rebooting NFS is itself disruptive and may have caused other jobs (both on the grid or in k8s) to fail. We're currently rebooting all k8s worker nodes, which will take a couple of hours to complete. During those reboots some jobs may fail or experience surprise rescheduling. Sorry for the outage! If your grid job was disrupted by this outage, please take this as a sign to migrate your service off the grid! Details about the grid shutdown can be found here: https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation#… -Andrew (+ Taavi who did most of the actual recovery work) _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 0

Reminder about the externallinks migration
by Amir Sarabadani 13 Dec '23

13 Dec '23

Hello, We have communicated <https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…> this change in August 2022 but here is a reminder that if you query externallinks table in wikireplicas, you will need to rework your queries. Changes are: el_to, el_index and el_index_60 fields will be dropped and you need to query el_to_domain_index and el_to_path fields instead. The data has been migrated in all wikis except English Wikipedia, Wikimedia Commons and Wikidata, you can keep track of the data migration in T326314 <https://phabricator.wikimedia.org/T326314>. In late June, we will stop updating the old fields and will start dropping them This is done to optimize storage of external links (drastically reducing its size by removing duplication) and enabling user requested features such as resolving two <https://phabricator.wikimedia.org/T14810> fifteen-year-old tickets <https://phabricator.wikimedia.org/T17218> or easier aggregating of external links domain [1], something that wasn’t possible until now. [1] For example, If you want to get list of the top-linked websites, you can query `SELECT el_to_domain_index, count(*) from externallinks group by el_to_domain_index order by count(*) desc limit 50;` Thank you and sorry for the inconvenience. -- *Amir Sarabadani (he/him)* Staff Database Architect Wikimedia Foundation <https://wikimediafoundation.org/>

3 4

Introducing Wishathon for Wikimedia’s Community Wishlist!
by Srishti Sethi 13 Dec '23

13 Dec '23

Hello everyone, Wishathon is a new initiative that encourages collaboration across the Wikimedia community to develop solutions for wishes collected through the Community Wishlist Survey: < https://meta.wikimedia.org/wiki/Community_Wishlist_Survey>. The inaugural community edition will take place from *March 15th to 17th, 2024*. You can learn more and sign up here: < https://meta.wikimedia.org/wiki/Event:WishathonMarch2024>. Since May 2022, the Community Tech team has organized five Wishathons < https://meta.wikimedia.org/wiki/Community_Tech/Wishathon> internal to Foundation staff to bring greater technical capacity to address open wishes and promote collaboration on wishes within the product & technology teams. Since the inception of the Wishathon, Foundation staff have helped Community Tech complete work on 10 wishes that were otherwise stalling or incomplete. This next edition of the Wishathon extends to Wikimedia contributors as a next step in these efforts. If you are interested in a project proposal as a user, developer, designer, or product lead, you are welcome to join the event! If you wish to take part in the Wishathon, you can register for the event on this wiki page: < https://meta.wikimedia.org/wiki/Event:WishathonMarch2024>. Organizers will send a message to your talk page with more details about the event in January 2024 and post updates on this page. Read the full announcement: < https://diff.wikimedia.org/2023/12/11/introducing-wishathon-for-wikimedias-…> Cheers, Srishti On behalf of the Wishathon organizing committee (Harumi, TheresNoTime, Karolin, Mary, Sheila, Srishti) *Srishti Sethi* Senior Developer Advocate Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

[Cloud-announce] Assurances and Support Regarding the Grid Engine Shutdown
by Seyram Komla Sapaty 09 Dec '23

09 Dec '23

Hello! After our initial announcement of the Grid Engine shutdown timeline[0], some of you raised concerns about losing your tools. We want to address those apprehensions while hopefully providing reassurance. No tools will be deleted until the grid engine shutdown date on 14 February 2023. However, for tools with unreachable maintainers, an outage will happen starting on 14 December 2023[1]. This is intended to raise awareness for users or maintainers who have not otherwise been reached. A list of these tools can be found here[2]. If you are a maintainer or a user of a tool in this list, comment on the associated phabricator ticket with migration plans or a request for more support. The goal is to have a plan for all tools running on the grid. We want all actively used tools to be migrated, and will help support users of critical tools without a maintainer. Thanks for your help in identifying and migrating those tools you maintain and depend on. We acknowledge that the timeline might seem tight, and we want to clarify that our approach is to make this process as seamless as possible. We have been actively engaging with tool maintainers over the past year, and we genuinely appreciate the efforts many of you have already made to migrate your tools to Kubernetes. We will continue to work closely with maintainers who might need additional time or assistance. If for any reason you have not received a phabricator ticket for your tool, please reach out. The phabricator ticket is a good place to communicate your needs and plans for any remaining tools or jobs. This will help us further organize and plan this process. Our primary goal is to support you through this transition. If you have further concerns about the deadline or if you need assistance with the migration process, please don't hesitate to reach out to us. We are available on IRC, Telegram, Phabricator[3], and through our other support channels[4]. Do you still have concerns or questions? Please let us know. We want to do this together with you, in a way which makes sense to everyone. We’re very grateful for all the hard work you do, and our only goal here is to secure the future of tools in the Wikimedia sphere, not to make your lives more difficult. Thank you! [0]: https://lists.wikimedia.org/hyperkitty/list/cloud-announce@lists.wikimedia.… [1]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation#… [2]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation/… [3]: https://phabricator.wikimedia.org/project/board/6135/ [4]: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/About_Toolforge#Commun… -- Seyram Komla Sapaty Developer Advocate Wikimedia Cloud Services _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 0

Re: [Cloud-announce] [IMPORTANT] Grid Engine Shutdown Timeline
by Amory 07 Dec '23

07 Dec '23

On Dec 7, 2023 at 07:00:08, cloud-request(a)lists.wikimedia.org wrote: > Hello Amory, > > We reached out directly on several occasions to maintainers via email, > phabricator and announcements were made on the mailing list. > What's the username you use and is this the same email address you use? Let > me crosscheck this again. Yes, I’m individually aware, but I was trying to note that, given I had not been directly contacted (via assigned phab task, etc.) AFAICT, it appeared possible some tools were being missed. For example, I am not listed on < https://grid-deprecation.toolforge.org>. ~A

2 1

[Cloud-announce] [IMPORTANT] Grid Engine Shutdown Timeline
by Seyram Komla Sapaty 07 Dec '23

07 Dec '23

Hello, all! Starting today we are kicking off the process to shut down Grid Engine and we want to share the timeline with you. == Background == WMCS made the Grid Engine available as a backend engine for hosting tools on Toolforge - our Platform as a Service(PaaS) offering. An additional backend engine, Kubernetes, was also made available on Toolforge. Over time, maintaining and securing the grid has proven to be difficult and making it harder to provide support to the community in other ways because a lot of man-hours of maintenance work is spent on this. This is mainly due to the fact that there has been no new Grid Engine releases (bug fixes, security patches, or otherwise) since 2016.[0] Maintenance work on the grid continued because it was widely popular with the community and the Kubernetes offering didn't yet have many grid-like features that contributors came to love. Once the Kubernetes platform could handle many of the workloads, we started the grid deprecation process by asking maintainers to migrate off the grid.[1] Over the past year, we've been reaching out to our tool maintainers and working with them to migrate their tools off the Grid to Kubernetes. We have reached out directly to all maintainers with their phabricator ticket IDs. The latest updates to Build Service[2] have addressed many of the issues that prevented tool maintainers from migrating. == Initial Timeline == The detailed grid shutdown timeline is available on wiki.[3] The important dates have been copied below. * 14th December, 2023: Any maintainer who has not responded on phabricator will have tools shutdown and crontabs commented out. Please plan to migrate or tell us your plans on phabricator before that date. * 14th February, 2024: The grid is completely shut down. All tools are stopped. If you need further clarification or help migrating your tool, don't hesitate to reach out to us on IRC, Telegram, Phabricator[4] or via any of our support channels.[5] Thank you. [0]: https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/ [1]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation [2]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service [3]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation#… [4]: https://phabricator.wikimedia.org/project/profile/6135/ [5]: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/About_Toolforge#Commun… -- Seyram Komla Sapaty Developer Advocate Wikimedia Cloud Services _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

8 8

Re: [Cloud-announce] [IMPORTANT] Grid Engine Shutdown Timeline
by Amory 07 Dec '23

07 Dec '23

Is there also the chance that those numbers are an undercount? I've got a bunch of simple cron tasks but they're using jlocal, not jsub, and I don't appear to have been contacted. The documentation vaguely alludes to this, but I wonder if there are a number of other tools/maintainers who have also fallen through the cracks. ~A

2 1

2024

2023

2022

2021

2020

2019

2018

2017

Cloud December 2023