Cloud February 2025

cloud@lists.wikimedia.org

10 participants
14 discussions

Several database changes happening in the coming months
by Amir Sarabadani 24 Apr '25

24 Apr '25

Hello, in the next coming months, these changes will happen in databases and the infrastructure. And it might affect you if you rely on them in your tools or queries. This list is ordered based on how soon the change will happen. We understand that updating your tools and systems can be time consuming, hence we are giving an advanced notice. I truly apologize for the inconvenience but many of these changes are needed to keep the site running smoothly. Image table redesign Around fourteen years after the creation of T28741 <https://phabricator.wikimedia.org/T28741>, we are implementing the changes described therein. Currently, every current version of an image has a row in the image table and if there are older versions of that file, those rows could be found in the oldimage table. These two tables (image and oldimage) will be dropped in around two months. The replacement will be two main tables: file and filerevision. Every file will have a row in the file table describing the name and the type. Every version of the file (current and old) will have a row in filerevision describing the file-specific information such as its size or the hash of the file, similar to the existing distinction between pages and revisions. Another improvement is that every file and file revision will get a unique auto increment id simplifying many operations and queries. You can check T28741 <https://phabricator.wikimedia.org/T28741> for more information. The new tables are already accessible in wikireplicas but the data hasn’t been fully migrated yet. Term store split out of wikidata’s database Wikidata’s database has been growing too fast and we need to move the term store (tables starting with wbt_) to a dedicated cluster to allow growth and improve wikidata’s performance by utilizing cache locality. The new section will be called x3 and you will be able to access it in wikireplicas but this also means you won’t be able to join these tables with the rest of wikidata’s database (such as page table) since they will be residing in two physically separate servers that also means most of your queries to wikidata’s database (and term store) will become faster. We are aiming for the switch to happen in three months’ time. You can follow the work in T351820 <https://phabricator.wikimedia.org/T351820>. Additionally, wb_type table will be dropped and the mapping will be hard-coded in the code instead. See gerrit:1110810 <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/1110810> for more details. This helped us simplify a lot of Wikibase code (example <https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/1110720>). Categorylinks normalization Categorylinks is the next table in the series of links tables being normalized via the linktarget table (parent ticket <https://phabricator.wikimedia.org/T300222>, RFC <https://phabricator.wikimedia.org/T222224>). Similar to templatelinks and pagelinks tables, cl_to will be dropped and instead the new field cl_target_id will point to lt_id in the linktarget table. We will also drop the cl_collation field and replace it with cl_collation_id which will point to the collation_id field on the new table we are introducing called collation. We are aiming to get this fully done by the end of the next quarter (end of June 2025) but it depends on how fast the migration script can operate and that’s outside of our control. You can follow the work in T299951 <https://phabricator.wikimedia.org/T299951>.It’s worth noting that after this migration is done, we will start working on the imagelinks table. Thank you -- *Amir Sarabadani (he/him)* Staff Database Architect Wikimedia Foundation <https://wikimediafoundation.org/>

1 1

Problems login into horzion
by Dirk Hünniger 11 Mar '25

11 Mar '25

Hi, I got problems logging in at https://horizon.wikimedia.org/ (see image below). When I click login I see "one moment please" and after that "Submitting..." and then code 401 message "The request you have made requires authentication." title "Unauthorized" The final url is https://openstack.eqiad1.wikimediacloud.org:25000/v3/auth/OS-FEDERATION/web… What can I do? Yours Dirk

3 5

[Cloud-announce] Wikitech SUL migration is complete!
by Bryan Davis 04 Mar '25

04 Mar '25

Thanks to a final burst of effort by Amir [0], the Wikitech SUL account migration [1] is now complete. Wikitech is currently a "normal" Wikimedia SUL wiki and no longer in any way connected to the Developer account system. [2] This change has long been imagined [3] as a way to make it easier for the general Wikimedia community to contribute to the technical documentation for the Wikimedia production services and Cloud VPS & Toolforge documentation maintained on Wikitech. We decided to undertake this work back in July 2024 [4] primarily to reduce the complexity of moving Wikitech's hosting into the same Kubernetes cluster [5] as the rest of the Wikimedia wikis. That initial goal was accomplished in October 2024 [6], but it took quite a bit more time and energy to attempt to reconcile the legacy Developer accounts with the SUL account system. After stalling out for a bit, we came up with a new roadmap and implementation plan in early February [7] that we were thankfully able to sustain to the conclusion today. [0]: https://en.wikipedia.org/wiki/User:Ladsgroup [1]: https://wikitech.wikimedia.org/wiki/News/2024_Migrating_Wikitech_Account_to… [2]: https://www.mediawiki.org/wiki/Developer_account [3]: https://phabricator.wikimedia.org/T161859 [4]: https://wikitech.wikimedia.org/w/index.php?title=News/2024_Migrating_Wikite… [5]: https://wikitech.wikimedia.org/wiki/Kubernetes/Clusters#WikiKube [6]: https://phabricator.wikimedia.org/T292707 [7]: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/… Bryan, on behalf of the Wikitech admins and everyone who helped with the migration -- Bryan Davis Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808 _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

2 1

Re: [Wikimedia-l] Upcoming Language Community Meeting (Feb 28th, 14:00 UTC) and Newsletter
by Srishti Sethi 25 Feb '25

25 Feb '25

Hello everyone, This is a reminder that the Language Community Meeting is taking place this coming Friday, February 28th, at 14:00 UTC. Please see more details and how to join here: < https://www.mediawiki.org/wiki/Wikimedia_Language_and_Product_Localization/… >. Highlights of this meeting will include learning about: * Keyboard and tools for the Sámi languages, presented by the Indigenous language technology group at UiT The Arctic University of Norway. * Translation contributions from the Bahasa Lampung community in Indonesia. Looking forward to your participation! Cheers, Srishti & Oscar *Srishti Sethi* Senior Developer Advocate Wikimedia Foundation <https://wikimediafoundation.org/> On Mon, Feb 10, 2025 at 10:01 AM Oscar Costero <oscarcostero86(a)gmail.com> wrote: > Hello everyone! > > We’re excited to announce that the next *Language Community Meeting* is > happening soon, *February 28th at 14:00 UTC*! If you’d like to join, > simply sign up on the *wiki page > <https://www.mediawiki.org/wiki/Wikimedia_Language_and_Product_Localization/…>* > . > > This is a participant-driven meeting where we share updates on > language-related projects, discuss technical challenges in language wikis, > and collaborate on solutions. In our last meeting, we covered topics like > developing language keyboards, creating the Moore Wikipedia, and updates > from the language support track at Wiki Indaba. > > *Got a topic to share?* Whether it’s a technical update from your > project, a challenge you need help with, or a request for interpretation > support, we’d love to hear from you! Feel free to *reply to this message* or > add agenda items to the document *here > <https://etherpad.wikimedia.org/p/language-community-meeting-feb-2025>*. > > Also, we wanted to highlight that the sixth edition of the Language & > Internationalization newsletter (January 2025) is available here > <https://www.mediawiki.org/wiki/Wikimedia_Language_and_Product_Localization/…>. > This newsletter provides updates from the October–December 2024 quarter on > new feature development, improvements in various language-related technical > projects and support efforts, details about community meetings, and ideas > for contributing to projects. To stay updated, you can subscribe to the > newsletter on its wiki page > <https://www.mediawiki.org/wiki/Wikimedia_Language_and_Product_Localization/…> > . > > We look forward to your ideas and participation at the language community > meeting, see you there! > > Cheers, > _______________________________________________ > Wikimedia-l mailing list -- wikimedia-l(a)lists.wikimedia.org, guidelines > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org… > To unsubscribe send an email to wikimedia-l-leave(a)lists.wikimedia.org

1 0

[Cloud-announce] ToolsDB maintenance on Monday, February 24
by Francesco Negri 24 Feb '25

24 Feb '25

On Monday, February 24th at 14:00 UTC, I am going to upgrade MariaDB to the latest patch version 10.6.20. [0] This will cause a brief downtime for ToolsDB (just a couple of minutes). [0] https://phabricator.wikimedia.org/T385885 -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 1

Versatile (Meta) Wiki Data Sets
by Jamie Willoughby 18 Feb '25

18 Feb '25

I am a data science student at Wesleyan University. This spring, my professor asked me to choose a data set to visualize. She suggested one that contains a combination of dates, geographic information, and quantitative and qualitative variables. My mind went to Wikipedia. I've spent a long time searching, but the volume of available data is overwhelming. Can you recommend a specific Wiki dataset? Thank you.

3 3

[Cloud-announce] brief network reset tomorrow, 2025-02-19 13:00 UTC
by Andrew Bogott 18 Feb '25

18 Feb '25

We will be running a brief network failover test tomorrow which will cause some network connections to reset. There shouldn't be an long-lasting effects or any intervention needed on the part of users, so please ignore the flap. -Andrew _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 0

[Cloud-announce] Changes to DNS lookups on Monday, 2025-02-17
by Andrew Bogott 12 Feb '25

12 Feb '25

tl;dr: Minor change to DNS resolution[0] for toolforge and cloud-vps services are coming on Monday. Should have no effect but please yell if you see things break. The whole story: Back in 2020[1] we stopped associated new VMs with the .wmflabs top-level domain; since then all VMs have been accessed via .wikimedia.cloud instead. As of a few weeks ago, the last remaining .wmflabs VM was deleted, so we're now cleaning up code and config that supported that domain. Right now if you try to resolve a stand-alone hostname (e.g. by typing "ping mycoolserver") the resolver will try three different fqdns: first 'mycoolserver.<projectname>.eqiad1.wikimedia.cloud.' and, failing that, 'mycoolserver.<projectname>.eqiad.wmflabs.' and, failing that, the simple name 'mycoolserver.'. After Monday, that second fallback won't happen, so it'll either be 'mycoolserver.<projectname>.eqiad1.wikimedia.cloud.' or 'mycoolserver.'. Most people don't use simple hostnames anyway; for those users this change will have no effect. Some toolforge applications still use the old-fashioned 'tools-redis' or 'tools-db' hostnames; we already have changes in place to resolve those correctly after the change. If there are single hostname edge cases that I don't know about and can't find, their behavior may change or break in surprising ways. Note that fully qualified service names (for example gurwiki.analytics.db.svc.eqiad.wmflabs) are unaffected by this update. I would love to eliminate them too, but it's unclear how to identify and remove all uses so that cleanup will be left for another day. [0] https://gerrit.wikimedia.org/r/c/operations/puppet/+/1118151 [1] https://wikitech.wikimedia.org/wiki/News/2020_Phasing_out_the_.wmflabs_doma… _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 0

[Cloud-announce] Last call for claiming a SUL name for your Wikitech account before 2025-02-24
by Bryan Davis 10 Feb '25

10 Feb '25

As announced on 2025-02-04 [0] there is a new timeline for the Wikitech SUL finalization. Today I completed the planned step of renaming and attaching all legacy local accounts that had been linked to a Wikimedia SUL account via idm.wikimedia.org or toolsadmin.wikimedia.org. Anyone who has missed, or ignored, the mailing list notifications and site notice on wikitech.wikimedia.org now has until 17:00 UTC on 2025-02-24 to use https://idm.wikimedia.org/login/mediawiki/ to claim the association of their Developer account and SUL account. At that time I will be making the final run of the scripts to rename claimed accounts. Soon after Amir or I will begin the work to rename all unclaimed accounts with a `~labswiki` suffix and attach them to SUL to complete the SUL migration. [0]: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/… Bryan, on behalf of the Wikitech admins and everyone helping with the migration -- Bryan Davis Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808 _______________________________________________ Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…

1 0

Changing SSH access username
by calendulaasteraceae＠gmail.com 06 Feb '25

06 Feb '25

I recently created a developer account, and only noticed after creation that there was a typo in my shell username. Who can I ask to fix this?

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

Cloud February 2025