Wikitech-l February 2024

wikitech-l@lists.wikimedia.org

29 participants
33 discussions

MediaWiki Insights: February edition
by Birgit Müller 01 Mar '24

01 Mar '24

Hi All, welcome to the monthly MediaWiki Insights email! Like last time <https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Reports/January_2…>, we’re starting this email by celebrating volunteer contributors who got their first patch merged in MW core, WMF deployed extensions or services over the past month: A big thanks to GergesShamon and Agent Isai for their contributions! Welcome :-) Many thanks also to the reviewers - volunteers and staff - of patches by new contributors. Your support is invaluable for helping people contribute to MediaWiki effectively and make it a fun first experience! The focus of this edition of the monthly MW Insights email lies on our multi-year initiatives: Migration of MediaWiki from bare metal to Kubernetes <https://wikitech.wikimedia.org/wiki/MediaWiki_On_Kubernetes> in Wikimedia Production, Parser Unification <https://www.mediawiki.org/wiki/Parsoid/Parser_Unification> and RESTBase deprecation <https://phabricator.wikimedia.org/project/profile/6289/>. These initiatives involve many people and projects, require orchestrated efforts and coordination, and will help us in many different ways. Another thing that these initiatives have in common is that we’re getting closer to being able to benefit from these efforts :-). Project Snapshots: Milestones reached on 3 multi-year projects MediaWiki on Kubernetes migration is 75% complete, completing migration of all internal traffic, all scheduled jobs and reaches 50% global traffic milestone! Often shortened as mw-on-k8s, MediaWiki on Kubernetes is a multi-year effort to move all of the MediaWiki deployments running on WMF production infrastructure to the new WikiKube platform. The migration is underway and very recently a new milestone was hit where the WikiKube platform now serves 50% of end user requests <https://phabricator.wikimedia.org/T290536>. At half mark, having more percentage of traffic on WikiKube, Service Operations is working with Release Engineering to make changes to monitoring tools to surface any issues during deployments to continue with further ramps. Also, almost the entirety of what is called internal traffic <https://phabricator.wikimedia.org/T333120>, that is traffic that is generated by applications running in the infrastructure and reaching out to MediaWiki for various purposes is migrated to WikiKube. A special mention should go to MediaWiki Jobs fully migrated just a week ago <https://phabricator.wikimedia.org/T349796>. Feel free to follow the higher umbrella task <https://phabricator.wikimedia.org/T290536> and/or MediaWiki_On_Kubernetes on Wikitech <https://wikitech.wikimedia.org/wiki/MediaWiki_On_Kubernetes>. This migration will unleash the ability to deploy multiple versions of code simultaneously. Also, this will help enhance platform capabilities to build dockerized isolated environments for coding, testing and even production debugging. MediaWiki on Kubernetes will allow us to deprecate and eventually remove a lot of our in-house developed code. Another benefit is that we will be able to react to sudden traffic spikes, like newsworthy events better, as the flexing up and down is a matter of configuration change. This enables efficient placement of workloads, packing workloads in a more environmentally friendly way, increasing hardware utilization. It takes a village to get this far: A big thanks to the Service Operations team (Clément Goubert, Giuseppe Lavagetto, Alexandros Kosiaris, Kamila Součková, Hugh Nowlan, Effie Mouzeli, Reuven Lazarus, Jannis Mayboom and Kavitha Appakayala) for their leadership on this project, the Release Engineering team (specifically: Dan Duvall, Jeena Huneidi, Tyler Cipriani and Ahmon Dancy) for their work on Blubber <https://wikitech.wikimedia.org/wiki/Blubber>, the Deployment Pipeline <https://wikitech.wikimedia.org/wiki/Deployment_pipeline> and Scap <https://wikitech.wikimedia.org/wiki/Scap>, Dom Walden from Quality and Test Engineering for crafting and executing a plan to test the first deployment of MediaWiki on Kubernetes, and everyone else who has contributed in the one way or the other! <3 More milestones: We’re seeing the first lights at the end of the tunnel of another multi-year initiative: The *parser unification* <https://www.mediawiki.org/wiki/Parsoid/Parser_Unification>. One week ago, Parsoid Read Views was rolled out to first wikis: Parsoid is now the default read views renderer on the Foundations’ Office Wiki and Wikitech DiscussionTools. This early experimentation allows us to find issues in a limited space, which will help us evaluate readiness of the feature and increase our confidence for future rollouts <http://mediawiki.org/wiki/Parsoid/Parser_Unification/Confidence_Framework>. A huge thanks to Subbu Sastry, Mateus Santos, CScott, Isabelle Hubert-Pallatin, Arlo Breault, Shannon Bailey, Yiannis Giannelos and Sérgio Lopes for making this work! Many thanks also to Daniel Kinzler: His work on the RESTBase deprecation directly helped us get to this milestone :-) *RESTBase deprecation*: We have been continuously working on decoupling services from RESTBase, aiming for the modernisation and sustainability of Wikimedia products in our services platform. The MediaWiki Interfaces team finished up reimplementation of Reading Lists endpoints in MW REST API <https://phabricator.wikimedia.org/T348491> and are now confirming with affected callers <https://phabricator.wikimedia.org/T357478> that the new endpoints meet their needs before rerouting calls <https://phabricator.wikimedia.org/T348493> and retiring old code <https://phabricator.wikimedia.org/T348494>. The overall effort <https://phabricator.wikimedia.org/T336693> will not only move us forward on RESTBase retirement, but also reduce the total amount of code we have to maintain. Many thanks to Bill Pirkle, Atieno Njira, Wendy Quarshie, and Daniel Kinzler for making this work! We also fully turned off Parsoid cache storage in RESTBase - clients will get outputs direct from MediaWiki and the cache will be handled by ParserCache. Next, we will re-route clients directly to MediaWiki and fully remove Parsoid from RESTBase (T344944 <https://phabricator.wikimedia.org/T344944>). The Page Content Service (PCS) will also handle its own cache <https://phabricator.wikimedia.org/T348995> and we are ready to test the new capabilities in staging soon. Many thanks to Yiannis Giannelos and the Content Transform team and to Daniel Kinzler for his efforts and support on decoupling Parsoid from RESTBase! All of these multi-year initiatives help us increase sustainability and maintainability of the platform, streamline engineering and developer workflows, and unlock the path for new and improved platform capabilities and product opportunities. Outlook: Knowledge Platform in the annual plan 2024/2025 The Wikimedia Foundation has recently published the draft objectives by the Product & Technology department <https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2024-2025/…> for the next annual plan on Meta, alongside an introduction by Selena Deckelmann and a few questions that we’re exploring <https://meta.wikimedia.org/wiki/Talk:Wikimedia_Foundation_Annual_Plan/2024-…>. Your input on these questions is very welcome! The draft objectives include “Knowledge Platform I” - centered around MediaWiki platform evolution and “Knowledge Platform II” - centered around developer/engineering services and workflows. The objectives show only the high-level direction for next year. The draft “key results” (currently work in progress) will give a better idea of what areas of work we’re thinking about. We’ll be publishing these in March and share the link + invitation for feedback with this list again. Thanks all for reading, Birgit -- Birgit Müller (she/her) Director of Product, MediaWiki and Developer Experiences Wikimedia Foundation <https://wikimediafoundation.org/>

2 1

[Train] 1.42.0-wmf.19 / wmf.20 status: blocked on T336504 (Transcluding Special:Prefixindex forces default skin), may require Monday rollback
by Brennen Bearnes 26 Feb '24

26 Feb '24

This week's 1.42.0-wmf.20 version of MediaWiki is blocked[0], and 1.42.0-wmf.19 may require a Monday rollback, which is a situation we hope to avoid. We can't proceed until the following issue is resolved: * T336504 - Transcluding Special:Prefixindex can force the default skin - https://phabricator.wikimedia.org/T336504 For context, see the Village pump discussion linked here: * https://phabricator.wikimedia.org/T336504#9572477 Once this issue is resolved, the train can proceed normally for the week. Thank you for any help! -- Your designated sacrificial train victims [0]. https://phabricator.wikimedia.org/T354438

1 0

Northward Datacentre Switchover (March 2024)
by effie mouzeli 26 Feb '24

26 Feb '24

Dear all, *"There is nothing permanent except change - Heraclitus"* On *Wednesday March 20th 2024,* the SRE team will run a planned datacentre switchover, moving all wikis from *codfw* to *eqiad*. This is an important periodic test of our tools and procedures, to ensure the wikis will continue to be available even in the event of major technical issues in our primary home. It also gives all our SRE and ops teams a chance to do maintenance and upgrades on systems in codfw that normally run 24 hours a day. The switchover process requires a brief read-only period for all Foundation-hosted wikis, which will start on *Wednesday March 20th 2024 @ 14:00 UTC*, and will last for just a few minutes while we execute the migration as efficiently as possible. All our public and private wikis will be continuously available for reading, as usual, but editing will be unavailable during the process. Users will see a notification of the upcoming maintenance, and anyone still editing will be asked to try again in a few minutes. CommRel will soon begin notifying communities of the read-only window. If you like, you can follow along on the day in the public *#wikimedia-operations* channel on IRC. To report any issues, you can reach us in *#wikimedia-sre* on IRC, or file a Phabricator ticket <https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=Data…>with the #datacenter-switchover tag; we'll be monitoring closely for reports of trouble during and after the switchover. The switchover and its preparation will be tracked under https://phabricator.wikimedia.org/T357547. On behalf of the SRE team, please excuse the disruption, and we would like to thank everyone in various departments who are involved in planning this work. If you have any questions, please reply directly to this email. Kind Regards, -effie

2 1

Etherpad maintenance 2024-02-26 09:00 - 10:00 UTC
by Jelto Wodstrcil 26 Feb '24

26 Feb '24

Hi all, we have to do *maintenance* on etherpad.wikimedia.org next *Monday, 2024-02-26** 09:00 UTC*. The maintenance involves the switch of the underlying hardware and a version upgrade. So the estimated downtime is around *60 minutes*. If you are working on a document currently please make a backup outside of Etherpad and do not use Etherpad during the maintenance window. I'll let you know when the maintenance is finished. You can find more information in https://phabricator.wikimedia.org/T316421. Greetings jelto

1 1

23 Feb '24

Hello all, The next language community meeting is coming up in a few weeks - *February 21st, 12:00 pm UTC*. If you're interested, you can sign up on this wiki page: < https://www.mediawiki.org/w/index.php?title=Wikimedia_Language_engineering/… >. This meeting is for individuals involved in creating content or managing technical aspects across different language communities. This will be a participant-driven meeting, where we collectively discuss specific technical issues related to language wikis and work together to find possible solutions. This could involve anything from fixing a broken template on the Kurdish wiki to brainstorming ideas for growing content on the Tulu Wiktionary, currently in the Wikimedia Incubator, or celebrating the creation of Fon Wikipedia, to using MinT for content translation. Feel free to add any *technical updates* related to your project or ideas for *problem-solving* discussion that you would like to share during the meeting to the notes document here: < https://etherpad.wikimedia.org/p/language-community-meeting-february-2024>. If you need interpretation support from English to another language, please let us know by sending an email to ssethi(a)wikimedia.org. Looking forward to your participation! Cheers, Jon, Mary & Srishti *Srishti Sethi* Senior Developer Advocate Wikimedia Foundation <https://wikimediafoundation.org/>

2 4

APIi page generator namespace order
by Bináris 22 Feb '24

22 Feb '24

Hi, I query special pages through API (concretely Special:UnconnectedPages), and I noticed that main namespace articles are yielded first. It seems that pages are ordered by namespace numbers. Is it a guaranteed behaviour? (If so, I can break from the loop at te first page not in 0 namespace which would be nice.) -- Bináris

2 2

Parsoid Read Views is enabled for DiscussionTools in Wikitech
by Mateus Santos 22 Feb '24

22 Feb '24

Hello everyone, I am glad to inform you that as part of our initial experimentations with Parsoid Read Views, Parsoid is turned on as the default read views renderer in Wikitech for DiscussionTools. Here's the rationale <https://www.mediawiki.org/wiki/Parsoid/Parser_Unification/Confidence_Framew…> behind our confidence to roll-out this experimental feature and collect feedback. This is an opportunity to experiment the changes confidently before we start rolling out more wikis. If this disrupts your workflow, don’t worry you can still opt out through a user preference or turn Parsoid off on the page you’re navigating in the Tools submenu. To report bugs and issues, please look at our known issues <https://www.mediawiki.org/wiki/Parsoid/Parser_Unification/Known_Issues> documentation and if you found a new bug please create a phab ticket and tag the Content Transform Team in Phabricator <https://phabricator.wikimedia.org/project/view/5846> or send us a message in our slack channel #content-transform-team. More relevant info about our roll-out strategy can be found on this wiki page <https://m.mediawiki.org/wiki/Parsoid/Parser_Unification/Confidence_Framework> . Best regards, Mateus

1 0

About problematic Chinese language variant support on mobile apps
by psnbaotg 22 Feb '24

22 Feb '24

Hi all, Today when playing around Wikipedia iOS app I'm surprised to find out that the Chinese language variant support is still somewhat broken. There is no zh-cn variant available, only zh-hans (displayed as "Simplified Chinese"). I haven't touched the Android app for a while, but in my last review it mistakenly includes zh-hans and zh-hant variants. As I mentioned in https://phabricator.wikimedia.org/T305319, the correct list of variants the app should include should be: zh-cn, zh-sg, zh-my, zh-tw, zh-mo, zh-hk. Other zh variants have incomplete conversion, thus should be excluded. I hope somebody at the relevant team could reach out to me and discuss possible solutions. If you don't understand the reason to do this I can give a detailed explanation. I believe this task should not be triaged as "Low" because it has negative impacts on Chinese readers. Best regards, Diskdance

2 5

Parsoid template transclusion behavior
by psnbaotg 18 Feb '24

18 Feb '24

Hello, Recently I'm researching Parsoid's design as MW is migrating to Parsoid. I found out that due to its single-pass tokenizing design, templates are not handled textually as the legacy parser does. This is good as the HTML now have information about which template they are transcluded from. However, https://www.mediawiki.org/wiki/Parsoid/limitations says "We have since decided to use the PHP preprocessor for template expansions, which side-steps these issues by reverting to the traditional textual preprocessor pass". Is this still true now? Best regards, Diskdance

2 1

OOUI v0.49.0 released
by atomasevich＠wikimedia.org 17 Feb '24

17 Feb '24

Hi all, OOUI v0.49.0 was released yesterday. It will be rolled out in MediaWiki via the deployment train starting on Tuesday, 13 February 2024. There are several breaking changes in this release. Please check your code carefully and make the suggested updates as necessary. - SearchInputWidget, TextInputWidget: getSaneType() has been changed to getValidType() to reflect more inclusive language. Please check your code to make sure you're using the updated method name. - SelectWidget: getItemMatcher() no longer supports the value of `true` for the mode argument, which has been deprecated since v0.30.4. Use `'exact'` instead. - WindowManager: The alias toggleAriaIsolation(), deprecated since v0.44.1, has been removed. Use toggleIsolation() instead. Other changes include: - JavaScript documentation has been migrated from JSDuck to JSDoc3. - The Less variable @line-height-base, deprecated since v0.42.0, has been removed. Use @line-height-label instead. You can find a full list of changes in the changelog[1]. As always, interactive demos[2] and code-level documentation[3] are available on doc.wikimedia.org, along with library documentation[4] on mediawiki.org. [1] - https://gerrit.wikimedia.org/g/oojs/ui/+/v0.49.0/History.md [2] - https://doc.wikimedia.org/oojs-ui/master/demos/#widgets-mediawiki-vector-ltr [3] - https://doc.wikimedia.org/oojs-ui/master/js [4] - https://www.mediawiki.org/wiki/OOUI Happy UI-ing,Anne -- Anne Tomasevich (she/her) Staff Software Engineer, Design System Wikimedia Foundation

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l February 2024