Hi All,
Welcome to the monthly MediaWiki Insights email!
First of all, many thanks and congratulations to the 5 new volunteer contributors: Ivi104, Matrix, obamwonyi, Ahonc and Abhii5599. Each of them got their first patch merged in MediaWiki core, extensions or services deployed in Wikimedia production during the month of September!
Updates to Wikimedia’s authentication system: Single User Login phase 3
Single User Login, the system which allows users to fill out the login form on one Wikimedia site and get logged in on all others at the same time, has been under increasing strain in the last few years as browsers have increasingly restricted the ways in which websites on different domains can exchange information, to prevent large-scale user tracking.
The MediaWiki Platform team has been working on updating our authentication system to be more compatible with these new browser requirements by moving all login and signup forms to a dedicated domain. The project, dubbed SUL3 (phase 1 was centralizing the user database in 2008, phase 2 was centralizing session cookies in 2013), is tracked in T348388 https://phabricator.wikimedia.org/T348388 and expected to finish later this year.
Besides being more in line with how browser vendors expect cross-domain authentication to work, these changes will also make Wikimedia sites more friendly to password managers, make it harder to steal passwords via hacked user scripts, and maybe pave the way before a wider rollout of WebAuthn, an advanced security feature which allows the use of physical devices such as USB tokens or fingerprint readers as second factors during login. The changes should be mostly invisible to users, other than the browser showing a different URL during login.
Internally, the way individual wikis interact with the central authentication system will become more standard, using the remote identity provider support built into MediaWiki’s authentication framework.
This work is part of our annual plan goal to improve the sustainability of the MediaWiki platform (WE5.1 https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2024-2025/Product_%26_Technology_OKRs#Key_Results). The initiative also required us to enable more people to work on the authentication stack (MediaWiki Core AuthManager, CentralAuth & Co) and will come with beneficial side effects such as improved test coverage and monitoring, which in return makes it easier to maintain the authentication infrastructure on the way forward. We will share more about this work in the coming weeks and months as we begin to prepare for the rollout to production wikis.
Simplify feature development in MediaWiki platform: Two reports published
In the last Insights email https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Reports/August_2024#Project_Snapshots:_Progress_on_annual_plan_initiatives we shared about our explorations to simplify feature development in the MediaWiki platform (WE5.2 https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Artifacts/KR_5.2:_Simplify_feature_development ). In the first quarter, we set out to better understand how developers are interacting with the platform from two perspectives: 1) Examine how core mechanisms support extensions and where those entry points can be improved (hook survey), and 2) Identify existing platform feature(s) that would benefit the most from directly redesigning the architecture to enable simpler and more sustainable feature development (notifications architecture).
- Hook survey (report https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Artifacts/KR_5.2:_Simplify_feature_development/5.2.1:Hook_Survey ): We analyzed 76 (out of 464) hook definitions and 165 (of 1408) hook handler implementations used by MediaWiki extensions deployed on Wikimedia sites. Based on the survey data we identified several opportunities for exploration and evolution. Our first experiment in this space will focus on introducing a new Domain Event and Listener pattern into the platform, to enable a simplified visibility into system state changes. We also believe this pattern will enable better flexibility for extension interfaces, as well as better boundaries between core components by removing the complexity of needing to know what other areas of code need to be informed about changes. More information can be found in the survey results summary https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Artifacts/KR_5.2:_Simplify_feature_development/5.2.1:Hook_Survey . - Notifications architecture (report https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Artifacts/KR_5.2:_Simplify_feature_development/Hypothesis_5.2.2:_Notifications_platform_in_core ): We identified the Notifications feature set as an opportunity to focus and scope the work to simplify feature development in the MediaWiki platform, because it contains several antipatterns and pain points that we believe can be addressed through improved modularity and defining simplified, use-case centric interfaces. The exploration contained a combination of high-level design work intended to define a unified notification system, paired with deep diving existing capabilities spread across the Echo extension and ENotif component in MediaWiki core. This resulted in a design proposal which we will continue to explore through experimentation and proof of concept work as a next step. Please see the report https://www.mediawiki.org/wiki/MediaWiki_Product_Insights/Artifacts/KR_5.2:_Simplify_feature_development/Hypothesis_5.2.2:_Notifications_platform_in_core for more information!
Project snapshots: Parsoid Read Views deployed to Wikivoyages, MathML rollout and Wikitech wiki major milestones met
The Parser Unification project has achieved a major milestone this past quarter: we have rolled out Parsoid Read Views https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/7UDJ52FG5JWMYJJCTJTYG7YMYC6XGNMZ/ to 22 of 26 Wikivoyages, including Mobile Frontend, which accounts for roughly 77% of monthly page views. The remainder of the wikis have documented challenges that we plan to tackle soon. You can learn more about our rollout strategy and how deployment readiness is determined on this page https://www.mediawiki.org/wiki/Parsoid/Parser_Unification/Confidence_Framework/Reports#Deployment_Readiness_for_Wikivoyages .
MathML rollout (part of Mathoid deprecation, which is a dependency for RESTBase sunsetting): The Math extension provides support for math formula in page content, and deployed on all Wikipedias and bundled with MediaWiki core by default. The new MathML mode embraces the native MathML support in web browsers today, and removes dependency on an unmaintained Node.js service. It also resolves a longstanding problem whereby Math rendering in MediaWiki involved a live dependency on the production wikimedia.org RESTBase service, which made it awkward to support MediaWiki LTS. Once MathML is stable and enabled on more Wikipedias, we plan to backport the new MathML mode to MediaWiki 1.39, which in turn removes the third-party RESTBase dependency. This work is tracked under https://phabricator.wikimedia.org/T271001 and carried out by volunteers Moritz (Physikerwelt), Johannes (Stegmujo), and AndreG-P, supported with review and guidance by Timo (Krinkle). In September we completed two items:
1. Enable the new MathML mode on group0 wikis. 2. Disable Mathoid by default in the Math extension (for MediaWiki 1.43) so that new third-party installs don't start with a dependency on a deprecated service.
Wikitech updates: Until recently, Wikitech wiki https://wikitech.wikimedia.org/wiki/Main_Page offered unique functionalities, posing interesting challenges during maintenance or upgrades. On October 1st, we successfully migrated it to Kubernetes, simplifying maintenance and administration. This achievement wouldn’t have been possible without the collaborating work of various teams and individuals to decouple developer account management https://wikitech.wikimedia.org/wiki/IDM from Wikitech, and moving authentication to Wikimedia Unified Login (aka SUL), which means: Wikitech now supports SUL authentication. We are currently in the process of linking existing Wikitech accounts https://wikitech.wikimedia.org/wiki/Wikitech/SUL-migration to their respective SUL accounts to eventually (end of November 2024) offer a unified user experience. This marks the end of a long road separating developer accounts from Wikitech, and deprecating the LDAP backend in the process. Originally, we aimed to complete the migration by mid-September. However, October 1st proved to be a more feasible target, allowing us enough time to align the efforts across teams. Kudos to: Alexandros, Amir, Effie, Joanna, Moritz, Simon, and Taavi!
OpenTelemetry: Jaeger can store and visualize distributed tracing https://wikitech.wikimedia.org/wiki/Distributed_tracing information for a high-level overview of calls between different services (T320549 https://phabricator.wikimedia.org/T320549). This complements the detailed and low-level function profiling we already have for MediaWiki, such as Excimer flame graphs https://techblog.wikimedia.org/2023/06/08/flame-graphs-arrive-in-wikimediadebug/, and XHGui https://wikitech.wikimedia.org/wiki/WikimediaDebug#XHGui_profiling. Today, Jaeger focuses on high-level metadata from services around MediaWiki. This quarter, SRE and MediaWiki Engineering are expanding Jaeger to include some tracing from MediaWiki itself. Last month, Maté finished development of a lightweight and sustainable OpenTelemetry client https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1061573 for MediaWiki. Implementation was guided by Timo and Piotr. Next month we will use this client to instrument parts of MediaWiki (T340552 https://phabricator.wikimedia.org/T340552).
Outlook: Improving API documentation sustainability
We are kicking off work on improving API documentation sustainability https://phabricator.wikimedia.org/project/profile/7456/. This intervention will introduce three main benefits: 1) automatically generate OpenAPI specifications for all of the MediaWiki endpoints so that we can adopt industry standard tooling, 2) leverage the generated specs for automated endpoint validation to ensure documentation consistency and prevent accidental breaking changes, and 3) automatically translate the generated specs so that our documentation is more accessible for the global developer community. By adopting automatically generated API documentation using standard, open source OpenAPI specifications https://swagger.io/specification/, we can improve the sustainability for API producers while increasing stability for API consumers. This approach reduces reliance on manual maintenance for API documentation pages and ensures that the documented API responses always match the actual behavior of the underlying endpoints by unlocking automated validation and error handling. This will prevent accidental breaking changes from entering the Wikimedia developer ecosystem, improving platform stability and reducing disruption for API users. Additionally, the generated specs can be used to automatically enable interactive experiences using SwaggerUI https://swagger.io/tools/swagger-ui/, where API users will be able to test the endpoints inline with the documentation, all without additional overhead from the Wikimedia teams creating and managing the APIs.
Reminder: MediaWiki 1.43 LTS release
The next MediaWiki Release is coming https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/XD2LSFZYXUPGTGL7OYVWPVPV5VPQYN6H/, and it will be an LTS. Version 1.43 will branch for alpha on October 22nd and contain essential changes you should look for, like native MathML in favor of Mathoid deprecation and drop of support to PHP versions < 8.1. If you consider your task a relevant blocker for the next MW Release, please tag it on our board https://phabricator.wikimedia.org/tag/mw-1.43-release/ for the team’s evaluation.
With this, we’re wrapping up this edition!
Thanks all for reading,
Birgit