Wikitech-l December 2021

wikitech-l@lists.wikimedia.org

28 participants
39 discussions

[Train] 🚂🔥1.38.0-wmf.13 blocked
by Tyler Cipriani 16 Dec '21

16 Dec '21

Please don't scroll past this. This Wednesday, for the 1st time recently, we humbly ask you to defend Wikipedia's train.[0] The 1.38.0-wmf.13 <https://phabricator.wikimedia.org/T293954> train is blocked. The new version is rolled back to group0 <https://versions.toolforge.org/> until we resolve: - Error: Call to a member function getId() on null <https://phabricator.wikimedia.org/T297827> - tl;dr: users can't login - Already being worked on - PHP Notice: Array to string conversion <https://phabricator.wikimedia.org/T297828> - Impact unclear, maybe not a blocker Once these issues are resolved train can resume. If these issues are resolved on a Friday the train will resume Monday. Thank you for your help resolving these issues! – The Management Tyler Cipriani (he/him) Engineering Manager, Release Engineering Wikimedia Foundation

2 3

Regarding the open source contributions
by Shivam Purohit 16 Dec '21

16 Dec '21

2 1

Invite to explore code ownership in MediaWiki core
by Jon Robson 16 Dec '21

16 Dec '21

The most recent developer satisfaction survey supports the claim that volunteers find it hard to get code reviews. The hypothesis is that by having clearer ownership around code, there should be clearer accountability, and it should be clearer what code lacks ownership that probably should. This also applies to Wikimedia staff. It's often not clear when needing to make changes to core, who is the best person to talk to about that code, and who should be responsible/accountable/consulted/informed about any changes to the code. I am involved in the development of the Desktop improvements project [1] and when we began the work there was no recognized owner for a lot of the skin code in core. Since then my team has taken the role of maintainer and it has allowed us to drive a lot of changes. I'm seeing new skins pop up and we have started hearing from skin engineers, so I feel this has been a positive thing. However, as we've navigated other parts of the codebase, we've had to ask a lot of questions such as "Who maintains the legacy parser? Are they open to changes there, or should that code be considered frozen?"; "Who should we talk to about changes relating to the JavaScript of the watchstar?" What with the proposed move to Gitlab, I have been very interested in the code owners file documented on their site [2] I hypothesize that identifying code owners and getting Wikimedia engineering to agree to some basic expectations around code ownership might go a long way to helping set expectations around who to talk to about making changes, and actually getting those changes carried to completion. The experiment of adding an OWNERS file to MediaWiki core seems like a cheap one, given if we find it does nothing the file can easily be removed. If you are interested in helping me explore this, feel free to join the conversation on https://gerrit.wikimedia.org/r/c/mediawiki/core/+/745946 Wishing everyone a happy holiday Jon [1] mediawiki.org/wiki/Desktop_improvements [2] https://docs.gitlab.com/ee/user/project/code_owners.html

1 0

Security and maintenance release: 1.35.5 / 1.36.3 / 1.37.1
by Sam Reed 15 Dec '21

15 Dec '21

2 2

Production Excellence #38: November 2021
by Krinkle 14 Dec '21

14 Dec '21

How’d we do in our strive for operational excellence last month? Read on to find out! Incidents 6 documented incidents last month. That's above the two-year and five-year median of 4 per month (per Incident graphs <https://codepen.io/Krinkle/full/wbYMZK>). 2021-11-04 large file upload timeouts <https://wikitech.wikimedia.org/wiki/Incident_documentation/2021-11-04_large…>; Impact: For 9 months, editors were unable to upload large files (e.g. to Commons). Editors would receive generic error messages, typically after a timeout. In retrospect, a dozen different distinct production errors had been reported and regularly observed that were related and provided different clues, however most of these remained untriaged and uninvestigated for months. This may be related to the affected components having no active code steward. 2021-11-05 TOC language converter <https://wikitech.wikimedia.org/wiki/Incident_documentation/2021-11-05_TOC_l…>; Impact: For 6 hours, wikis experienced a blank or missing table of contents on many pages. For up to 3 days prior, wikis that have multiple language variants (such as Chinese Wikipedia) displayed the table of contents in an incorrect or inconsistent language variant (which are not understandable to some readers). 2021-11-10 cirrussearch commonsfile outage <https://wikitech.wikimedia.org/wiki/Incident_documentation/2021-11-10_cirru…>; Impact: For ~2.5 hours, the Search results page was unavailable on many wikis (except English Wikipedia). On Wikimedia Commons the search suggestions feature was unresponsive as well. 2021-11-18 codfw ipv6 network <https://wikitech.wikimedia.org/wiki/Incident_documentation/2021-11-18_codfw…>; Impact: For 8 minutes, the Codfw cluster experienced partial loss of IPv6 connectivity for upload.wikimedia.org. This did not affect availability of the service because the "Happy Eyeballs <https://en.wikipedia.org/wiki/Happy_Eyeballs>" algorithm ensures browsers (and other clients) automatically fallback to IPv4. The Codfw cluster generally serves Mexico and parts of the US and Canada. The upload.wikimedia.org service serves photos and other media/document files, such as displayed in Wikipedia articles. 2021-11-23 core network routing <https://wikitech.wikimedia.org/wiki/Incident_documentation/2021-11-23_Core_…>; Impact: For about 12 minutes, Eqiad was unable to reach hosts in other data centers via public IP addresses. This was due to a BGP routing error. There was no impact on end-user traffic, and impact on internal traffic was limited (only Icinga alerts themselves) because internal traffic generally uses local IP subnets which we currently route with OSPF instead of BGP. 2021-11-25 eventgate-main outage <https://wikitech.wikimedia.org/wiki/Incident_documentation/2021-11-25_event…>; Impact: For about 3 minutes, eventgate-main was down. This resulted in 25,000 MediaWiki backend errors due to inability to queue new jobs. About 1000 user-facing web requests failed (HTTP 500 Error). Event production briefly dropped from ~3000 per second to 0 per second. Incident follow-up Remember to review and schedule Incident Follow-up work <https://phabricator.wikimedia.org/project/view/4758/> in Phabricator, which are preventive measures and tech debt mitigations written down after an incident is concluded. Read more about past incidents at Incident status <https://wikitech.wikimedia.org/wiki/Incident_status> on Wikitech. Recently resolved incident follow-up: Disable DPL on wikis that aren't using it <https://phabricator.wikimedia.org/T287916> Filed after a July 2021 incident, done by Amir (Ladsgroup) and Kunal (Legoktm). Create easy access to MySQL ports for faster incident response and maintenance <https://phabricator.wikimedia.org/T291352> Filed in Sep 2021, and carried out by Stevie (Kormat). Create paging alert for primary DB hosts <https://phabricator.wikimedia.org/T233684> Filed after a Sept 2019 incident, done by Stevie (Kormat). Trends November saw 27 new production error reports of which 14 were resolved, and 13 remain open and carry over to the next month. Of the 301 errors still open from previous months, 16 were resolved. Together with the 13 carried over from November that brings the workboard to 298 unresolved tasks. Figure 1: Unresolved error reports by month <https://phabricator.wikimedia.org/phame/post/view/261/production_excellence…>. Outstanding errors Take a look at the workboard and look for tasks that could use your help. → https://phabricator.wikimedia.org/tag/wikimedia-production-error/ 💡 Did you know: *To find your team's error reports, use the appropriate ***"Filter" link in the sidebar of the workboard***.* Issues carried over from recent months: Apr 2021: 9 of 42 issues left. May 2021: 16 of 54 issues left. Jun 2021: 9 of 26 issues left. Jul 2021: 11 of 31 issues left. Aug 2021: 10 of 46 issues left. Sep 2021: 10 of 24 issues left. Oct 2021: 20 of 49 issues left. Nov 2021: 13 of 27 new issues <https://phabricator.wikimedia.org/maniphest/query/0W0Nuk9umBDc/#R> are carried forward. Thanks! Thank you to everyone who helped by reporting, investigating, or resolving problems in Wikimedia production. Thanks! Until next time, – Timo Tijhof 🔗 Share or read later via https://phabricator.wikimedia.org/phame/post/view/261/

2 1

Security pre-release announcement: 1.35.5 / 1.36.3 / 1.37.1
by Sam Reed 13 Dec '21

13 Dec '21

Hi all, On Wednesday we will be issuing a security and maintenance release to all supported branches of MediaWiki. The new releases will be: - 1.35.5 - 1.36.3 - 1.37.1 This release includes fixes for multiple high severity authorization bypasses in MediaWiki core, it is recommended you patch immediately. A short LocalSettings.php configuration snippet will also be shared to disable the vulnerable functionality if you are unable to patch right away. This snippet should work across all vulnerable MediaWiki versions, including end-of-life ones. In addition to that, this will resolve other issues in MediaWiki core and also includes some fixes previously committed to git, including minor security and hardening patches along with bug fixes included for maintenance reasons. It also fixes 2 issues in MediaWiki tarball bundled extensions. We will make the fixes available in these respective release branches and master. Tarballs will be available for the above mentioned point releases as well. A summary of some of the security fixes that have gone into non-bundled MediaWiki extensions will also follow later.

1 0

🚂🌈Summary of 1.38.0-wmf.12 train deployment
by Ahmon Dancy 10 Dec '21

10 Dec '21

- 1.38.0-wmf.12 has been rolled out to all wikis. There are a couple of low frequency errors that are being logged bug they are being worked on. - Conductor: Ahmon Dancy (yours truly) - Backup Conductor: Brennen Bearnes - Blocker Task: T293953 <https://phabricator.wikimedia.org/T293953> - Current Status <https://versions.toolforge.org/> 📊 By the Numbers Sparklines comparing with the last 5 trains. - 231 Patches █▅▁▅▂ - 1 Rollbacks ▁▃█▆▃ - 0 Days of delay ▁▂▆█▁ - 8 Blockers ▃▃▁█▇ ✨ Trainbow Love 🎉Thanks to folks who reported or resolved blockers: - Amir Sarabadani (WMF) - Timo Tijhof - Taavi Väänänen - EBernhardson - Daniel Kinzler - Roan Kattouw - Jon Robson - Zabe - and many more! -

1 0

Looking for new maintainer for packagist-mirror.wmcloud.org
by Kunal Mehta 10 Dec '21

10 Dec '21

Hi, <https://packagist-mirror.wmcloud.org/> is exactly what it sounds like, a mirror of the metadata on packagist.org. It's very simple, just a systemd timer and some Apache config. Since setting it up 2 or 3 years ago I've barely had to touch it. And based on access logs, some people are actually using it (though we never switched CI over to it, as the idea/goal once was). But after being reminded about it by the current Cloud VPS cleanup, I'm not interested in running this anymore. packagist.org is actually open source (*looks at npmjs.com in sadness*) and hasn't had any reliability issues recently AFAIR. There's also a bit of work to do, the VM needs to be upgraded from stretch, and it should be switched over to the new composer/mirror script from upstream. So if someone would like to take it over, comment on <https://phabricator.wikimedia.org/T296968> and I'm happy to add you and explain how it works. Otherwise please consider this notice that I intend to shut it down at the end of the year. -- Legoktm

1 0

Making $wgLegalTitleChars easier to use outside PHP
by Kunal Mehta 10 Dec '21

10 Dec '21

Hi, Myself and Erutuon are working on writing a Rust library[1] that validates and normalizes MediaWiki page titles, initially for use in bots and tools, but it has other potential use cases too. There is some prior art for this, a mediawiki-title npm package[2] is used by various nodejs services, and there's a mediawiki.Title ResourceLoader module in core too, all of which reimplement MediaWiki's title parsing, validation and normalization routines...basically MediaWikiTitleCodec and what it calls. One problem area is $wgLegalTitleChars. It's part of a PHP regex that gets used to check if any invalid characters are present by inverting it, basically [^{legalchars}]. It's also exposed via the meta=siteinfo API, which suggests that it's useful for external callers, but in its current form it's really not. The current value is: $wgLegalTitleChars = " %!\"$&'()*,\\-.\\/0-9:;=?@A-Z\\\\^_`a-z~\\x80-\\xFF+"; It's is a mix of literal characters, some escaped and some not, and various ranges. Specifically, it's tuned for PHP regexes, like / is escaped because it delimits regexes (an unnecessary escape and therefore problematic in Rust), and it uses syntax like \x80-\xFF, while in JavaScript we want \u0080-\uFFFF. We'd also like to use the Unicode escape class syntax in Rust, but for now have worked around it to use the byte class. In fact, we have functions that parse the byte class and turn it into a unicode class, Title::convertByteClassToUnicodeClass() in PHP and a JavaScript version too[3]. This seems entirely unnecessary to me, given that we could just write this sequence in a more regex-neutral manner to make it more portable: $wgLegalTitleCharacters = [ 'characters' => " %!\"$&'()*,-./:;=?@\\\\^_`~", // + 'plus' => true, // A-Z, a-z, 0-9 'alphanumeric' => true, // \x80-\xFF 'non-ascii' => true, ]; Characters are literal characters to run through preg_quote or mw.util.escapeRegExp, and the ranges are to be specified in whatever format the specific regex engine would like. But this also opens up the question - is there a valid use case for customizing $wgLegalTitleChars anymore? We already have a comment that says "Don't change this unless you know what you're doing". There's also one that says "In some rare cases you may wish to remove + for compatibility with old links." - is that still a consideration today? I would think that for easy importing/exporting across various MediaWiki wikis we want the set of legal title characters to be rather static. And if someone wants to ban some character from being used, Extension:TitleBlacklist (bundled) provides a much less invasive way to do so. So to recap: 1. Can we get rid of the ability to customize legal title characters? 2. If #1 is no, any objections to the breaking change of swapping out $wgLegalTitleChars (string) for $wgLegalTitleCharacters (array)? Note that extensions can still read from the old global, just it can't be overridden in LocalSettings anymore. The actual patch to review is <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/745386>. [1] https://gitlab.com/mwbot-rs/mwbot/-/tree/master/mwtitle [2] https://github.com/wikimedia/mediawiki-title/ [3] https://github.com/wikimedia/mediawiki-title/blob/master/lib/utils.js

1 0

MediaWiki Language Extension Bundle 2021.12 (Bug fix release)
by Kartik Mistry 03 Dec '21

03 Dec '21

Hello all, I would like to announce the release of MediaWiki Language Extension Bundle 2021.12. This is a bug-fix release. Previous release was [ https://lists.wikimedia.org/hyperkitty/list/mediawiki-i18n@lists.wikimedia.… MLEB 2021.11]. This bundle is compatible with '''MediaWiki 1.35.3''' or above and requires '''PHP 7.3.19''' or above. Next MLEB is expected to be released next month. If there are very important bug fixes, we will do an intermediate release. Please give us your feedback at [[Talk:MLEB|https://www.mediawiki.org/wiki/Talk:MLEB]]. * Download: https://translatewiki.net/mleb/MediaWikiLanguageExtensionBundle-2021.11.tar… * sha256sum: 61c9c7a700c9f98c6b3c59adf2f0fc7d1919fc87e0c3584308d313536dd81f37 * Signature: https://translatewiki.net/mleb/MediaWikiLanguageExtensionBundle-2021.11.tar… Quick links: * Installation instructions are at: https://www.mediawiki.org/wiki/MLEB * Announcements of new releases will be posted to a mailing list: https://lists.wikimedia.org/mailman/listinfo/mediawiki-i18n * Report bugs to: https://phabricator.wikimedia.org/project/view/1464/ Release notes for each extension are below. -- Kartik Mistry == Babel, cldr, LocalisationUpdate, CleanChanges and UniversalLanguageSelector == * No changes. == Translate == * Fix broken SpecialMovePage on MediaWiki 1.35 ([[phab:T294605|Phab:T296450]]) -- Kartik Mistry | કાર્તિક મિસ્ત્રી kartikm.wordpress.com

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l December 2021