Wikitech-l July 2023

wikitech-l@lists.wikimedia.org

23 participants
31 discussions

Codex now available on Toolforge CDNjs
by Lucas Werkmeister 19 Jul '23

19 Jul '23

Hi all, Tool developers may be interested to know that Codex [1], the design system for Wikimedia, is now available on the Toolforge CDNjs Mirror [2] (and on upstream CDNjs [3], but you should prefer the Toolforge mirror :P). This means you can relatively easily use the Codex components, icons, and/or design tokens in your tools if you want, just like e.g. Bootstrap. See [4] for an example in one of my tools. (The most important technical bits are the importmap, import statements, createApp call, and `components: codex` and `...codexIcons`, if you want to search for those parts in particular.) The Codex maintainers would probably like me to remind you that Codex has not reached version 1.0 yet ;) I gather that it’s coming up (and not, like, years away), but until then you should probably be ready for breaking changes to happen. Once version 1.0 is out, CDNjs and the Toolforge mirror should pick it up automatically, but you’ll need to adjust the URLs in your tools like for any version upgrade (since the version number is part of the URL – CDNjs doesn’t offer auto-updating “latest” URLs). The Phabricator task for Codex being available on CDNjs is T338834 [5]; if you experience any issues, you can bring them up there. Enjoy! Lucas [1]: https://doc.wikimedia.org/codex/latest/ [2]: https://cdnjs.toolforge.org/ [3]: https://cdnjs.com/ [4]: https://gitlab.wikimedia.org/toolforge-repos/wd-image-positions/-/commit/0e… [5]: https://phabricator.wikimedia.org/T338834

1 0

Testing on MW-on-K8s during deployments
by Clément Goubert 13 Jul '23

13 Jul '23

Hi y'all, Now that we are up to 1% of global traffic <https://phabricator.wikimedia.org/T341463> and soon to be 5% <https://phabricator.wikimedia.org/T341780> on our Mediawiki-on-Kubernetes infrastructure <https://wikitech.wikimedia.org/wiki/MediaWiki_On_Kubernetes>, we would like to ask deployers and developers to test MediaWiki patches on it as well as the usual mwdebug servers. To that end, scap, at the testserver stage, also synchronizes the mw-debug MW-on-K8s deployment. You can reach that deployment by either: - setting your WikimediaDebug extension's dropdown menu to k8s-experimental - setting the X-Wikimedia-Debug header with backend=k8s-experimental. It would help us tremendously in diagnosing potential issues early. You can reach out to us with any question or issues, either via phabricator with the #serviceops and #MW-on-K8s tags, or on IRC in the #wikimedia-mw-on-k8s channel. Thanks in advance from the ServiceOps team, and the whole MW-on-K8s group -- Clément Goubert (they/them) Senior SRE Wikimedia Foundation

1 0

New `composer phpunit:entrypoint` command (phpunit.php deprecated)
by Daimona 12 Jul '23

12 Jul '23

TL;DR: The tests/phpunit/phpunit.php script is deprecated. Use `composer phpunit:entrypoint` instead. Thus far there have been three ways to run PHPUnit tests in MediaWiki: 1. Via simple composer command: `composer phpunit` / `vendor/bin/phpunit` (only for non-DB tests) 2. Via composer entrypoint: `composer phpunit:entrypoint` 3. Via custom PHP script: `php tests/phpunit/phpunit.php` The third method is now deprecated [1]. The `composer phpunit:entrypoint` command, previously pointing to the phpunit.php script, now uses `vendor/bin/phpunit`, specifying a config file that can be used with integration and database tests. Where you would previously use phpunit.php, you should now use `composer phpunit:entrypoint`. The rationale behind this change is to make it easier for developers to run PHPUnit tests, without having to determine what entry point they should use for a given test. The previous setup also made it more difficult to configure IDEs, because you had to switch between the available entry points to make sure that tests were run with the expected config. After this change, the only difference between `composer phpunit` and `composer phpunit:entrypoint` is the config file they use (phpunit.xml.dist and tests/phpunit/suite.xml, respectively). Note that we are also planning to phase out the latter; you can follow the conversation in [2]. When that happens, we will truly have a single entry point for all PHPUnit tests! The switch from phpunit.php to composer comes with a small but potentially significant difference: LocalSettings.php is no longer loaded in the global scope. This should work transparently to you. If one of your settings appear to have no effect, please report this on Phabricator [1]. The main scenario where these issues may arise is if your configuration variable is read early during MediaWiki setup, for instance in an extension function [3]. You can temporarily workaround this by making the config variable explicitly global in the place where it's being set, with `global $wgMyGlobal;`. Huge thanks to Timo, James F., Kosta, Antoine, Umherirrender, and everyone who helped with this long-overdue change. ---- [1] - https://phabricator.wikimedia.org/T90875 [2] - https://phabricator.wikimedia.org/T227900 [3] - https://www.mediawiki.org/wiki/Manual:$wgExtensionFunctions -- https://meta.wikimedia.org/wiki/User:Daimona_Eaytoy "Daimona" is not my real name -- he/him

1 0

contint Jenkins maintenance Tue, 11 Jul 2023 08:00-9:30 UTC
by Jelto Wodstrcil 11 Jul '23

11 Jul '23

Hi all, We will perform a short maintenance on contint hosts tomorrow 11 Jul 2023 08:00-9:30 UTC. We will switchover from the old host contin2001 to the new host contint2002 due to a hardware refresh. The machines host integration.wikimedia.org, an instance of jenkins and zuul. Services have to be stopped and restarted, so expect some unavailability. Alias/cname contint.wikimedia.org also maps to these machines. If you have any questions, reach out to me or @Antoine Musso <amusso(a)wikimedia.org> in https://phabricator.wikimedia.org/T324659. Greetings Jelto

2 2

Technical Community Newsletter: July 2023 Edition
by Onyinyechi Onifade 11 Jul '23

11 Jul '23

Hi everyone, The July 2023 edition of the Technical Community Newsletter is available: https://www.mediawiki.org/wiki/Technical_Community_Newsletter/2023/July <https://www.mediawiki.org/wiki/Technical_Community_Newsletter/2023/July> The newsletter is compiled by staff working on developer outreach. It aims to share highlights, news, and information of interest from and about the Wikimedia technical community. The Wikimedia Technical Community is large and diverse - it's hard to capture everything. We would love to hear your ideas for future newsletters. Got something you would like to see or something you want to highlight in the next quarterly newsletter? Please add your suggestion to the talk page: https://www.mediawiki.org/wiki/Talk:Technical_Community_Newsletter If you'd like to keep up with updates and information, subscribe to the Technical Community Newsletter: https://www.mediawiki.org/wiki/Newsletter:Technical_Community_Newsletter Thanks, -- *Onyinyechi Onifade * Technical Community Program Manager Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

Sending 1% of global traffic to mw-on-k8s
by Clément Goubert 10 Jul '23

10 Jul '23

Hi y'all, I'm about to start sending 1% of global traffic to mw-on-k8s <https://phabricator.wikimedia.org/T341463> On-call has been made aware (it's me and slyngs), emergency rollback is: 1. Revert https://gerrit.wikimedia.org/r/c/operations/puppet/+/936697 2. sudo cumin -b 20 'A:cp-text and P{P:trafficserver::backend}' 'run-puppet-agent -q' If you want to see what happens, you can check out the mw-on-k8s dashboard <https://grafana.wikimedia.org/d/U7JT--knk/mw-on-k8s> This will impact the mw-web and mw-api-ext deployments. Cheers, -- Clément Goubert (they/them) Senior SRE Wikimedia Foundation

1 0

Service Decommission Notice: Mobile Content Service - July 2023
by Adam Baso 06 Jul '23

06 Jul '23

TL;DR: The legacy Mobile Content Service is going away in July 2023. Please switch to Parsoid or another API before then to ensure service continuity. Hello World, I'm writing about a service decommission we hope to complete mid-July 2023. The service to be decommissioned is the legacy Mobile Content Service ("MCS"), which is maintained by the Wikimedia Foundation's Content Transform Team. We will be marking this service as deprecated soon. We hope that with this notice, people will have ample time to update their systems for use of other endpoints such as Parsoid [1] (n.b., MCS uses Parsoid HTML). The MCS endpoints are the ones with the relative URL path pattern /page/mobile-sections* on the Wikipedias. For examples of the URLs see the "Mobile" section on the online Swagger (OpenAPI) specification documentation with matching URLs here: https://en.wikipedia.org/api/rest_v1/#/Mobile == History == The Mobile Content Service ("MCS") is the historical aggregate service that originally provided support for the article reading experience on the Wikipedia for Android native app, as well as some other experiences. We have noticed that there are other users of the service. We are not able to determine all of the users, as it's hard to tell with confidence from the web logs. The Wikimedia Foundation had already transitioned the Wikipedia for Android and iOS apps to the newer Page Content Service ("PCS") several years ago. PCS has some similarities with MCS in terms of its mobility focus, but it also has different request-response signatures in practice. PCS, as with MCS, is intended to primarily satisfy Wikimedia Foundation-maintained user experiences only, and so this is classified with the "unstable" moniker. == Looking ahead == Generally, as noted in the lead, we recommend that folks who use MCS (or PCS, for that matter) switch over to Parsoid for accessing Wikipedia article content programmatically for the most predictable service. The HTML produced by Parsoid has a versioned specification [2] and because Parsoid is accessed regularly by a number of components across the globe tends to have fairly well cached responses. However, please note that Parsoid may be subject to stricter rate limits that can apply under certain traffic patterns. At this point, I do also want to note that in order to keep up with contemporary HTML standards, particularly those favoring accessibility and machine readability enhancements, Parsoid HTML will undergo change as we further converge parsing stacks [3]. Generally, you should expect iteration on the Parsoid HTML spec, and of course as you may have come to appreciate that the shape of HTML in practice can vary nontrivially wiki-by-wiki as practices across wikis vary. You may also want to consider Wikimedia Enterprise API options, which range from no cost to higher volume access paid options. https://meta.wikimedia.org/wiki/Wikimedia_Enterprise#Access == Forking okay, but not recommended == Because MCS acts as a service aggregate and makes multiple backend API calls, caveats can apply for those subresources - possibility of API changes, deprecation, and the like. We do not recommend a plain fork of MCS code because of the subresource fetch behavior. This said, of course you are welcome to fork in a way compatible with MCS's license. == Help spread the word == Although we are aware of the top two remaining consumers of MCS, we also are not sure who else is accessing MCS and anticipate that some downstream tech may break when MCS is turned off. As we are cross-posting this message, we hope most people who have come to rely upon MCS will see this message. Please feel free to forward this message to contacts if you know they are using MCS. == Help == Although we intend to decommission MCS in July 2023, we would like to share resources if you need some help. We plan to hold office hours in case you would like to meet with us to discuss this or other Content Transform Team matters. We will host these events on Google Meet. We will provide notice of these office hours on the wikitech-l mailing list in the coming weeks and months. Additionally, if you would like to discuss your MCS transition plans, please visit the Content Transform Team talk page: https://www.mediawiki.org/wiki/Talk:Content_Transform_Team Finally, some Content Transform Team members will also be at the Wikimedia Hackathon [4] if you would like some in-person support. Thank you. Adam Baso (he/him/his/Adam), on behalf of the Content Transform Team Director of Engineering Wikimedia Foundation [1] https://www.mediawiki.org/wiki/Parsoid [2] https://www.mediawiki.org/wiki/Specs/HTML [3] https://www.mediawiki.org/wiki/Parsoid/Parser_Unification/Updates [4] https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2023

3 7

[User pageviews dumps per wiki] From 2022-01 to 2022-06
by Dušan Kreheľ 06 Jul '23

06 Jul '23

Hi. The user pageviews dumps per local wikis are published: https://archive.org/details/2023-daily_user_pageviews Statistic (suma): - Files: 952 - Size: 13GB (13096923579B) - Uncompressed size: ~53GB (52988029285B) Dušan Kreheľ

1 0

GSoC Contribution 2024
by Sehaj Singla 05 Jul '23

05 Jul '23

Dear Sir/ Ma’am, I am Sehaj, a CSE student from Chandigarh University. I just entered my second year at university. I wish to make contributions and work with your organisation in 2024 for GSoC. I know Javascript, React, HTML, CSS, SQL, MongoDB. Kindly guide me further. Thank you for your time.

2 1

MediaWiki Extensions and Skins Security Release Supplement (1.35.11/1.38.7/1.39.4/1.40.0)
by Maryum Styles 03 Jul '23

03 Jul '23

Greetings- With the security/maintenance release of MediaWiki 1.35.11/1.38.7/1.39.4/1.40.0, we would also like to provide this supplementary announcement of MediaWiki extensions and skins with now-public Phabricator tasks, security patches and backports [1]: CheckUser + (T333569, CVE-2023-37255) - Special:CheckUser 'get edits' is vulnerable to HTML injection through user agent string. https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CheckUser/+/905706/ GoogleAnalyticsMetrics + (T333980, CVE-2023-37251) - GoogleAnalyticsMetrics parser function in extension does not properly escape js in onclick handler and does not prevent using javascript urls. https://gerrit.wikimedia.org/r/c/905661 CheckUser + (T330968, CVE-2023-37252) - Special:CheckUserLog shows usernames which have been hidden. https://gerrit.wikimedia.org/r/c/933686 https://gerrit.wikimedia.org/r/c/932822 Cargo + (T331311, CVE-2023-37256) - Cargo allows storing javascript URLs in URL fields, and automatically linking them. https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cargo/+/894679 Cargo + (T331065, CVE-2023-37254) - XSS in Special:CargoQuery using default format. https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cargo/+/894666 ProofreadPage + (T326952, CVE-2023-37253) - ProofreadPage leaks suppressed user via the API and config variables. https://gerrit.wikimedia.org/r/q/Ibe5f8e25dea155bbd811a65833394c0d4b906a34 DoubleWiki + (T323651, CVE-2023-37304) - XSS in DoubleWiki extension (Wikisource). https://gerrit.wikimedia.org/r/c/933666 https://gerrit.wikimedia.org/r/c/933667 https://gerrit.wikimedia.org/r/c/932825 CheckUser + (T338276, CVE-2023-37303) - Wikimedia\Rdbms\DBQueryDisconnectedError when blocking user. https://gerrit.wikimedia.org/r/c/932823 Wikibase + (T250720, CVE-2023-37301) - Wikidata edit filter does not fire when test tool says it should. https://gerrit.wikimedia.org/r/c/933663 Wikibase + (T339111, CVE-2023-37302) - Style injection into badges on Wikidata due to unescaped quotes. https://gerrit.wikimedia.org/r/c/933649 https://gerrit.wikimedia.org/r/c/933650 The Wikimedia Security Team recommends updating these extensions and/or skins to the current master branch or relevant, supported release branch [2] as soon as possible. Some of the referenced Phabricator tasks above _may_ still be private. Unfortunately, when security issues are reported, sometimes sensitive information is exposed and since Phabricator is historical, we cannot make these tasks public without exposing this sensitive information. If you have any additional questions or concerns regarding this update, please feel free to contact security(a)wikimedia.org or file a security task within Phabricator [3]. [1] https://phabricator.wikimedia.org/T333626 [2] https://www.mediawiki.org/wiki/Version_lifecycle [3] https://www.mediawiki.org/wiki/Reporting_security_bugs

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l July 2023