Wikitech-l September 2020

wikitech-l@lists.wikimedia.org

61 participants
58 discussions

Security and maintenance release: 1.31.9 / 1.34.3

by Sam Reed

I would like to announce the release of MediaWiki 1.34.3, and 1.31.9! These releases also serve as a maintenance release for these branches. While tarballs have already been uploaded, git tags will follow later on today. An "MediaWiki Extensions Security Release Supplement" email will follow this one. As mentioned in the pre-release announcement, this will potentially be the final release of the MediaWiki 1.34 branch, barring any unforeseen issues. For continued support in the future, you are advised to upgrade to MediaWiki 1.35 in the near future. The release announcement for MediaWiki 1.35 will follow this one before the end of day tomorrow. MediaWiki 1.35 will be supported until September 2023. == Security fixes == * (T232568, CVE-2020-25813) SECURITY: SpecialUserrights: If a viewer lacks `hideuser`, ignore hidden users. * (T255918, CVE-2020-25812) SECURITY: Unescaped message used in HTML on Special:Contributions. * (T256171, CVE-2020-25815) SECURITY: Unescaped message used in HTML within LogEventsList. * (T258763, CVE-2020-17367, CVE-2020-17368) SECURITY: Prevent invoking firejail's --output functionality. * (T86738, CVE-2020-25814) SECURITY: mediawiki.jqueryMsg: Sanitize URLs and 'style' attribute. * (T115888, CVE-2020-25828) SECURITY: mediawiki.js: Escape HTML in mw.message( ... ).parse(). * (T260485, CVE-2020-25869) SECURITY: ActorMigration: Load user from the correct database. * (T260485, CVE-2020-25869) SECURITY: ensure actor ID from correct wiki is used. * (T251661, CVE-2020-25827) SECURITY: TOTP throttle not enforced cross-wiki. == Links to all mentioned tasks == * https://phabricator.wikimedia.org/T232568 * https://phabricator.wikimedia.org/T255918 * https://phabricator.wikimedia.org/T256171 * https://phabricator.wikimedia.org/T258763 * https://phabricator.wikimedia.org/T86738 * https://phabricator.wikimedia.org/T115888 * https://phabricator.wikimedia.org/T260485 * https://phabricator.wikimedia.org/T251661 == Release notes == Full release notes for 1.31.9: https://phabricator.wikimedia.org/diffusion/MW/browse/REL1_31/RELEASE-NOTES… https://www.mediawiki.org/wiki/Release_notes/1.31 Full release notes for 1.34.3: https://phabricator.wikimedia.org/diffusion/MW/browse/REL1_34/RELEASE-NOTES… https://www.mediawiki.org/wiki/Release_notes/1.34 For information about how to upgrade, see <https://www.mediawiki.org/wiki/Manual:Upgrading> ********************************************************************** Download: https://releases.wikimedia.org/mediawiki/1.31/mediawiki-1.31.9.tar.gz Download without bundled extensions: https://releases.wikimedia.org/mediawiki/1.31/mediawiki-core-1.31.9.tar.gz Patch to previous version (1.31.8): https://releases.wikimedia.org/mediawiki/1.31/mediawiki-1.31.9.patch.gz GPG signatures: https://releases.wikimedia.org/mediawiki/1.31/mediawiki-core-1.31.9.tar.gz.… https://releases.wikimedia.org/mediawiki/1.31/mediawiki-1.31.9.tar.gz.sig https://releases.wikimedia.org/mediawiki/1.31/mediawiki-1.31.9.patch.gz.sig Public keys: https://www.mediawiki.org/keys/keys.html ********************************************************************** Download: https://releases.wikimedia.org/mediawiki/1.34/mediawiki-1.34.3.tar.gz Download without bundled extensions: https://releases.wikimedia.org/mediawiki/1.34/mediawiki-core-1.34.3.tar.gz Patch to previous version (1.34.2): https://releases.wikimedia.org/mediawiki/1.34/mediawiki-1.34.3.patch.gz GPG signatures: https://releases.wikimedia.org/mediawiki/1.34/mediawiki-core-1.34.3.tar.gz.… https://releases.wikimedia.org/mediawiki/1.34/mediawiki-1.34.3.tar.gz.sig https://releases.wikimedia.org/mediawiki/1.34/mediawiki-1.34.3.patch.gz.sig Public keys: https://www.mediawiki.org/keys/keys.html

3 years, 7 months

Allow HTML email

by Tim Starling

* Should Mailman collapse multipart/alternative to its first part content? * Should Mailman convert text/html parts to plain text? This conversion happens after MIME attachments have been stripped. These mailman options are both enabled for this list. I would argue that they should both be disabled. It is nice to write emails with cutting-edge features like client-side word wrapping and links. Are there any objections to this change? -- Tim Starling

3 years, 7 months

Security pre-release announcement: 1.31.9 / 1.34.3 / 1.35.0

by Sam Reed

Hi all, Tomorrow we will be issuing a security and maintenance release to all supported branches of MediaWiki. The new releases will be: - 1.34.3 - 1.31.9 This will resolve eight issues in MediaWiki core (two of which aren't applicable to MediaWiki 1.31), and also includes some fixes previously committed to git, including minor security and hardening patches along with bug fixes included for maintenance reasons. For those of you waiting on MediaWiki 1.35.0, this will also come either tomorrow, or at latest Friday (depending on CI load), including the security fixes applied to these other supported release branches. We thank you for your patience; the current global situation means things have taken longer than would have been expected, but it has meant more bug fixes being incorporated from testing across the board. It also meant not having a security 1.35.1 release followup only a couple of weeks after 1.35.0 coming out. Which, for many people would mean extra work to upgrade again, and it was decided to avoid this. We will make the fixes available in these respective release branches, and also master. Tarballs will be available for the above mentioned point releases as well. A summary of some of the security fixes that have gone into non-bundled MediaWiki extensions will also follow. As per the MediaWiki Version lifecycle [1], November 2020 is the scheduled EOL date for the REL1_34. 1.34.3 will therefore potentially be the final release of the MediaWiki 1.34 branch, barring any unforeseen issues. As per above, MediaWiki 1.35.0 will be released this week, and will be supported until at least September 2023, and would be the recommended upgrade path. [1] https://www.mediawiki.org/wiki/Version_lifecycle

3 years, 7 months

Production Excellence #23: July & August 2020

by Krinkle

📘 Read on Phabricator at https://phabricator.wikimedia.org/phame/post/view/204/ ------- How’d we do in our strive for operational excellence last month? Read on to find out! ## 📈 *Incidents* 4 documented incidents in July, and 2 documented incidents in August. [1] Historically, that's on average for this time of year. [5] For more about recent incidents see Incident documentation <https://wikitech.wikimedia.org/wiki/Incident_documentation> on Wikitech, or Preventive measures <https://phabricator.wikimedia.org/project/view/4758/> in Phabricator. ------- ## 📊 *Tre**nds* Take a look at the workboard and look for tasks that could use your help. → https://phabricator.wikimedia.org/tag/wikimedia-production-error/ Summary over recent months: - ⚠️ July 2019 (4 of 18 tasks left): One task closed. - ⚠️ August 2019 (1 of 14 tasks left): *no change.* - ⚠️ September 2019 (3 of 12 tasks left): Two tasks closed. - October (6 of 12 tasks left), *no change.* - November (3 of 5 tasks left): *no change.* - December (3 of 9 tasks left), Two tasks closed. - January 2020 (5 of 7 tasks lef), *no change.* - February (2 of 7 tasks left), Two tasks closed. - March (2 of 2 tasks left), *no change.* - April (10 of 14 tasks left): One task closed. - May (7 of 14 tasks left): Four tasks closed. - June (10 of 14 tasks left): Four tasks closed. - July 2020: 13 of 24 new tasks <https://phabricator.wikimedia.org/maniphest/query/s__D8Kd0xuQH/#R> survived the month of July and remain open today. - August 2020: 37 of 53 new tasks <https://phabricator.wikimedia.org/maniphest/query/hu1yhWu4sXkP/#R> survived the month of August and remain open today. Recent tally: 72 open, as of Excellence #22 <https://phabricator.wikimedia.org/phame/post/view/203/production_excellence…> (Jul 23rd). -16 closed, of the previous 72 recent tasks. +13 opened and survived July 2020. +37 opened and survived August 2020. 106 open, as of today (Sep 23rd). Previously <https://phabricator.wikimedia.org/phame/post/view/203/production_excellence…>, we had 72 open production errors over the recent months up to June. Since then, 16 of those were closed. But, the 13 and 37 errors surviving July and August raise our recent tally to 106. The workboard overall (including tasks from 2019 and earlier) held 192 open production errors on July 23rd. As of writing, the workboard holds 296 open tasks in total. [4] This +104 increase is largely due to the merged backlog of JavaScript client errors, which were previously untracked. Note that we backdated the majority of these JS errors under “Old”, and thus are not amongst the elevated numbers of July and August. ------- 🎉 *Thanks!* Thank you to everyone else who helped by reporting, investigating, or resolving problems in Wikimedia production. Thanks! Until next time, – Timo Tijhof ------- Footnotes: [1] Incidents. – https://wikitech.wikimedia.org/wiki/Incident_documentation <https://wikitech.wikimedia.org/wiki/Incident_documentation#2020> [2] Tasks created. – https://phabricator.wikimedia.org/maniphest/query… <https://phabricator.wikimedia.org/maniphest/query/JuSOycDOe.7R/#R> [3] Tasks closed. – https://phabricator.wikimedia.org/maniphest/query… <https://phabricator.wikimedia.org/maniphest/query/aKNrCHMosori/#R> [4] Open tasks. – https://phabricator.wikimedia.org/maniphest/query… <https://phabricator.wikimedia.org/maniphest/query/Fw3RdXt1Sdxp/#R> [5] Wikimedia incident stats. – https://codepen.io/Krinkle/full/wbYMZK

3 years, 7 months

Spell4Wiki App - Audio Upload Tool for Wikimedia Commons

by Manimaran_K

Hello all, Happy to invite you to contribute your voice for wiktionary words using Spell4Wiki android application. 'Spell4Wiki' a GPLv3 licensed app to record and upload audio for Wiktionary words to Wikimedia Commons. The app is also a multilingual dictionary based on Wiktionary. This app was developed collaboratively by Kaniyam Foundation, Villupuram GNU/Linux Users Group, various FOSS activists and the Tamil Wiki Community. App Link : https://play.google.com/store/apps/details?id=com.manimarank.spell4wiki Note : Shortly release on F-Droid also. We started the process. You can look here : https://gitlab.com/fdroid/rfp/-/issues/1490 Below I shared links that have a detailed working process and development history of Spell4Wiki app. Everyone please look into this. That helps to understand how this FOSS tool made possible. Release and History of Development Tamil : https://manimaran96.wordpress.com/2020/06/26/spell4wiki-app-release-and-his… English : https://manimaran96.wordpress.com/2020/09/11/spell4wiki-app-release-history… More details : https://commons.wikimedia.org/wiki/Commons:Spell4Wiki Source code : https://github.com/manimaran96/Spell4Wiki Videos : Introduction - https://youtu.be/IMku3FL7s3I How to Use[Tamil] - https://youtu.be/4y5I1sUW1ys How to Use[English] - https://youtu.be/Fu4kQcv04kA Credits goes to - Kaniyam Foundation(http://www.kaniyam.com/) , - VGLUG(https://vglug.org/) <https://vglug.org/>, - All FOSS activists who have contributed ( https://github.com/manimaran96/Spell4Wiki#credits-to) and - Tamil Wiki Community(https://en.wikipedia.org/wiki/Tamil_Wikipedia) <https://en.wikipedia.org/wiki/Tamil_Wikipedia> Also, Thanks to FSHM(https://fshm.in/) for recommending the Spell4Wiki App to their circle. Install, Use and share your feedback. Contribution guidelines details : https://github.com/manimaran96/Spell4Wiki/blob/master/docs/CONTRIBUTING.md Communication details : https://github.com/manimaran96/Spell4Wiki/blob/master/docs/CONTRIBUTING.md#… Thanks Manimaran K -- Manimaran K https://manimaran96.wordpress.com

3 years, 7 months

Require tag for API writes

by Hogan (US), Michael C

Hi, is there an example that shows how to reject writeapi edits made using scripts unless the edit has a required tag? For example, changes made using Visual Editor are labeled with “(Tag: Visual edit)”. Thank you!

3 years, 7 months

📈 Wikimedia production errors help

by Tyler Cipriani

Hello all! Over the past few months we've reached the ignominious milestone of the most open tasks of all time on the wikimedia-production-error dashboard[0]. Background: The wikimedia-production-error dashboard is a workboard of tasks created while digging through the Wikimedia production error logs. All tasks there are log messages that have originated on production servers. The number of new tasks being created with this tag in a given week is outpacing the number of tasks being closed in a given week: this past week we added 41 tasks and only closed 22. This is beginning to be unsustainable :( There are currently 281 open tasks filed for errors in production. Although we're triaging this workboard weekly, we rely on the expertise of developers most familiar with the error messages to triage them, prioritize them, and "fix" them (for whatever value of "fix" is appropriate). Below is a smattering of selected issues that could use some attention: 1. PHP Fatal error: Out of memory in cdb/src/Reader/DBA.php[1] 2. Uncaught ReferenceError: collectionCall is not defined[2] 3. Flow: PHP Notice: Undefined index: flow-workflow-change[3] 4. PHP Warning: unpack(): Type H: not enough input, need 4, have 0[4] 5. TypeError: undefined is not an object (evaluating 'this.getMIMEType')[5] 6. Elastica\Exception\ResponseException from line 56 of GeoData/includes/Searcher.php[6] 7. Wikimedia\CSS\Objects\ComponentValueList may not contain tokens of type "[".[7] Please help to triage or resolve these problems or any of the other 166 tasks needing triage[8] if you are able. <3 -- Tyler [0]: <https://phabricator.wikimedia.org/tag/wikimedia-production-error/> [1]: <https://phabricator.wikimedia.org/T260234> [2]: <https://phabricator.wikimedia.org/T259809> [3]: <https://phabricator.wikimedia.org/T259739> [4]: <https://phabricator.wikimedia.org/T259592> [5]: <https://phabricator.wikimedia.org/T259419> [6]: <https://phabricator.wikimedia.org/T258641> [7]: <https://phabricator.wikimedia.org/T258093> [8]: <https://phabricator.wikimedia.org/maniphest/query/LW5WTEnToXDn/#R.>

3 years, 7 months

TechCom meeting 2020-09-23

by Tim Starling

This is the weekly TechCom board review in preparation of our meeting on Wednesday. If there are additional topics for TechCom to review, please let us know by replying to this email. However, please keep discussion about individual RFCs to the Phabricator tickets. Activity since Monday 2020-09-13 on the following boards: https://phabricator.wikimedia.org/tag/techcom/ https://phabricator.wikimedia.org/tag/techcom-rfc/ Committee inbox: * Two tasks have been sitting in there for multiple weeks Committee board activity: * RFCs only, see below New RFCs: * None Phase progression: * Timo hid the "Old" column and moved old RFCs to P1. Some had additional changes or comments. * Several were closed by Timo with status "declined", which James Forrester changed to "invalid". * It is hard to find justification for these actions in the linked RFC process document. There is no mention of it in the committee meeting minutes for the last three weeks. * T157402 <https://phabricator.wikimedia.org/T157402> Provide a reliable way to pass information between hook handlers, "hooked" objects o An RFC that was stalled since 2019, closed "declined" * T487 <https://phabricator.wikimedia.org/T487> Associated namespaces <https://phabricator.wikimedia.org/T487> o Timo asks if it can be merged with something. * T96384 <https://phabricator.wikimedia.org/T96384> * T154675 <https://phabricator.wikimedia.org/T154675> Introduce a listener interface for LinkRenderer hooks o Closed * T259771 <https://phabricator.wikimedia.org/T259771> Drop support for database upgrade older than two LTS releases o Moved to P3 * T252091 <https://phabricator.wikimedia.org/T252091>Site-wide edit rate limiting with PoolCounter o Moved to P2. Timo asks who is stewarding it. * T240775 <https://phabricator.wikimedia.org/T240775> Support PHP 7.4 preload o Moved to P3 * T128351 <https://phabricator.wikimedia.org/T128351>Notifications should be in core o Some back and forth over the status of this, ending up with it being P1 and "stalled" * T215046 <https://phabricator.wikimedia.org/T215046>Use Github login for mediawiki.org o Timo closed "declined". * T105766 <https://phabricator.wikimedia.org/T105766>Dependency graph storage; sketch: adjacency list in DB o Timo closed due to lack of owner * T484 <https://phabricator.wikimedia.org/T484>Scoped language converter o Timo closed due to lack of owner * T114662 <https://phabricator.wikimedia.org/T114662>Per-language URLs for multilingual wiki pages o Timo closed due to lack of owner * T120380 <https://phabricator.wikimedia.org/T120380>Allow JSON values to be included in the API results o Timo closed due to lack of owner * T193690 <https://phabricator.wikimedia.org/T193690>How should we fix the undeletion system? o Timo moved to P1 and stalled * T113034 <https://phabricator.wikimedia.org/T113034>Overhaul Interwiki map, unify with Sites and WikiMap o Old -> P1 * T119043 <https://phabricator.wikimedia.org/T119043>Graph/Graphoid/Kartographer - data storage architecture o Timo closed due to lack of owner, Yurik's RFC superseded by Dan's RFC * T196950 <https://phabricator.wikimedia.org/T196950>Pages do not have stable identifiers o Timo closed due to lack of owner * T158360 <https://phabricator.wikimedia.org/T158360>Reevaluate LocalisationUpdate extension for WMF o Old -> P1 * T181451 <https://phabricator.wikimedia.org/T181451>WebAssembly and compiled JS code best practices o Old -> P1 * T114445 <https://phabricator.wikimedia.org/T114445>Balanced templates o Old -> P1 * T213345 <https://phabricator.wikimedia.org/T213345>Spin off (Parsoid) language variants functionality as a microservice? o Timo closed due to lack of owner * T202673 <https://phabricator.wikimedia.org/T202673>Multiblocks - let admins create multiple, overlapping blocks on a single user o Old -> P1 * T111588 <https://phabricator.wikimedia.org/T111588>API-driven web front-end o Timo closed due to lack of owner * T117550 <https://phabricator.wikimedia.org/T117550>Content bundler o Timo closed due to lack of owner * T111604 <https://phabricator.wikimedia.org/T111604>: Split parser tests info multiple files o Timo closed due to lack of owner * T106099 <https://phabricator.wikimedia.org/T106099>Page composition using service workers and server-side JS fall-back o Timo closed due to lack of owner * T40010 <https://phabricator.wikimedia.org/T40010>Re-evaluate librsvg as SVG renderer on Wikimedia wikis o Old -> P1 * T347 <https://phabricator.wikimedia.org/T347>CentralNotice Caching Overhaul - Frontend Proxy o Timo closed due to lack of owner IRC meeting request: * None Other RFC activity: * T260714 <https://phabricator.wikimedia.org/T260714> Parsoid Extension API o Prior to last week's meeting, Subbu described the consultation work that has been done. o Suggest last call * T260330 <https://phabricator.wikimedia.org/T260330> PHP microservice for containerized shell execution o Some comments leading up to last week's meeting * T179680 <https://phabricator.wikimedia.org/T179680>deprecate VirtualRESTService o Timo tagged PE * T249164 <https://phabricator.wikimedia.org/T249164>: Better interface for generating metrics in MediaWiki o Timo poked it * T172477 <https://phabricator.wikimedia.org/T172477>Attribute anonymous contributions to the first IP address used in a session o Timo renamed

3 years, 7 months

[Data Release] Editors by Country in AQS

by Marcel Ruiz Forns

Hi everyone! We're announcing the *release of an API endpoint* that has been requested for a long time: *Editors by country*. This data set was already made public in the form of dumps last November (see email <https://lists.wikimedia.org/pipermail/analytics/2019-November/006702.html>). And now it's available via the Analytics Query Service API. Check it out: - Get *last month*'s breakdown of *active editors* (5-99 edits) for *Portuguese Wikipedia*: https://wikimedia.org/api/rest_v1/metrics/editors/by-country/pt.wikipedia/5… - Get *last January*'s breakdown of *very active editors* (100+ edits) for *English Wikipedia*: https://wikimedia.org/api/rest_v1/metrics/editors/by-country/en.wikipedia/1… Two-letter ISO country codes <https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes> are used for breakdowns. For more detailed information, refer to the full documentation of the API endpoint <https://wikitech.wikimedia.org/wiki/Analytics/AQS/Editors_by_country>, or the documentation of the underlying data <https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/Geoeditors/Pu…> . As a next step, we'll add the corresponding visualization to Wikistats2 <http://stats.wikimedia.org/>. Cheers! On behalf of the Analytics team, -- *Marcel Ruiz Forns** (he/him)* Senior Software Engineer

3 years, 7 months

[Wikimedia Tech Talks] (Modern) Event (Data) Platform, Tuesday 22 September 2020 at 17:00 UTC

by Sarah R

It's time for Wikimedia Tech Talks 2020 Episode 8! This talk will take place Tuesday 22 September 2020 at 17:00 UTC. Title: (Modern) Event (Data) Platform Speaker: Andrew Otto, Staff Site Reliability Engineer Summary: Wikimedia's Event (Data) Platform provides a foundation for building loosely coupled event driven software systems. This talk will go over why we built Event Platform and give an overview of its components and how they work. The link to the Youtube Livestream can be found here: <https://www.youtube.com/watch?v=jNNy8ALGjaE> https://www.youtube.com/watch?v=Tb_9gOYsWJQ During the live talk, you are invited to join the discussion on IRC at #wikimedia-office You can browse past Tech Talks here: https://www.mediawiki.org/wiki/Tech_talks If you are interested in giving your own tech talk, you can learn more here: https://www.mediawiki.org/wiki/Project:Calendar/How_to_schedule_an_event#Te… Note: This is a public talk. Feel free to distribute through appropriate email and social channels! Kindly, Sarah R. Rodlund Senior Technical Writer, Developer Advocacy <https://meta.wikimedia.org/wiki/Developer_Advocacy> srodlund(a)wikimedia.org

3 years, 7 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l September 2020