Wikitech-l November 2019

wikitech-l@lists.wikimedia.org

53 participants
71 discussions

BREAKING CHANGE: schema update, xml dumps
by Ariel Glenn WMF 27 Nov '19

27 Nov '19

We plan to move to the new schema for xml dumps for the February 1, 2020 run. Update your scripts and apps accordingly! The new schema contains an entry for each 'slot' of content. This means that, for example, the commonswiki dump will contain MediaInfo information as well as the usual wikitext. See https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/master/docs… for the schema and https://www.mediawiki.org/wiki/Requests_for_comment/Schema_update_for_multi… for further explanation and example outputs. Phabricator task for the update: https://phabricator.wikimedia.org/T238972 PLEASE FORWARD to other lists as you deem appropriate. Thanks! Ariel Glenn

1 0

Thank you Tuesday
by Amir Sarabadani 27 Nov '19

27 Nov '19

Hey, Since it's still Tuesday in some parts of the Pacific ocean, I want to start the thread of Thank you Tuesday for this week [1] by thanking James Forrester for working on improving production mediawiki configs that reached an important milestone yesterday [2] This work is extremely important not just because of migration to containers but also because it makes our systems less prone to breakage. Thank you James. [1] We haven't done thank you Tuesdays for a long time now, we should do it more often. [2] https://lists.wikimedia.org/pipermail/wikitech-l/2019-November/092816.html Best -- Amir Sarabadani (he/him) Software engineer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 https://wikimedia.de Unsere Vision ist eine Welt, in der alle Menschen am Wissen der Menschheit teilhaben, es nutzen und mehren können. Helfen Sie uns dabei! https://spenden.wikimedia.de Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

Parsoid/PHP deployment update
by Subramanya Sastry 26 Nov '19

26 Nov '19

Hi all, Further to my email from Nov 19 [1], here is the next update about Parsoid/PHP deployment on the Wikimedia cluster. As of today, * Parsoid/PHP is serving Visual Editor (VE), Content Translation (CX), and Mobile Content Service (MCS, used by the Android app) on group 0 and group 1 wikis [2]. * Parsoid/PHP is serving Flow on group 0 wikis [2]. It will be deployed to group 1 wikis on Monday, Dec 2. * During the week of Dec 2, we plan to enable Parsoid/PHP on all group 2 wikis for VE, CX, MCS, and Flow. * Linter output on all wikis is currently served by Parsoid/JS and will be the last product to be switched over to Parsoid/PHP. If you notice any problems, please flag it on T238928 [3] or file a phab task and tag Parsoid-PHP. You can follow the deployment schedule on T229015 [4]. Subbu (on behalf of the Parsing team). [1] https://lists.wikimedia.org/pipermail/wikitech-l/2019-November/092777.html [2] https://wikitech.wikimedia.org/wiki/Deployments/One_week (info about what wikis are in groups 0, 1, 2) [3] https://phabricator.wikimedia.org/T238928 [4] https://phabricator.wikimedia.org/T229015

1 0

Production config dblists are now auto-generated at merge time from YAML
by James Forrester 26 Nov '19

26 Nov '19

Hey all, [Unless you're writing Wikimedia production MediaWiki-land configuration changes, you can ignore this note.] As part of my work on moving variant configuration in static files[0], I've just merged and deployed a change to how dblists work. They are now auto-generated at merge time[1] (except for the very few lists which are live-computed in production, which remain). The lists are calculated from the per-wiki YAML files in wmf-config/config (and if you try to change the dblist files manually, CI will error at you). Changes to dblist memberships are quite rare, and this will enforce that we don't make mistakes. (As part of this deployment, I removed three duplicate entries and noted several irregularities we probably didn't intend.) This means that if you're writing a change that previously would have meant manually editing the dblist files, you now have to touch the YAML file instead (and run composer build locally, or use CI to tell you what to change). This mostly affects the creation of new wikis (which should now be slightly simpler), plus a few major features currently enabled through dblists. For an example, if you were to remove FlaggedRevisions from Chechen Wikipedia, you'd now need to edit cewiki.yaml[2] to remove the " - flaggedrevs" line, as well as changes to InitialiseSettings.php. This work unblocks some of the future re-architecting of config, which I've mentioned before, as part of the long-term plan to move Wikimedia production into containers. Configuration traits related to FR will inherit from flaggedrevs.yaml, and static-like code in CommonSettings.php (and in this case, flaggedrevs.php) can be significantly shrunk. If anything has broken or has become, if you have any questions, or want to talk about things, I'd be happy to discuss, here, on IRC, or on Phabricator. Thanks! [0] - https://phabricator.wikimedia.org/T223602 [1] - https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/545411 [2] - https://gerrit.wikimedia.org/r/plugins/gitiles/operations/mediawiki-config/… Yours, -- *James D. Forrester* (he/him <http://pronoun.is/he> or they/themself <http://pronoun.is/they/.../themself>) Wikimedia Foundation <https://wikimediafoundation.org/>

1 0

FOSDEM 2020 web performance devroom CfP
by Gilles Dubuc 26 Nov '19

26 Nov '19

We (the Wikimedia Performance Team <https://www.mediawiki.org/wiki/Wikimedia_Performance_Team>) are hosting a devroom at FOSDEM 2020 <https://fosdem.org/2020/>. We are still looking for talk proposals on the topic of Web Performance, with a focus on FLOSS, open standards and research. Deadline is December 2. Details here: https://github.com/wikimedia/fosdem20-web-performance-cfp

1 0

s7 primary master switchover (read-only required) 26th Nov 06:00 AM UTC
by Manuel Arostegui 26 Nov '19

26 Nov '19

Hello, We have requested a 30 minutes read-only window for s7 (T238046) for the 26th November from 06:00-06:30 AM UTC to switchover that section primary database master (T238044) db1062 is an old host and out of warranty that will be decommissioned (T217396). The new master will be db1086 We are going to do this on Tuesday 26th Nov from 06:00 to 06:30 AM UTC (we do not expect to use the 30 minutes window, if everything goes as expected). Impact: Writes will be blocked on the following wikis: arwiki cawiki eswiki fawiki frwiktionary hewiki huwiki kowiki metawiki rowiki ukwiki viwiki Reads will remain unaffected. This also includes centralauth database, which means some operations might fail during the read-only period, such as: GlobalRenames, Changing/Confirming emails, logging into new wikis, password changes... Communication will happen at #wikimedia-operations If you are around at that time and want to help with the monitoring, please join us! Thanks

1 2

This Week in Logspam (Thanksgiving Edition)
by Brennen Bearnes 25 Nov '19

25 Nov '19

Hi all, A high volume of error messages in production logs can make it hard to glance at error logs after a deployment and reason about the deployment's impact.[0] This recurring mail is part of an ongoing effort to reduce log noise. At present I'm aware of three unresolved-but-tracked issues which are causing a fair amount of noise: * https://phabricator.wikimedia.org/T143756 ** UnresolvedRedirectException in EntityAccessor::getEntity (scribunto) ** ~8000 occurrences today * https://phabricator.wikimedia.org/T226751 ** PHP error "non well formed numeric value encountered" from FormatMetadata->formatCoords ** ~600 occurrences today * https://phabricator.wikimedia.org/T239165 ** PHP Warning: count(): Parameter must be an array or an object that implements Countable in WikibaseMediaInfo ** ~120 occurrences today As ever, help in eliminating these errors from production logs is greatly appreciated. As a procedural note, I'm sending this early in the week due to the Thanksgiving holiday. We'll be releasing 1.35.0-wmf.8 to group 0 tomorrow (Tuesday) and then pausing there until 2019-12-04. [0]. https://wikitech.wikimedia.org/wiki/Deployments/Holding_the_train#Logspam -- Brennen Bearnes (he/him) Release Engineering Wikimedia Foundation

1 0

Cloud VPS users, please claim your projects
by Andrew Bogott 25 Nov '19

25 Nov '19

Every year or so the Cloud Services team tries to identify and clean up unused projects and VMs. We do this via an opt-in process: anyone can mark a project as 'in use,' and that project will be preserved for another year. I've created a wiki page the lists all existing projects, here: https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2019_Purge If you are a VPS user, please visit that page and mark any projects that you use as {{Used}}. Note that it's not necessary for you to be a project admin to mark something -- if you know that you're currently using a resource and want to keep using it, go ahead and mark it accordingly. If you /are/ a project admin, please take a moment to mark which VMs are or aren't used in your projects. When December arrives, I will shut down and begin the process of reclaiming resources from unused projects. If you think you use a VPS project but aren't sure which, I encourage you to poke around on https://tools.wmflabs.org/openstack-browser/ to see what looks familiar. Worst case, just email cloud(a)lists.wikimedia.org with a description of your use case and we'll sort it out there. Exclusive toolforge users are free to ignore this task. Thank you! -Andrew and WMCS team

2 4

Google Code-in 2019 starts in one week (and we need more tasks)
by Andre Klapper 25 Nov '19

25 Nov '19

Wikimedia is one of the organizations in Google Code-in (GCI) 2019! GCI starts on December 02 and runs for seven weeks. It's a contest for 13-17 year old students working on small tasks and an opportunity to let new contributors help with tasks on your To-Do list! :) PLEASE CONSIDER BECOMING A MENTOR: 1. Go to https://www.mediawiki.org/wiki/Google_Code-in/2019 and add yourself to the mentor's table. 2. Get an invitation email to register on the contest site. ...AND MENTOR SMALL TASKS: * Do your docs on your wiki need some improvements? * Does your template or gadget code needs some updates? * Do you have small and self-contained bugs you'd love to get fixed? * Does your UI have small design issues? * Do your old bugs welcome some testing? We need tasks in the following areas: code, outreach/research, documentation/training, quality assurance, user interface/design. See https://www.mediawiki.org/wiki/Google_Code-in/Mentors Looking for task ideas? Check the good_first_bugs in Phabricator: https://phabricator.wikimedia.org/maniphest/query/agR7xiuqv_bc/#R Any questions? Feel free to ask the org admins at google-code-in-admins(a)lists.wikimedia.org - we are happy to help! Thank you for your help broadening our contributor base! andre -- Andre Klapper (he/him) | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/

1 0

Help on creating a new extension
by Sohom Datta 25 Nov '19

25 Nov '19

Hi guys, I'm Sohom Datta. I recently decided to create a draft for an hypothetical extension ProtectionIndicator, which will add a protection icon on the top of every protected page. I did this primarily out of my own interest in getting familiar with the code-base of MediaWiki. I've hosted all the code I have written (still unfinished) in this github repository <http://github.com/sohomdatta1/ProtectionIndicator>. I'd like some guidance on how to implement the unfinished parts of the extension especially the integration with FlaggedRevs and the addition of a log entry to a popup ( Details are there in the readme.md of the repo) . Also any feedback on my code would also be very good. Also, it would be great if somebody could guide me on how to make this a official extension someday. Regards, Sohom Datta

5 6

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l November 2019