Wikitech-l May 2021

wikitech-l@lists.wikimedia.org

33 participants
50 discussions

GitLab: Implementation status update and a call for early adopters
by Brennen Bearnes 14 May '21

14 May '21

tl;dr: We're still moving to GitLab. WMF Release Engineering & SRE are working with contractors on an MVP installation of GitLab CE, which we hope to have running by July, and in use by early-adopter projects soon thereafter. Read on for more details. ---- Hi all, It's been over six months since we wrapped up the GitLab consultation process[0] with a decision to migrate from Gerrit to GitLab's Community Edition. It seemed like a good idea to update the community with our progress since then. We currently have contractors from Speed & Function working on a minimum-viable installation of GitLab CE. We hope to be fairly confident installing this by the end of June (end Q4 of WMF's fiscal year). You can follow this work on the "GitLab (Initialization)" workboard in Phabricator.[1] Additionally, we're hiring for two Service Operations SRE positions to build and support a more robust production installation[3], and planning for new job runner capacity to handle CI on GitLab. Tentatively, we in RelEng hope to begin using GitLab for some of our own projects around July (early Q1 of WMF fiscal year) and begin migrating other projects thereafter. (Keeping in mind, of course, that everything takes longer than you expect.[4]) We've already heard from some people that their project or team is interested in being an early adopter of GitLab. If that describes you, please add a comment on T282842: https://phabricator.wikimedia.org/T282842 Feel free to reach out with any questions! [0]. https://www.mediawiki.org/wiki/GitLab_consultation#Outcome [1]. https://phabricator.wikimedia.org/project/view/5212/ [3]. https://boards.greenhouse.io/wikimedia/jobs/2659677 [4]. https://en.wikipedia.org/wiki/Hofstadter%27s_law -- Brennen Bearnes Release Engineering Wikimedia Foundation

1 0

[Train] 1.37.0-wmf.5 status update (blocked again at group1)
by Brennen Bearnes 13 May '21

13 May '21

Hi all, A patch was provided for T282735 earlier today and the 1.37.0-wmf.5[0] train rolled to all wikis at the scheduled time. Thanks to Zabe for speedy assistance there. Unfortunately, the train is now blocked at group1 by the following issues: * MapSourcesPage.php: PHP Notice: Undefined offset: 13 - https://phabricator.wikimedia.org/T282833 * LogEventsList.php: PHP Notice: Undefined variable: pageName - https://phabricator.wikimedia.org/T282834 Thanks in advance for any assistance with these! As ever, you can follow train progress on Freenode's #wikimedia-operations as well as on the blocker task[0]. Regards, -- Your irascible train crew [0]. <https://phabricator.wikimedia.org/T281146> [1]. <https://wikitech.wikimedia.org/wiki/Deployments/Holding_the_train#Issues_th…>

1 0

Nomination for the +2 right in the Mediawiki group
by Zoran Dori 13 May '21

13 May '21

Hello everyone, per policy [0], I want to notify you that I've nominated myself again for +2 right in the MediaWiki group on the Gerrit. [1] Be free to comment if you want. Best regards, Zoran [0] https://www.mediawiki.org/wiki/Gerrit/Privilege_policy#Requesting%20Gerrit%… [1] https://phabricator.wikimedia.org/T282786

1 0

Page security extension disclaimer
by Charlot Cobben 13 May '21

13 May '21

Under the tag of WP:BOLD I took the liberty of copying the "Page security extension disclaimer" that is put on extension pages whenever an extension uses a legit hook like "usercan" . While I get why the disclaimer is added, the words used in the disclaimer are certainly up for improvements. It now states, whenever you need security, use another package, not MediaWiki. It also brands an extension using legit hooks to be a hack. So I made a simple template "Page security extension disclaimer 2021" that still shares the sames bottomline message : "MediaWiki is not responsible for exposed content", but using words that are more appropriate. I would like people to review the original disclaimer and the text I have used to come up with a suitable solution.

2 1

Production Excellence #31: April 2021
by Krinkle 12 May '21

12 May '21

How’d we do in our strive for operational excellence last month? Read on to find out! Read on Phabricator at https://phabricator.wikimedia.org/phame/post/view/235/ Incidents 6 documented incidents. That's above the historical average <https://codepen.io/Krinkle/full/wbYMZK> of 3–4 per month. Learn about recent incidents at Incident status <https://wikitech.wikimedia.org/wiki/Incident_status> on Wikitech, or Preventive measures <https://phabricator.wikimedia.org/project/view/4758/> in Phabricator. ------- Trends In April, we saw a continuation of the healthy trend that started this January <https://phabricator.wikimedia.org/phame/post/view/227/production_excellence…> — a trend where the back of the line is moving forward at least as quickly as the front of the line. We did take a little breather in March <https://phabricator.wikimedia.org/phame/post/view/229/production_excellence…> where we almost broke even, but otherwise the trend is going well. Last month we bade farewell to the production errors we found in July 2019. This month we cleared out the column for October 2019. One point of concern is that we did encounter a high number of new production errors — are errors that we failed to catch during development, code review, continuous integration, beta testing, or pre-deployment checks. Where we used to discover about a dozen of those a month, whereas we found 42 during this month. As of writing, 17 of the 42 April-discovered errors have been resolved. The "Old" column (generally tracking pre-2019 tasks) grew for the first time in six months. This increase can largely be attributed to improved telemetry of client-side errors uncovering issues in under-resourced products, such as the old Kaltura video player. Figure 1, Figure 2: Unresolved error reports stacked by month. <https://phabricator.wikimedia.org/phame/post/view/235/production_excellence…> ------- Outstanding errors Take a look at the workboard <https://phabricator.wikimedia.org/tag/wikimedia-production-error/> and look for tasks that could use your help. Summary over recent months, per spreadsheet <https://docs.google.com/spreadsheets/d/e/2PACX-1vTrUCAI10hIroYDU-i5_8s7pony…> : Aug 2019 (1 of 14 left) ⚠️ Unchanged (over one year old). Oct 2019 (0 of 12 left) ✅ Last three tasks resolved! -3 Jan 2020 (1 of 7 left) ⚠️ Unchanged (over one year old). Mar 2020 (2 of 2 left) ⚠️ Unchanged (over one year old). Apr 2020 (5 of 14 left) ⚠️ Unchanged (over one year old). May 2020 (5 of 14 left) ⏸ — Jun 2020 (5 of 14 left) ⬇️ One task resolved. -1 Jul 2020 (4 of 24 issues <https://phabricator.wikimedia.org/maniphest/query/s__D8Kd0xuQH/#R>) ⬇️ One task resolved. -1 Aug 2020 (13 of 53 issues <https://phabricator.wikimedia.org/maniphest/query/hu1yhWu4sXkP/#R>) ⬇️ Two tasks resolved. -2 Sep 2020 (7 of 33 issues <https://phabricator.wikimedia.org/maniphest/query/CGFQViLShnOY/#R>) ⏸ — Oct 2020 (20 of 69 issues <https://phabricator.wikimedia.org/maniphest/query/MYnnBybPTYpd/#R>) ⬇️ Two tasks resolved. -2 Nov 2020 (9 of 38 issues <https://phabricator.wikimedia.org/maniphest/query/CkC_VqQq5VC0/#R>) ⏸ — Dec 2020 (7 of 33 issues <https://phabricator.wikimedia.org/maniphest/query/10NQy74iKaZJ/#R>) ⬇️ Four tasks resolved. -4 Jan 2021 (1 of 50 issues <https://phabricator.wikimedia.org/maniphest/query/WIP9W8q54HB6/#R>) ⬇️ One task resolved. -1 Feb 2021 (8 of 20 issues <https://phabricator.wikimedia.org/maniphest/query/5MzPJAb5oJgv/#R>) ⬇️ One task resolved. -1 Mar 2021 (18 of 48 issues <https://phabricator.wikimedia.org/maniphest/query/RsVPep46KRY4/#R>) ⬇️ Sixteen tasks resolved. -16 *Apr 2021* (25 of 42 issues <https://phabricator.wikimedia.org/maniphest/query/rYyMt_gYYymb/#R>) 42 new issues found, of which 25 remained open. +42; -17 ------- Tally 139 issues open, as of Excellence #30 <https://phabricator.wikimedia.org/phame/post/view/229/production_excellence…> (March 2021). -31 issues closed, of the previously open issues. +25 new issues that survived April 2021. 133 issues open, as of today (12 May 2021). ------- Thanks Thank you to everyone who helped by reporting, investigating, or resolving problems in Wikimedia production. Thanks! Until next time, – Timo Tijhof

1 0

Summary of *last week*'s deployment of 1.37.0-wmf.4
by Tyler Cipriani 12 May '21

12 May '21

This is a belated summary of *last week*'s deployment of the 1.37.0-wmf.4 train. The primary train conductor for the week was Brennen Bearnes, with Lars Wirzenius as backup in European timezones and considerable assistance from Ahmon Dancy. The blocker task for the week was: https://phabricator.wikimedia.org/T281145 == Stats == * 422 patches * 2 risky patches identified * 1 rollback * 19 hours spent rolled back * group2 was delayed by 1 day * 10 train blockers were added, 6 were resolved, 4 were determined to not be blockers == 🚂😻 == It takes a village[0] to release a train. Thank you to the folks that helped us this past week: * Marius Hoch * DannyS712 * Petr Pchelko * Tim Starling * Eric Gardner * The inimitable Timo Tijhof * The one and only James Forrester * Majavah * RhinosF1 * Urbanecm * AntiCompositeNumber And anyone else whose brain was occupied by thoughts of the train this past week: thanks and sorry. XOXO – Train Gang [0]: <https://en.wikipedia.org/wiki/It_takes_a_village>

1 0

[Train] 1.37.0-wmf.5 status update (blocked at group0)
by Ahmon Dancy 12 May '21

12 May '21

Hi all, The 1.37.0-wmf.5[0] train is currently blocked at group0 by the following issue: * Wikimedia\Rdbms\DBQueryError: Error 1048: Column 'gt_page_id' cannot be null (db1138)Function: GeoData\Hooks::doLinksUpdateQuery: INSERT INTO `geo_tags` (gt_page_id,gt_id,gt_lat,gt_lon,gt_globe,gt_primary,gt_dim,gt_type,gt_name,gt_country,gt_region) VALUES (NULL,NULL,'45.811666666667','4.9194444444444','earth',1,1000,'camera',NULL,NULL,NULL) - https://phabricator.wikimedia.org/T282735 If this issue is resolved before the 19:00 UTC train deployment window, the train will proceed as normal to group1. As ever, you can follow train progress on Freenode's #wikimedia-operations as well as on the blocker task[0]. Regards, -- Your lovely train crew [0]. <https://phabricator.wikimedia.org/T281146> [1]. < https://wikitech.wikimedia.org/wiki/Deployments/Holding_the_train#Issues_th…>

1 0

2021-05-12 Scrum of Scrums meeting notes
by Grace Gellerman 12 May '21

12 May '21

https://www.mediawiki.org/wiki/Scrum_of_scrums/2021-05-12 = 2021-05-12 = == Callouts == * … Fundraising-Tech => Releng * a little help from releng to push through an update to our CivCRM CI job would be hugely appreciated! Patch is here https://gerrit.wikimedia.org/r/c/integration/config/+/688350/ == Gerrit patches or GitHub Pull Requests for reviews or feedback == * === No updates === CommTech, AHT, Editing, Growth, iOS, Android, Parsing, Language, Analytics, Cloud Services, Platform, Performance, Security == SoS Meeting Bookkeeping == * Updates: ** future of the synchronous meeting == Product == === Web === * Blocked by: * Blocking: * Thank yous: ** Belated thanks to Sam Reed for submitting the Voice & Tone issues, e.g. https://phabricator.wikimedia.org/T277955 * Updates: ** Prep for the language switcher treatment A/B test ** Started work on the new user links (personal tools) treatment: https://phabricator.wikimedia.org/T266536 ** QA'ing/analysing data from the search autocomplete widget treatment A/B test === Product Infrastructure === * Blocked by: ** Need input from Language Team on https://phabricator.wikimedia.org/T277059#6976371 * Blocking: * Thank yous: * Updates: === Structured Data === * Blocked by: * Blocking: * Thank yous: * Updates: ** No significant updates, continuing to make improvements to MediaSearch and to work on the image recommendations testing tool === Abstract Wikipedia === * Blocked by: ** None. * Blocking: ** None known. * Thank yous: * Updates: ** We finished Phase δ (delta) this week; hurrah. ** Now working on Phase ε (epsilon) – wiring up user-written code to the evaluator: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Phases ** Security & Performance reviews scheduled to end this Phase; our thanks to them! === Inuka === * Blocked by: * Blocking: * Thank yous: * Updates: ** Working on the Wikipedia Preview Wordpress Plugin: https://phabricator.wikimedia.org/T278383 ** Improving Wikipedia Preview gallery feature: https://phabricator.wikimedia.org/T269787 === Library === * Blocked by: * Blocking: * Thank yous: * Updates: ** The Wikipedia Library Extension is now live on Beta! ** Finished work on making tags translatable, we just need to add extra configuration on TranslateWiki ** Finished setting up Matomo ** Starting work on the Library homepage redesign === Vue.js === * Blocked by: * Blocking: * Thank yous: * Updates: ** Rethinking the Icon component in WVUI ( https://phabricator.wikimedia.org/T282625) ** Working on a Radio component and introducing the composition API plugin to WVUI (https://phabricator.wikimedia.org/T281186) == Technology == === Fundraising Tech === * Blocked by: ** a little help from releng to push through an update to our CivCRM CI job would be hugely appreciated! Patch is here https://gerrit.wikimedia.org/r/c/integration/config/+/688350/ * Blocking: * Thank yous: * Updates: ** Deployed new email preferences center wiki ** Started integration work for backup processor's new API === Engineering Productivity === ==== Quality and Test Engineering ==== * Blocked by: * Blocking: * Thank yous: * Updates: ** Blog post by John Bolorinos - Improvements to Email QA https://phabricator.wikimedia.org/phame/post/view/239/improvements_to_email… ==== Release Engineering ==== * Blocked by: ** Web: Upgrading or dropping the legacy Ruby selenium jobs in MinervaNeue blocks the CI-wide upgrade from stretch to buster. T280901 ** Wikibase: Upgrading or dropping the legacy Ruby selenium jobs in Wikibase blocks the CI-wide upgrade from stretch to buster. T280491 * Blocking: * Thank yous: ** Thank you to C. Scott for his support and patience waiting for T271863 to be resolved! * Updates: ** [All] Deployments/Covid-19 https://wikitech.wikimedia.org/wiki/Deployments/Covid-19 ** Train Health *** Last week: 1.37.0-wmf.4 [[phab:T281145]]  *** This week: 1.37.0-wmf.5 [[phab:T281146]]  *** Next week: 1.37.0-wmf.6 [[phab:T281147]]  === Search Platform === * Blocked by: * Blocking: * Thank yous: * Updates: ** Data missing from WCQS - https://phabricator.wikimedia.org/T280022 ** Get baseline measurements/expectations for splitting lexemes from Wikidata graph - https://phabricator.wikimedia.org/T275068 === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** None * Thank yous: * Updates: ** work on getting mw in k8s picking up, shellbox soon to be deployed. === WMDE Technical Wishes === * Blocked by: ** by a failing reportupdater query, https://phabricator.wikimedia.org/T273748#7051951 * Blocking: * Thank yous: * Updates: ** Reference Previews deployment in some wikis was successful: https://phabricator.wikimedia.org/T271206 ** Suggested Values feature (TemplateData) enabled everywhere: https://phabricator.wikimedia.org/T273857 ** Currently working on template search improvements: https://phabricator.wikimedia.org/T271802 == Cross-cutting == * Blocked by: ** [long term] Search Platform: PHP 8.0 work is long-term blocked on the migration to ElasticSearch 7.0 https://phabricator.wikimedia.org/T263142 (or at least 6.7). * Blocking: ** None known. * Thank yous: * Updates: ** REL1_36 release process: *** 1.36 RC.0 was released. Only a small handful of bugs found so far, all now fixed. If you have a release blocker, please file it urgently: https://phabricator.wikimedia.org/tag/mw-1.36-release/ ** CI tools: *** mediawiki-codesniffer v36 upgrade going slowly (lots of manual fixes); 67% of repos updated so far. *** CI tools' upgrade status: https://libraryupgrader2.wmcloud.org/status?branch=master ** PHP 8.0: *** No changes this week. *** Our target is REL1_35 (and thus also REL1_36), as well as master. *** Upstream libraries: Elastica-related PHP code is theoretically the last one. *** Core: Some unit and integration tests still fail; thank you to everyone working on fixing them. [[Category:Scrum of scrums{{#translation:}}|*]]

1 0

Is possible to use Wikimedia's public REST API on third-party wikis?
by lorentz0021＠gmail.com 11 May '21

11 May '21

Hello, I am a maintainer of a third-party wiki and the wiki is run on a machine with restricted resources. Our wiki is running its own Mathoid on the same machine and has a demand for Citoid. Recently, I have known the REST API of Wikimedia is open to the public[1]. I wonder if the public API is available and the usage is encouraged for third-party wikis like instant commons[2]. If so, I would make a plan to replace the Mathoid run on our machine with Wikimedia's one and add support for Citoid. Best regards, lens0021 [1]: https://en.wikipedia.org/api/rest_v1/ [2]: https://www.mediawiki.org/wiki/InstantCommons

1 0

Hackathon program: call for proposals open until May 9th
by Léa Lacroix 11 May '21

11 May '21

Hello all, The Wikimedia hackathon <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2021> is coming up in a few weeks, on May 22-23! We’re very excited to gather online with you all and hack together. On top of the informal hacking, we would like to build a light, community powered program, with sessions and discussions about the technical environment in the Wikimedia movement. We will have two different “tracks”, each of them having its own setup: 1. The main track, a livestream of sessions that will be recorded and available for replay, hopefully using a fully open source set up. This will be great for presentations, talks and demos. 2. The open rooms: two Jitsi rooms will be available to schedule your sessions. This is perfect for informal discussions, Q&As and workshops. If you’re interested in running a session in the hackathon’s program, it’s easy: please create a task on the Phabricator board <https://phabricator.wikimedia.org/project/board/5303/>, using this template <https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?title=[Sessio…>, that will give you all the required information. The coordination team will come back to you to help you prepare and schedule your session. You will also find the different formats that we are offering: presentation, discussion, workshop or lightning talk. All topics are welcome, as long as they are connected to the technical environment of the Wikimedia projects. We are particularly interested in sessions that are useful for newcomers, and connected to the main topics of this year’s hackathon: open source tools for events, documentation and finding co-maintainers. Because we will need some time to schedule the proposed sessions and to make technical tests with the speakers, please note that the deadline to submit a session for the main track is May 9th. For the open rooms, you will have a bit more time, until May 19th. You will find more information and details about the call for proposals on the how-to page <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2021/How_to>. If you have any questions, feel free to ask on the discussion page <https://www.mediawiki.org/wiki/Talk:Wikimedia_Hackathon_2021>. If you want to brainstorm ideas or find a co-speaker for your session, feel free to ask the crowd on the hackathon social channels <https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2021/Discussions>. Feel free to share the word anywhere you find it useful, and talk about it with your contacts! Thanks in advance, For the coordination team, -- Léa Lacroix Community Engagement Coordinator Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l May 2021