Wikitech-l March 2021

wikitech-l@lists.wikimedia.org

58 participants
59 discussions

Forthcoming failover of irc.wikimedia.org to a new server (2021-03-25)
by Moritz Mühlenhoff 24 Mar '21

24 Mar '21

Hi, the DNS name for irc.wikimedia.org [1] will be redirected to a new instance (irc2001.wikimedia.org) tomorrow (European) morning. This new instance is running a more recent OS (Debian 10/Buster). Mediawiki changes are currently broadcasted to both the old server (kraz.wikimedia.org) and the new irc2001.wikimedia.org. As such, the failover should be transparent, unless for some reason a bot has hardcoded kraz.wikimedia.org or it's IP address 208.80.153.44 (which will then need to be updated to resolve irc.wikimedia.org). The old instance will be around for another week. If there are any issues, please report them at https://phabricator.wikimedia.org/T224579 Cheers, Moritz Footnotes: [1] https://wikitech.wikimedia.org/wiki/Irc.wikimedia.org

2 1

2021-03-24 Scrum of Scrums meeting notes
by Grace Gellerman 24 Mar '21

24 Mar '21

https://www.mediawiki.org/wiki/Scrum_of_scrums/2021-03-24 = 2021-03-24 = (meeting conflict with Product Platform Strategy overview) == Callouts == * RelEng: All extensions will have a buster variant of their main CI job added as voting next week. This shouldn't break anything (it's been running in production for months now), but be aware if your extension suddenly stops passing tests. https://phabricator.wikimedia.org/T252432 * SRE Service Ops: ** Eqiad kubernetes cluster upgraded, reinitialized and fully repooled. Some Job backlog was created, it's being served though. ** Many thanks to all the teams for having worked on https://phabricator.wikimedia.org/T274262 * RelEng: Developer satisfaction survey extended until until 2021-03-31 * RelEng: When to use train vs backports quick guide: https://wikitech.wikimedia.org/wiki/Deployments/Train_vs_backport == Gerrit patches or GitHub Pull Requests for reviews or feedback == * === No updates === iOS === '''No notes provided''' === Editing, Parsing, Inuka, Cloud Services, Platform, Performance, Security, == SoS Meeting Bookkeeping == * Updates: == Product == === Community Tech === * Blocked by: * Blocking: * Thank yous: ** Subbu and other parsoid team members for their help and support with several tasks * Updates: ** We'll be wrapping up WS-Export work this week and move on full-time onto the OCR Improvements wish https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2020/Wikisource/N… === Anti-Harassment Tools === * Blocked by: * Blocking: * Thank yous: ** Huge thanks to David Lynch for excellent work on SecurePoll, and helping mentor our new engineers ** Thanks to everyone who has helped with onboarding the engineers * Updates: ** No updates === Growth === * Blocked by: * Blocking: * Thank yous: ** SRE with their continued support with the link recommendation service ** Search for their help with indexing pages with recommendations * Updates: ** Continuing work on Add Link https://wikitech.wikimedia.org/wiki/Add_Link *** Available (in its current unfinished form) via Special:Homepage on testwiki and beta cswiki now ** Continuing to work on on-wiki configuration ** Continuing design of mentor dashboard https://www.mediawiki.org/wiki/Growth/Mentor_dashboard ** Deploying Growth features in opt-in mode this week to tewiki, jawiki, eowiki, nbwiki, sqwiki, hiwiki === Android native app === * Blocked by: * Blocking: * Thank yous: * Updates: No big updates. Continuing our work w/ Platform Engineering Team on image recommendations - we're getting near the MVP launch date and cooperatively pruning the remaining desired API features. === Web === * Blocked by: * Blocking: * Thank yous: * Updates: ** No updates. We're continuing our work on the language switcher instrumentation and A/B test === Product Infrastructure === * Blocked by: * Blocking: * Thank yous: From SRE Serviceops for work on https://phabricator.wikimedia.org/T274262 * Updates: === Structured Data === * Blocked by: * Blocking: * Thank yous: * Updates: ** Wrapping up final work before MediaSearch goes default on Commons ** Continuing to work on improved search profile ** Continued discussions across teams regarding Structured Data Across Wikimedia (image recommendations API and structured content) === Abstract Wikipedia === * Blocked by: ** None. * Blocking: ** None. * Thank yous: ** Scott B. from Security for running a great threat modelling exercise with us. * Updates: ** Closing on end of Phase γ: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Phases === Language === * Blocked by: None. * Blocking: None. * Thank yous: ** Alexandros Kosiaris for cluster upgrades and for T278141 help. * Updates: ** No major updates deployed in Section Translation this week. ** MassMessage remote wiki delivery was briefly broken. Now fixed. ** Working on new translation variable syntax for better Parsoid support. ** There is now a limit how big translatable pages can be moved. Working on a script to move manually pages that are too big. ** Improving ULS language selector dialog positioning (for Web team). ** Translatable templates are usable on translatable pages with the normal syntax for transclusion. (https://phabricator.wikimedia.org/T47096) === Library === * Blocked by: * Blocking: * Thank yous: * Updates: ** Started work on making library partner's descriptions translatable ** Getting first mock-ups of the Library redesign. They look great and we look forward to implementing them! === Vue.js === * Blocked by: * Blocking: * Thank yous: * Updates: ** Onboarding our new program manager! ** Planning for the experimental shared internal component library, both on the design front and the development side == Technology == === Analytics === * Blocked by: * Blocking: ** SRE Service Ops on https://phabricator.wikimedia.org/T274262 * Thank yous: * Updates: ** mw server replacements in codfw proceeding ** Eqiad kubernetes cluster upgraded, reinitialized and fully repooled. Some Job backlog was created, it's being served though. === Fundraising Tech === * Blocked by: * Blocking: * Thank yous: * Updates: ** More work on email prefs page https://phabricator.wikimedia.org/T268510 ** Enhancements to optional employer autocomplete field (used for matching gifts) ** CiviCRM contribution import enhancements https://phabricator.wikimedia.org/T271416, https://phabricator.wikimedia.org/T275445 ** Investigating software libraries that handle multiple payment providers https://phabricator.wikimedia.org/T277246, https://phabricator.wikimedia.org/T277245 ** Better error handling for CentralNotice invalid banner name: https://phabricator.wikimedia.org/T173782 ** Audit / reconciliation file processing improvements: https://phabricator.wikimedia.org/T277244, https://phabricator.wikimedia.org/T265545 === Engineering Productivity === ==== Quality and Test Engineering ==== * Blocked by: * Blocking: * Thank yous: * Updates: ** "Outreachy, December 2020-March 2021" blog post by Željko Filipin https://phabricator.wikimedia.org/phame/post/view/233/outreachy_december_20… ==== Release Engineering ==== * Blocked by: * Blocking: * Thank yous: ** Daniel Zahn on ServiceOps for Gerrit apache help! * Updates: ** Deployments/Covid-19 https://wikitech.wikimedia.org/wiki/Deployments/Covid-19 ** Train Health *** Last week: 1.36.0-wmf.35 [[phab:T274939]]  *** This week: 1.36.0-wmf.36 [[phab:T274940]]  *** Next week: 1.36.0-wmf.37 [[phab:T278343]]  ** All extensions will have a buster variant of their main CI job added as voting next week. This shouldn't break anything (it's been running in production for months now), but be aware in case your extension suddenly stops passing tests. https://phabricator.wikimedia.org/T252432 ** (MediaWiki core, vendor, and skins all are already running the buster-based PHP72 job as voting in CI.) ** Backport / config deploys vs. train patch guidance; see https://wikitech.wikimedia.org/wiki/Deployments/Train_vs_backport === Search Platform === * Blocked by: * Blocking: * Thank yous: * Updates: ** Fix Glent M2 CJK suggestion tokenization - https://phabricator.wikimedia.org/T265081 ** Add hasrecommendation: search keyword - https://phabricator.wikimedia.org/T269493 ** [Log noise] "Prefix search request was longer than the maximum allowed length." - https://phabricator.wikimedia.org/T276385 ** Create new elasticsearch cookbook that combines a plugin upgrade with a full reboot - https://phabricator.wikimedia.org/T277792 ** Update TextCat / Language ID documentation - https://phabricator.wikimedia.org/T271249 ** Unsustainable increases in Elasticsearch cluster disk IO - https://phabricator.wikimedia.org/T264053 ** selenium-daily-beta-CirrusSearch is failing - https://phabricator.wikimedia.org/T276571 ** Fix Flink Serialization Bug - https://phabricator.wikimedia.org/T276131 ** Investigate Resource Needs for Commons and Wikidata Elasticsearch indices - https://phabricator.wikimedia.org/T265914 ** Uncaught Error: Widget not found / Call to a member function getNsIndex() on null on CirrusSearch result page with internal error - https://phabricator.wikimedia.org/T277332 === Site Reliability Engineering === * Blocked by: ** Analytics on https://phabricator.wikimedia.org/T274262 * Blocking: * Thank yous: * Updates: ** === WMDE Technical Wishes === * Blocked by: * Blocking: * Thank yous: To WMF Analytics, especially MForns and AOtto for the code review, feedback, and many deployments on our behalf. * Updates: ** Currently working on the Kartographer map extension throughout prototyping week, in order to enable it on dewiki we will need to fix a bug with how map overlay content is linked to page revision. ** Two successful deployments today: enabling bracket matching for group0 wikis and wikitech, and an update to syntax highlighting colors everywhere. == Cross-cutting == * Blocked by: ** [long term] Search Platform: PHP 8.0 work is long-term blocked on the migration to ElasticSearch 7.0 https://phabricator.wikimedia.org/T263142 (or at least 6.7) * Blocking: ** None? * Thank yous: ** Daimona for their continued work on phan tooling. ** Antoine from RelEng for support on the stretch -> buster CI migration. * Updates: ** PHP 8.0 work is focussed on helping upstream provide forwards and backwards compatibility in Elastica-related PHP code; this is theoretically the last upstream blocker to PHP 8.0. ** CI tools' upgrade status: https://libraryupgrader2.wmcloud.org/status?branch=master [[Category:Scrum of scrums{{#translation:}}|*]]

1 0

Bracket matching in wikitext available on first wikis
by Johanna Strodt 24 Mar '21

24 Mar '21

/ sorry for cross-posting Hi, On a few first wikis[1], you can now highlight pairs of brackets in wikitext. For this to work, you need to turn on the syntax highlighting feature, which is part of the 2010 and 2017 wikitext editors. By placing your cursor next to or within a set of brackets, you can then match round, square and curly brackets. For more information about this feature please visit its project page.[2] Deployment to other wikis is planned for later this year. If your wiki community wants to get bracket matching now, please contact me. This change has been implemented by the Technical Wishes team who is currently working on several projects within the focus area "Make working with templates easier"[3]. Other projects in this focus area, including those for the Visual Editor, are in the making. Many thanks to all who have contributed to the realization of this project through comments, interviews and more. Feedback is, as always, welcome on the project's talk page.[4] Thanks, Johanna for the Technical Wishes team [1] dewiki, cawiki and trwiki [2] https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Bracket_Matching [3] https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Templates [4] https://meta.wikimedia.org/wiki/Talk:WMDE_Technical_Wishes/Bracket_Matching

1 2

eqiad kubernetes cluster upgrade on 2021-03-23
by Alexandros Kosiaris 24 Mar '21

24 Mar '21

Hello everyone, TL;DR if you are not deploying services to the eqiad kubernetes cluster, you can safely skip this. Long version: After having tested thrice our cluster reinitialization procedure, next week, on Tuesday 2021-03-23 we will be reinitializing our eqiad kubernetes cluster. All traffic will be drained from it beforehand and we expect no user visible impact. However, for the duration of the process, the kubernetes eqiad cluster will be unavailable to deployers and thus efforts to deploy to it will fail or worse, not have the expected outcomes. This is normal until SRE serviceops announces that the cluster is fully operational again. SRE service-ops will be deploying all services before marking the cluster as usable and pooling traffic back to it, so there will be no need for deployers to re-deploy their services. For your convenience the list of services that are currently deployed on that cluster is: apertium api-gateway blubberoid changeprop changeprop-jobqueue citoid cxserver echostore eventgate-analytics eventgate-analytics-external eventgate-logging-external eventgate-main eventstreams eventstreams-internal linkrecommendation mathoid mobileapps proton push-notifications recommendation-api sessionstore similar-users termbox wikifeeds zotero Regards, -- Alexandros Kosiaris Principal Site Reliability Engineer Wikimedia Foundation

2 3

Train vs Backport for Wikimedia Deployment
by Tyler Cipriani 23 Mar '21

23 Mar '21

Hi All! tl;dr: if you're a developer looking for guidance on how to deploy changes to Wikimedia's MediaWiki cluster read https://w.wiki/36nY ; if you have thoughts on our existing deployment documentation comment on https://w.wiki/36nZ --- Over time The Train™ has become the default way to deploy changes to Wikimedia's MediaWiki cluster -- for some patches that may not always be the right path. If a developer needs a change deployed *now*, or if there is a desire to deploy a change in isolation then backports might be a better path. As with all things, some exceptions may apply. The Release Engineering team has created some guidelines[0] that will hopefully help explain when something MUST, SHOULD, or MAY[1] be deployed via the train or via backport. This documentation is a bookmarkable quick reference for developers. It does not change our backport window guidelines[2] or special deployment window guidelines[3], those documents should not be in conflict with the advice in the new guidelines. The new guidelines target a different use-case. Our deployment documentation is up-to-date but sprawling. The same information is in multiple places and multiple audiences and use-cases are often mixed into the same documents. Work to improve deployment documentation is tracked on Phabricator[4]. Please reach out in #wikimedia-releng on freenode in IRC or attend the Deployment Office Hours meeting (weekly on Mondays at 17:00UTC in #wikimedia-office on freenode in IRC) if you have questions. Thank you! -- Tyler [0]: <https://wikitech.wikimedia.org/wiki/Deployments/Train_vs_backport> [1]: <https://tools.ietf.org/html/rfc2119> [2]: <https://wikitech.wikimedia.org/wiki/Backport_windows#Guidelines> [3]: <https://wikitech.wikimedia.org/wiki/Deployments/Inclusion_criteria> [4]: <https://phabricator.wikimedia.org/T273802>

2 2

Maps Modernization plan - FYI
by Erica Litrenta 23 Mar '21

23 Mar '21

Greetings, This is a follow up from our last email some months ago (and a crosspost). You may already have seen today's announcement from Legal about the upcoming changes to the Maps Terms of Use. Here is an extra heads-up that Wikimedia Maps are transitioning towards a more modern architecture. The first phase of this transition will be replacing Tilerator [0] with Tegola [1] as our vector tile server. This is a change in the Maps infrastructure, so there should be little to no impact to the end users’ experience. It is important that we are able to provide software that is sustainable to support, before we can guarantee a reliable user experience. Wikimedia Maps aim to provide Wikimedia users a consistent experience contributing to and learning about geoinformation. To achieve this goal, we will empower those engineers maintaining the Wikimedia Maps infrastructure to do so with ease and low effort. If you want to learn more, please head to mediawiki.org [2], where you will also find a Questions & Answers section. Thanks, and take care, Erica Litrenta (on behalf of the Product Infrastructure team) [0] https://wikitech.wikimedia.org/wiki/Maps/Tilerator [1] https://tegola.io/ [2] https://www.mediawiki.org/wiki/Wikimedia_Maps/2021_modernization_plan -- ------------------------------ Erica Litrenta (she/her) Manager, Community Relations Specialists Wikimedia Foundation <https://meta.wikimedia.org/wiki/User:Elitre_(WMF)>

3 2

Joining Trusted-Contributors
by Klaas Skelte van der Werf 23 Mar '21

23 Mar '21

I just noticed that I can no longer amend other peoples patch sets. I used to have this ability, but it appears that it was moved to Trusted-Contributors. [1] Can someone in that group add me? Thanks in advance. Mainframe98 [1] https://phabricator.wikimedia.org/T238651

2 2

Gerrit authentication fails
by Swathi Kasikala 22 Mar '21

22 Mar '21

Hello, This is Swathi new to this MediaWiki environment and really excited to contribute to it for which I already created a Wikimedia developer account using this https://wikitech.wikimedia.org/wiki/Special:CreateAccount and now when I am trying to login to Gerrit (https://gerrit.wikimedia.org/) using the same credentials the message is been displayed that the authentication failed. Can someone help me out to solve this issue, please-- Regards, Swathi Kasikala. amFOSS <https://amfoss.in/@swathi>

2 1

Production Excellence #29: February 2021
by Krinkle 22 Mar '21

22 Mar '21

How’d we do in our strive for operational excellence last month? Read on to find out! Read on Phabricator at https://phabricator.wikimedia.org/phame/post/view/228 📈 Incidents 3 documented incidents last month, [1] which is average for the time of year. [2] Learn about these incidents at Incident status <https://wikitech.wikimedia.org/wiki/Incident_status> on Wikitech, and their Preventive measures <https://phabricator.wikimedia.org/project/view/4758/> in Phabricator. For those with NDA-restricted access, there may be additional private incident reports <https://drive.google.com/drive/u/0/folders/1d2hlER4s_GPe0TWDqNfa2RQcYFuOdhOo> available. 💡 *Did you know: Our Incident reports have switched to using the ISO date format in their titles and listings, for improved readability and edit-ability (esp. when publishing on a later date). So long 202010221, and hello 2021-02-21!* ------- 📊 Trends In February we saw a continuation of the new downward trend that began this January <https://phabricator.wikimedia.org/phame/post/view/227/production_excellence…>, which came after twelve months of continued rising. Let's make sure this trend sticks with us as we work our way through the debt, whilst also learning to have a healthy week-to-week iteration where we monitor and follow-up on any new developments such that they don't introduce lasting regressions. The recent tally (issues filed since we started reporting in March 2019) is down to 138 unresolved errors, from 152 last month. The old backlog (pre-2019 issues) also continued its 5-month streak and is down to 148, from 160 last month. If this progress continues we'll soon have fewer "Old" issues than "Recent" issues, and possibly by the start of 2022 we may be able to report and focus only on our rotation through recent issues as hopefully we are then balancing our work such that issues reported this month are addressed mostly in the same month or otherwise later that quarter within 2-3 months. Visually that would manifest as the colored chunks having a short life on the chart with each drawn at a sharp downwards angle – instead of dragged out where it was building up an ever-taller shortcake. I do like cake, but I prefer the kind I can eat. 🍰 Month-over-month plots based on spreadsheet data <https://docs.google.com/spreadsheets/d/e/2PACX-1vTrUCAI10hIroYDU-i5_8s7pony…>. [3] [4] Figure 1, Figure 2: Unresolved error reports stacked by month. <https://phabricator.wikimedia.org/phame/post/view/228/production_excellence…> ------- 📖 Outstanding errors Summary over recent months: - ⚠️ July 2019 (2 of 18 issues left): *no change*. - ⚠️ August 2019 (1 of 14 issues): *no change*. - ⚠️ October 2019 (4 of 12 issues): *no change*. - ⚠️ November 2019 (1 of 5 issues): *no change*. - ⚠️ December 2019 (1 of 9 issues): One task resolved (-1). - ⚠️ January 2020 (2 of 7 issues): *no change*. - ⚠️ February 2020 (1 of 7 issues): *no change*. - ⚠️ March 2020 (2 of 2 issues): *no change*. - April 2020 (9 of 14 issues left): *no change*. - May 2020 (6 of 14 issues left): *no change*. - June 2020 (7 of 14 issues left): *no change*. - July 2020 (9 of 24 new issues <https://phabricator.wikimedia.org/maniphest/query/s__D8Kd0xuQH/#R>): *no change*. - August 2020 (20 of 53 new issues <https://phabricator.wikimedia.org/maniphest/query/hu1yhWu4sXkP/#R>): Two tasks resolved (-2). - September 2020 (9 of 33 new issues <https://phabricator.wikimedia.org/maniphest/query/CGFQViLShnOY/#R>): Five tasks resolved (-5). - October 2020 (26 of 69 new issues <https://phabricator.wikimedia.org/maniphest/query/MYnnBybPTYpd/#R>): Five tasks resolved (-5). - November 2020 (11 of 38 new issues <https://phabricator.wikimedia.org/maniphest/query/CkC_VqQq5VC0/#R>): Three tasks resolved (-3). - December 2020 (12 of 33 new issues <https://phabricator.wikimedia.org/maniphest/query/10NQy74iKaZJ/#R>): Seven tasks resolved (-7). - January 2021 (5 of 50 new issues <https://phabricator.wikimedia.org/maniphest/query/WIP9W8q54HB6/#R>): Two tasks resolved (-2). - *February 2021*: 11 of 20 new issues <https://phabricator.wikimedia.org/maniphest/query/5MzPJAb5oJgv/#R> survived the month and remained unresolved (+20; -9) Recent tally 152 issues open, as of Excellence #28 <https://phabricator.wikimedia.org/phame/post/view/227/production_excellence…> (16 Feb 2021). -25 issues closed since, of the previous 152 open issues. +11 new issues that survived Feb 2021. 138 issues open, as of today 5 Mar 2021. For the on-going month of March 2021, we've got 12 new issues <https://phabricator.wikimedia.org/maniphest/query/aDItN1Nm0ron/#R> so far. Take a look at the workboard <https://phabricator.wikimedia.org/tag/wikimedia-production-error/> and look for tasks that could use your help! ------- 🎉 Thanks! Thank you to everyone else who helped by reporting, investigating, or resolving problems in Wikimedia production. Thanks! Until next time, – Timo Tijhof ------- Footnotes: [1] Incident status Wikitech <https://wikitech.wikimedia.org/wiki/Incident_status>. [2] Wikimedia incident stats by Krinkle, CodePen <https://codepen.io/Krinkle/full/wbYMZK>. [3] Month-over-month, Production Excellence spreadsheet <https://docs.google.com/spreadsheets/d/e/2PACX-1vTrUCAI10hIroYDU-i5_8s7pony…> . [4] Open tasks, Wikimedia-prod-error, Phabricator <https://phabricator.wikimedia.org/maniphest/query/Fw3RdXt1Sdxp/#R>.

2 1

Upcoming change to the Wikimedia Maps Terms of Use
by Stephen LaPorte 22 Mar '21

22 Mar '21

Hi all, On April 5, 2020, we will update the Wikimedia Maps Terms of Use. Following the access limits announced in August 2020 [0] and implemented in October 2020 [1], the Wikimedia Maps service no longer supports third-party usage of Wikimedia map tiles. We are now updating the terms to reflect this policy. This change will not affect support for Wikimedia Maps on the Wikimedia projects, tools hosted on Wikimedia Cloud Services, or other allowed Wikimedia-affiliated domains. [2] The updated Maps Terms of Use is available here: https://meta.wikimedia.org/wiki/Revised_Maps_Terms_of_Use_(March_2021) If you have any suggestions, you can add them to the talk page: https://meta.wikimedia.org/wiki/Talk:Revised_Maps_Terms_of_Use_(March_2021) Under our Maps Terms of Use, we've committed to providing 14 days notice before changes are made to the policy. Best, Stephen [0] https://lists.wikimedia.org/pipermail/maps-l/2020-August/001729.html [1] https://phabricator.wikimedia.org/T261424 [2] https://phabricator.wikimedia.org/T261694 -- Stephen LaPorte (he/him/his) Legal Director Wikimedia Foundation *NOTICE: This message may be confidential or legally privileged. If you have received it by accident, please delete it and let us know about the mistake. As an attorney for the Wikimedia Foundation, for legal and ethical reasons, I cannot give legal advice to, or serve as a lawyer for, community members, volunteers, or staff members in their personal capacity. For more on what this means, please see our legal disclaimer <https://meta.wikimedia.org/wiki/Wikimedia_Legal_Disclaimer>.*

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l March 2021