Wikitech-l March 2021

wikitech-l@lists.wikimedia.org

58 participants
58 discussions

Deployment calendar format change
by Tyler Cipriani 29 Mar '21

29 Mar '21

tl;dr: The deployment calendar format will change in 2 weeks (2021-04-05) to make it easier to edit with visual editor https://w.wiki/a3b I updated the deployment calendar for the week of 2021-04-05[0] to use a different format than in the past (compare to next week[1]). My hope is that this new format will make it much easier to schedule deployment windows and to schedule patches for backports using Visual Editor. Also, selfishly, less squinting at Wikitext for me :) All credit for the new format goes to Timo Tijhof. Thank you Timo! Thanks all! -- Tyler [0]: <https://wikitech.wikimedia.org/wiki/Deployments#Week_of_April_05> [1]: <https://wikitech.wikimedia.org/wiki/Deployments#Week_of_March_29>

4 3

Test Mailman3 in production
by Amir Sarabadani 26 Mar '21

26 Mar '21

Hello, tl;dr: https://lists-next.wikimedia.org is running mailman3. Please help us test the software before we upgrade the real mailing list server. Kunal and I have been working on deploying the new mailman (version 3) to replace mailman2 serving https://lists.wikimedia.org and powering all of our mailing lists. Mailman2 is a dinosaur that should have gone extinct years ago. Pretty old user interface (especially for admins and moderators), storing passwords in plain text, lack of any database (everything is file on disk), pretty old code, lack of ability to search in archives or send email from web interface, running on EOL python (python2), encoding issues with non-Latin languages, hard to redact archives, and the list goes on and on. The new version has been developed/puppetized/tested in the Cloud and is now ready for proper testing! Give it a try: https://lists-next.wikimedia.org. We have created some mailing lists you can join and can test. If you want to test the experience as a list administrator/moderator, we can give those permissions out as well. WARNING: All data on the lists-next server will be deleted after the test period is over. We will also need help updating documentation on wikis and elsewhere. If you find any bugs/issues (yay!), please file a ticket in the “Wikimedia-Mailing-lists” Phabricator project and we’ll check it out. In the coming days/weeks will also import some public mailing lists from the old version to the new version to check archive size, search index size, and other aspects. There are other TODOs left as well like monitoring, logging, anti-abuse, etc. Slowly and after testing (hopefully soon), we expect to deploy this on lists.wikimedia.org and mailing lists one by one or in batches can be upgraded to the 21st century. The overall task tracking this project is T52864 <https://phabricator.wikimedia.org/T52864> and a big thank you people who are helping this move forward. Regards, Kunal and Amir

4 4

Stuck/Missing Grid Job for tools.william-avery-bot
by William Avery 26 Mar '21

26 Mar '21

Hi, I got the email below telling me that my cron job running as william-avery-bot had throw an error, and I noticed that the Grid job that it kicks off hasn't run since. I tried deleting the job using the instructions at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid#Stopping_jobs_with_… but it appeared "stuck". "qstat -xml" outputs the following: <?xml version='1.0'?> <job_info xmlns:xsd=" http://arc.liv.ac.uk/repos/darcs/sge/source/dist/util/resources/schemas/qst… "> <queue_info> <job_list state="running"> <JB_job_number>9999749</JB_job_number> <JAT_prio>0.25319</JAT_prio> <JB_name>cron-TaxonbarSyncerBot</JB_name> <JB_owner>tools.william-avery-bot</JB_owner> <state>dr</state> <JAT_start_time>2021-03-25T17:49:16</JAT_start_time> <queue_name>task(a)tools-sgeexec-0916.tools.eqiad.wmflabs</queue_name> <slots>1</slots> </job_list> </queue_info> <job_info> </job_info> </job_info> But when I ssh to tools-sgeexec-0916.tools.eqiad.wmflabs I see no sign of any processes under tools.william-avery-bot, except the ones associated with my interactive session. Can anyone help resolve this or advise of a venue to raise it? Thanks in advance, Will ---------- Forwarded message --------- From: Cron Daemon <root(a)tools.wmflabs.org> Date: Thu, 25 Mar 2021 at 16:49 Subject: Cron <tools.william-avery-bot@tools-sgecron-01> /usr/bin/jsub -N cron-TaxonbarSyncerBot -once -quiet ~/TaxonbarSyncerBot.sh To: <tools.william-avery-bot(a)tools.wmflabs.org> error: commlib error: got select error (Connection refused) error: unable to send message to qmaster using port 6444 on host "tools-sgegrid-shadow.tools.eqiad1.wikimedia.cloud": got send error Traceback (most recent call last): File "/usr/bin/job", line 48, in <module> root = xml.etree.ElementTree.fromstring(proc.stdout.read()) File "/usr/lib/python3.5/xml/etree/ElementTree.py", line 1345, in XML return parser.close() xml.etree.ElementTree.ParseError: no element found: line 1, column 0

2 2

Wikimedia Developer Satisfaction Survey - 2021!
by Greg Grossmeier 26 Mar '21

26 Mar '21

Hello! tl;dr: It's time for the annual Developer Satisfaction Survey! *This survey is for everyone who uses any of the tools or services mentioned below*, whether you would call yourself a "developer" or not. *Closing date is March 24th.* *The survey*: https://forms.gle/xj2jNUcP7kmSgtwV8 Longer: The Wikimedia Foundation is soliciting your feedback to measure developer satisfaction and determine where to invest resources in the future. This is the third iteration of this survey. We want feedback from everyone who uses any of the below tools or services, whether you call yourself a "developer" or not. Topics covered include: * Local Development Environment * Beta Cluster / Staging Environment * Testing / CI * Code Review * Deployments * Account Management and Onboarding * Observability * Phabricator * Developer documentation * General Feedback We are soliciting feedback from all Wikimedia developers, including Staff, 3rd party contributors and volunteer developers. *The survey will be open for a little over 2 weeks, closing on Wednesday March 24th.* NOTE: This survey will be conducted via a third-party service, which may be subject to additional terms. For more information on privacy and data-handling, see the survey privacy statement https://foundation.wikimedia.org/wiki/Developer_Satisfaction_Survey_Privacy… *To participate in this survey, please start here:* https://docs.google.com/forms/d/e/1FAIpQLSfJCzxixT1wHCIgI5PijIMQvcIqDbsDTYb… Thank you for your participation, Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Dir. Engineering Productivity A18D 1138 8E47 FAC8 1C7D |

2 3

GSOC-2021 participant Introduction
by SULABH BANSAL 26 Mar '21

26 Mar '21

Hi everyone my name is Sulabh Bansal and I am a final year B.Tech computer science and engineering student. My specialization lies in the field of cloud and DevOps. And I am comfortable with Java, Html, CSS, Javascript, Git, Github, Docker, Mysql as well as cloud platforms. I am also a Red Hat Certified System administrator and ansible specialist. I would like to contribute to Wikimedia Foundation and looking forward to GSoC 2021. I would like to join this community and gain a better experience in terms of learning and sharing. Thank you.

1 0

Summary of this week's deployment of 1.36.0-wmf.36
by Lars Wirzenius 26 Mar '21

26 Mar '21

This is a summary of this week's deployment of the 1.36.0-wmf.36 branch of MediaWiki and its extensions. The primary person in charge this week is Antoine Musso, with Ahmon Dancy as backup, both from the Release Engineering team. The summary task for this week is https://phabricator.wikimedia.org/T274940 This week's deployment was not successful. The train got to group 2, but had to be rolled back all the way to group 0, which is where it is at right now. The current problems are: - T278478: FlaggedRevs: PHP Notice: Undefined index: status https://phabricator.wikimedia.org/T278478 - T278429: Translate\PageTranslation\ParserOutput::sourcePageTextForRendering() must be an instance of Language, instance of StubUserLang given https://phabricator.wikimedia.org/T278429 - T277795: User not found by actor ID: [id] https://phabricator.wikimedia.org/T277795 The general policy is to NOT deploy on Fridays. If we deploy something on a Friday and the sites break on Saturday, multiple people may need to be alerted to diagnose and fix or work around the problem. Thus, when fixes are found, we tend to deploy them on Monday instead. Then, on Tuesday, the next train can start. Issues that have already been dealt with along the week: - T277722: TypeError: this._languageCodes is undefined at getLanguageNameMap https://phabricator.wikimedia.org/T277722 - T278375: Class 'GlobalUsageHooks' not found https://phabricator.wikimedia.org/T278375 - T278379: Argument 1 passed to ProofreadPage\Index\IndexTemplateStyles::__construct() must be an instance of Title, null given, called in /srv/mediawiki/php-1.36.0-wmf.36/extensions/ProofreadPage/includes/Page/PageContent.php on line 284 https://phabricator.wikimedia.org/T278379 - T278400: PHP Warning: array_key_exists(): The first argument should be either a string or an integer https://phabricator.wikimedia.org/T278400 We saw some people highlight especially risky patches in the week's summary task. That was very welcome! Thank you, Amir Sarabadani and Roan Kattouw. As usual, a whole bunch of people helped to find, triage, analyze, fix, or work around problems this. Release Engineering thanks everyone, without help we wouldn't be able to deploy MediaWiki. - Abijeet Patro - Adam_WMDE - Amir Sarabadani - Asartea - Daniel Kinzler - DannyS712 - James Forrester - Johan Jönsson - Jon Robson - Klaas Skelte van der Werf - Max Binder - Niklas Laxström - Pchelolo - Roan Kattouw - Silvan_WMDE - Sohom Datta - Timo Tijhof - Tpt - Urbanecm There may have been other people, and if so, we apologize for not including them on the list above. For more information, please see: - https://phabricator.wikimedia.org/T274940 - https://wikitech.wikimedia.org/wiki/Heterogeneous_deployment/Train_deploys

1 0

Pywikibot login session period
by Shrinivasan T 26 Mar '21

26 Mar '21

Hello all, I am exploring pywikibot to write few bots. When importing it as a library in my python code, on initial execution it asks to login to the relevant wiki site. Then it works as expected. How long this login session is stored? Is one time login enough? Will it ask to login after some time? How to run this in a daily cron job on any remote server, if the session persistence is some limited time. Share some thoughts on this. Thanks. Shrini

2 1

[Train] 1.36.0-wmf.36 rolled back from group 1/2
by Antoine Musso 25 Mar '21

25 Mar '21

Hello, The 1.36.0-wmf.36 version of MediaWiki is blocked. The few blockers from Wednesday got addressed and we rolled to group 1 again. Today I have deployed to group 2 and then finally rolled back BOTH group 1 and 2. The new blockers: * FlaggedRevs: PHP Notice: Undefined index: status https://phabricator.wikimedia.org/T278478 Impact unknown to me. It might well cause user facing issue on enwiki. That is worth rolling back group 2. * Translate\PageTranslation\ParserOutput::sourcePageTextForRendering() must be an instance of Language, instance of StubUserLang given https://phabricator.wikimedia.org/T278429 Which I filed earlier today as part of log triage. We did not immediately recognized it prevented edits on pages using Translate. Notably on metawiki. That is worth rolling back group 1. * User not found by actor ID: [id] https://phabricator.wikimedia.org/T277795 Which is from last week. It still had at least one occurrence for the brief time we had 1.36.0-wmf.36 on all wikis. Once these issues are resolved the train can resume and we can try pushing 1.36.0-wmf.36 to all wikis. 1.36.0-wmf.36 general task: https://phabricator.wikimedia.org/T274940 Thank you for your assistance. -- Antoine "hashar" Musso

1 0

[Train] 1.36.0-wmf.36 blocked
by Antoine Musso 24 Mar '21

24 Mar '21

Hello, The 1.36.0-wmf.36 version of MediaWiki is blocked. Immediately after pushing it to group 1 wikis, it encountered three blockers: * Class 'GlobalUsageHooks' not found https://phabricator.wikimedia.org/T278375 * Constructing RevisionRecord for a page that can't exist: Special:MyLanguage/Main Page [Called from MediaWiki\Revision\MutableRevisionRecord::__construct] https://phabricator.wikimedia.org/T278376 * Argument 1 passed to ProofreadPage\Index\IndexTemplateStyles::__construct() must be a Title, null given. https://phabricator.wikimedia.org/T278379 The later had the most log entries and apparently prevents page from being submitted on wikisources wiki. Once these issues are resolved the train can resume and we can try pushing to group 1 wikis again. 1.36.0-wmf.36 general task: https://phabricator.wikimedia.org/T274940 Thank you for your assistance. -- Antoine "hashar" Musso

1 1

Forthcoming failover of irc.wikimedia.org to a new server (2021-03-25)
by Moritz Mühlenhoff 24 Mar '21

24 Mar '21

Hi, the DNS name for irc.wikimedia.org [1] will be redirected to a new instance (irc2001.wikimedia.org) tomorrow (European) morning. This new instance is running a more recent OS (Debian 10/Buster). Mediawiki changes are currently broadcasted to both the old server (kraz.wikimedia.org) and the new irc2001.wikimedia.org. As such, the failover should be transparent, unless for some reason a bot has hardcoded kraz.wikimedia.org or it's IP address 208.80.153.44 (which will then need to be updated to resolve irc.wikimedia.org). The old instance will be around for another week. If there are any issues, please report them at https://phabricator.wikimedia.org/T224579 Cheers, Moritz Footnotes: [1] https://wikitech.wikimedia.org/wiki/Irc.wikimedia.org

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l March 2021