Wikitech-l February 2021

wikitech-l@lists.wikimedia.org

42 participants
55 discussions

2021-02-10 scrum of scrums meeting notes
by Grace Gellerman 11 Feb '21

11 Feb '21

https://www.mediawiki.org/wiki/Scrum_of_scrums/2021-02-10 = 2021-02-10 = == Callouts == * RelEng: Trying to be a bit more strict about log messages, see Greg's email: https://lists.wikimedia.org/pipermail/wikitech-l/2021-February/094250.html * RelEng: Blocked on serviceops: [https://phabricator.wikimedia.org/T274306 docker-pkg: "certificate verify failed: unable to get local issuer certificate" for docker-registry.discovery.wmnet when publishing dev-images from contint2001] === No updates === Community Tech, Anti-Harassment, Parsing, Language, Library === '''No notes provided''' === Editing, Growth, iOS, Android, Product Infrastructure, Inuka, Analytics, Cloud Services, Platform, Quality & Test == SoS Meeting Bookkeeping == * Updates: ** from retro ideas to try: *** Bolding items to read allowed +JF +TC *** relaxing the start time *** Template **** Adding a section where teams can add Gerrit patches or GitHub Pull Requests to request reviews or feedback. +GG +TC **** Perhaps add a contact point (email, url, office hours, whatever) for easy reaching out to teams when a bullet point seems interesting. +GG +JF +TC **** Adding a "thanks" section +GG +JF +TC+AP == Product == === Web === * Blocked by: * Blocking: * Updates: ** Security review of WVUI component library is back with Security for discussion ** Logging Vue.js component errors to the error intake system (again): https://phabricator.wikimedia.org/T249826 ** Language button treatment and associated A/B test: *** https://phabricator.wikimedia.org/T260738 *** https://phabricator.wikimedia.org/T268241 *** https://phabricator.wikimedia.org/T268504 === Structured Data === * Blocked by: * Blocking: * Updates: ** MediaSearch: working on supporting complex queries, instrumenting events, improving the CSS-only grid ** Image recommendations: using machine learning to improve mediasearch ( https://phabricator.wikimedia.org/T271803), translating data analysis from Research team into an elastic search profile and query builder ( https://phabricator.wikimedia.org/T271799) ** Continuing SDAW architecture discussions === Abstract Wikipedia === * Blocked by: ** None * Blocking: ** None * Updates: ** Working on phase gamma of development plan: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Phases *** Creation of Functions coming soon; will need a code editor Vue UX component. *** First route for back-end executor service is landing soon. *** Also making WikiLambda changes as recommended by Platform; thanks again. ** Special thanks to Dave Pifke from Performance for their support and advice this & last week. ** Community suggestions for the logo concept for Wikifunctions continues: https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Wikifunctions_logo_conce… === Vue.js === * Blocked by: * Blocking: * Updates: ** Nearly done with updates to MediaWiki's JS minifier == Technology == === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Fixing truncated order IDs https://phabricator.wikimedia.org/T274301 ** Fixing another error handling case for nightly recurring charge job https://phabricator.wikimedia.org/T274361 ** Prepping for CIviCRM db encoding update https://phabricator.wikimedia.org/T273704 ** More refinements to docker images for fundraising development https://phabricator.wikimedia.org/T262975 https://phabricator.wikimedia.org/T273894 === Engineering Productivity === ==== Performance ==== * Blocked by: * Blocking: * Updates: ** FOSDEM hasn't published our devroom videos yet, but here they are: https://drive.google.com/drive/folders/1OIp-3XUdtgWykK7sOxw8-k3icv8BWDcY?us… ==== Release Engineering ==== * Blocked by: ** Serviceops: [https://phabricator.wikimedia.org/T274306 docker-pkg: "certificate verify failed: unable to get local issuer certificate" for docker-registry.discovery.wmnet when publishing dev-images from contint2001] *** Have checked the obvious stuff, but not really sure how to proceed. * Blocking: * Updates: ** Jessie-based node 6 / npm 3 CI jobs have been dropped (original deadline was April 2019). Specifically, this means that 3d2png and Maps no longer have CI; any relevant teams may wish to fix. ** Train: *** Last week: 1.36.0-wmf.29 [[phab:T271342]] was abandoned.  *** This week: 1.36.0-wmf.30 [[phab:T271343]] underway.  *** Next week: 1.36.0-wmf.31 [[phab:T271344]]  === Search Platform === * Blocked by: ** No one * Updates: ** Blazegraph journal for wcqs is too big - https://phabricator.wikimedia.org/T273636 ** Deleted item still gets shown in WDQS query results - https://phabricator.wikimedia.org/T272120 ** Programmatically categorize WDQS queries by potential alternative solution -https://phabricator.wikimedia.org/T266022 ** wikimedia-event-utilities should provide tools for JVM based apps producing directly to kafka -https://phabricator.wikimedia.org/T270371 ** Reboot wdqs hosts for kernel upgrade - https://phabricator.wikimedia.org/T274213 === Security === * Blocked by: ** none * Blocking: * Updates: ** Updated SOP for Security Reviews: https://www.mediawiki.org/wiki/Security/SOP/Security_Readiness_Reviews ** Current queue and reviews in progress: https://phabricator.wikimedia.org/tag/secscrum/ === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** None * Updates: ** Working with PI to get linkrecommendation updated in a structured fashion ** maps architectural discussions ongoing == Cross-cutting == * Blocked by: ** None * Blocking: ** None * Updates: ** It's quiet. ** No major progress on PHP 8.0 work. ** CI tools' upgrade status is mostly stable: https://libraryupgrader2.wmcloud.org/status?branch=master [[Category:Scrum of scrums{{#translation:}}|*]]

1 0

Where are the logs?
by Arthur Smith 11 Feb '21

11 Feb '21

I've been using mediawiki in the docker containers for a while, and I just did a git pull to the latest, replaced the containers I was running etc, and now I can't figure out where errors are being logged to or how I'm supposed to see them. Help! This may be related to the new '.env' file that seems to be required, which sets a MW_DOCKER_UID and MW_DOCKER_GID value... If I run docker-compose logs I do see some "Permission denied" errors for files in /etc/php/7.2/cli and /etc/php/7.2/fpm but those don't seem to be log files exactly (20-xdebug.ini files ?) If I do docker-compose exec mediawiki bash I get a login shell on the container and I can look around, but there are no mediawiki log files in /var/log where I would expect them, and again I don't have permission to create a file there. Also if I docker exec into the web container I don't have permission to look at the apache logs (but those do exist). If I try to add a $wgDebugLogFile to LocalSettings.php the mediawiki instance completely collapses, I can't look at anything, and I see no errors explaining what's going on! Help! Arthur

3 4

Want to Contribute
by Puneet Dwivedi 10 Feb '21

10 Feb '21

2 1

Re: [Wikitech-l] [Wikimedia-l] [REMINDER] Call for projects and mentors for Google Summer of Code 2021 and Outreachy Round 22 is ongoing!
by Srishti Sethi 10 Feb '21

10 Feb '21

Hi all, We are still looking for more projects for GSoC/ Outreachy. We are currently working on putting together a list of projects to submit along with our mentoring organization application to Google for GSoC next week. If you are interested in mentoring and have a project idea in mind, drop a note here <https://phabricator.wikimedia.org/T270429> [1]. Best, Srishti [1] https://phabricator.wikimedia.org/T270429 *Srishti Sethi* Senior Developer Advocate Wikimedia Foundation <https://wikimediafoundation.org/> On Tue, Jan 26, 2021 at 10:54 PM Pavithra Eswaramoorthy < pavithra.e(a)hotmail.com> wrote: > Hello everyone, > > TLDR; Wikimedia will soon be applying as a mentoring organization to > Google Summer of Code 2021 <https://summerofcode.withgoogle.com> [1] and > Outreachy Round 22 <https://www.outreachy.org/> [2]. The application > submission deadline for GSoC is February 20th, and Outreachy is March 7th. > We are currently working on a list of interesting project ideas to include > in the application. If you have some ideas for coding or non-coding > (design, documentation, translation, outreach, research) projects, share > them here: https://phabricator.wikimedia.org/T270429 [3]. > > > Timeline > > As a mentor, you will be engaging potential candidates in the application > period for GSoC between March 10th – April 13th, and for Outreachy between > March 15th – April 16th. During this time, you will help candidates make > small contributions to your project and answer any project related queries. > You will be working more closely with the accepted candidates during the > coding period between May-August. > > > Project ideas > > We have started compiling a list of projects that you can take a look at > here: > https://www.mediawiki.org/wiki/Google_Summer_of_Code/2021 [4], > https://www.mediawiki.org/wiki/Outreachy/Round_22 [5] > > If you don’t have an idea in mind and would like to pick one from an > existing list, check out these projects: > https://phabricator.wikimedia.org/tag/outreach-programs-projects/ [6] > > Through GSoC, you can mentor only coding but with Outreachy also > non-coding projects (including design, documentation, translation, > outreach, etc.). > > Note that GSoC has some changes this year, including: > > - Smaller project size ~175 hr project (previously 350 hr) > - Shortened coding period ~10 weeks long (previously 3 months) > - Eligibility criteria redefined; the program is now open to > candidates participating in a variety of academic programs (previously > accredited university programs only) > > Please keep these changes in mind while sharing your project ideas. > > > Some tips for proposing projects > > - Follow this task description template when you propose a project in > Phabricator: > https://phabricator.wikimedia.org/tag/outreach-programs-projects [7]. > Add #Google- Summer-of-Code (2021) or #Outreachy (Round 22) tag to it. > - Remember, the project should require an experienced developer ~15 > days to complete and a newcomer ~3 months for Outreachy and ~10 weeks for > GSoC. > - Each project should have at least 2 mentors, and one of them should > hold a technical background. > - When it comes to picking a project, you could propose one that is: > - Relevant for your language community or brings impact to the > Wikimedia ecosystem in the future. > - Welcoming and newcomer-friendly and has a moderate learning curve. > - A new idea you are passionate about, there are no deadlines > attached to it; you always wanted to see it happen but couldn't due to lack > of resources! > - About developing a standalone tool (possibly hosted on Wikimedia > Toolforge), with fewer dependencies on Wikimedia's core infrastructure, and > doesn't necessarily require a specific programming language, etc. > > To learn more about the roles and responsibilities of a mentor, visit our > resources on MediaWiki.org: > https://www.mediawiki.org/wiki/Outreachy/Mentors [8], > https://www.mediawiki.org/wiki/Google_Summer_of_Code/Mentors [9]. > > Cheers, > GSoC 2021 and Outreachy Round 22 Administrators > > [1] https://summerofcode.withgoogle.com > [2] https://www.outreachy.org/ > [3] https://phabricator.wikimedia.org/T270429 > [4] https://www.mediawiki.org/wiki/Google_Summer_of_Code/2021 > [5] https://www.mediawiki.org/wiki/Outreachy/Round_22 > [6] https://phabricator.wikimedia.org/tag/outreach-programs-projects/ > [7] https://phabricator.wikimedia.org/tag/outreach-programs-projects/ > [8] https://www.mediawiki.org/wiki/Outreachy/Mentors > [9] https://www.mediawiki.org/wiki/Outreachy/Mentors > > _______________________________________________ > Wikimedia-l mailing list, guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > New messages to: Wikimedia-l(a)lists.wikimedia.org > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe> >

1 0

tox upgraded on CI from 3.10 to 3.21.4
by Antoine Musso 10 Feb '21

10 Feb '21

Hello, For python projects, tox has been upgraded from 3.10 to 3.21.4. That was notably to address an issue when python3.9 is defined as an environment although it is not available in our image, tox would fallback to whatever default python 3 instead of failing (T274232). You can read the full tox changelog at: https://tox.readthedocs.io/en/latest/changelog.html If there is any issue, please file a task in Phabricator against #continuous-integration-config -- Antoine "hashar" Musso

1 0

GlobalWatchlist enabled on testwiki
by Danny S 10 Feb '21

10 Feb '21

Dear all, The GlobalWatchlist extension[1][2] is now deployed on testwiki. If you're interested, feel free to try it out.[3] It will be deployed to metawiki on Thursday[4] for general use. I developed the extension to replace a user script I wrote[5], funded through a grant from the WMF[6], and the deployment to meta follows 8 months of work on the extension, and an additional year of work on the script.[7] Please report any bugs found on phabricator. Thanks, -DannyS712 [1] https://www.mediawiki.org/wiki/Extension:GlobalWatchlist [2] https://gerrit.wikimedia.org/g/mediawiki/extensions/GlobalWatchlist [3] https://test.wikipedia.org/wiki/Special:GlobalWatchlist [4] https://phabricator.wikimedia.org/T260862 [5] https://meta.wikimedia.org/wiki/User:DannyS712/Global_watchlist.js [6] https://meta.wikimedia.org/wiki/Grants:Project/DannyS712/Create_a_global_wa… [7] I started the user script on 14 June 2019, with the first working version published on 21 June. I began work on the grant on 10 June 2020.

2 1

[Train] 1.36.0-wmf.30 status -- wmf.28, wmf.29: abandoned
by Tyler Cipriani 10 Feb '21

10 Feb '21

Hi all All wikis except testwikis are on 1.36.0-wmf.27; testwikis are running 1.36.0-wmf.30. We've rolled back to wmf.27 so that we have a stable base version from which to roll out wmf.30. We will proceed with rollout of wmf.30 once this cherry-pick for wmf.30 is code-reviewed: https://gerrit.wikimedia.org/r/662965 assistance appreciated! Thank you! -- Tyler

1 0

[Train] 1.36.0-wmf.29 to group0/1 at 14:00 UTC
by Antoine Musso 09 Feb '21

09 Feb '21

Hello, The 1.36.0-wmf.29 update had been blocked on a caching issue apparently caused by the FeaturedFeeds extension: * MemcachedPeclBagOStuff: Serialization of 'Closure' is not allowed - https://phabricator.wikimedia.org/T273242 The fix got reviewed and backported today and I will thus promote 1.36.0-wmf.29 to group 0 and immediately after to group 1 starting at 14:00 UTC today (15:00 CET). If all goes well, I guess I can try to promote all the remaining wiki as well or we might do it later today (at the usual 20:00 UTC). -- Antoine "hashar" Musso Wikimedia Release Engineering

1 1

A clearer line on logspam
by Greg Grossmeier 08 Feb '21

08 Feb '21

Hi all! tl;dr: There's a large backlog of production errors. Release Engineering is blocking the train for any new logspam. Your help is needed! A quick update on the deployment train: In the process of rolling out wmf/1.36.0-wmf.28 there were a number of issues that prevented us from rolling forward the train in a timely manner. After the issues were resolved and backports deployed to the current version in production (wmf/1.36.0-wmf.27), we realized there were a few remaining spammy log messages and blocked the following week's train on those issues. Release Engineering has long blocked the train on logspam issues[0]. Even when it does not indicate user-facing errors, logspam of any kind makes it harder for us to see real problems. We have, however, defaulted to pushing forward the train despite minor issues. Under this custom, many log messages have been accepted as "just occassional, not a big deal" or "yeah, we'll fix that eventually... it's not a big deal". Frequently, "eventually" never arrives. This results in an unmanageable accumulation of exceptions (see the ever-growing list of exceptions in the Wikimedia-production-error workboard[1] and logstash[2]). To deal with these issues we are now, as a matter of policy, blocking trains that cause any new error messages. In most cases new errors are the result of code changes that lack defensive coding practices and/or have unexpected interactions with other code. The best resolution in these cases is for the code to be fixed or reverted. Release Engineering organises a weekly "train log triage" meeting, on Wednesdays at 19:00 UTC, where we invite people who develop MediaWiki to help triage log messages. As of this week, there is also a second one, on Thursdays at 10:00 UTC, to be more suitable for people in EU time zones. We invite everyone who develops MediaWiki or its extensions to join one of the meetings each week. Thank you, Greg [0]: < https://wikitech.wikimedia.org/wiki/Deployments/Holding_the_train#Logspam> [1]: https://phabricator.wikimedia.org/project/view/1055/ [2]: https://logstash.wikimedia.org/app/dashboards#/view/0a9ecdc0-b6dc-11e8-9d8f… -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | Dir. Engineering Productivity A18D 1138 8E47 FAC8 1C7D |

1 0

[Train] 1.36.0-wmf.29 status update (still blocked, but only one blocker remaining!)
by Brennen Bearnes 05 Feb '21

05 Feb '21

Hi all, After considerable effort this week, we're down to one blocker for the 1.36.0-wmf.29[0] train: * MemcachedPeclBagOStuff: Serialization of 'Closure' is not allowed - https://phabricator.wikimedia.org/T273242 Thanks in advance for any help resolving this issue! Once resolved, the train can resume Monday (European) morning. -- Brennen, on behalf of your faithful train crew [0]. <https://phabricator.wikimedia.org/T271343>

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l February 2021