Wikitech-l January 2021

wikitech-l@lists.wikimedia.org

35 participants
45 discussions

Scrum of scrums/2021-01-27
by Deb Tankersley 27 Jan '21

27 Jan '21

Hello, This email contains updates for January 27, 2021 <https://www.mediawiki.org/wiki/Scrum_of_scrums/2021-01-27>. Cheers, Deb Callouts - Search is blocked by [Analytics] merge of the side-output schema and wikimedia-event utilities https://phabricator.wikimedia.org/T270371 / https://phabricator.wikimedia.org/T269619 No updates - Community Tech, Anti-Harassment Tools, Editing, iOS native app, Android native app *No notes provided* - Growth, Parsing, Language, Inuka, Cloud Services, Fundraising Tech, Platform, Quality and Test Engineering, Security ProductWeb - Updates: - Page Previews is now requesting larger thumbnails to fix blurry thumbnails on low-DPR screens: https://phabricator.wikimedia.org/T272169 (see patch: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Popups/+/658493 ) Product Infrastructure - Updates: - Client-side error logging was enabled on enwiki: https://phabricator.wikimedia.org/T255585 Structured Data - Updates: - Preparing for security review or MediaSearch UI, which will be moved into a separate extension - Focus on tackling remaining blockers for a wider release of MediaSearch - Figuring out how to wire up the MediaSearch elastic queries into a Learning To Rank machine learning model Abstract Wikipedia - Updates: - Vue refactoring to align our code with emerging standards continues; thanks to Vue team! - Thanks to Daniel for advice on how our code may want to change to be closer to Platform future plans, and forthcoming meetings to explore further. - Still working on programmatic type management. - Community contest to select the logo concept for Wikifunctions is currently running; ideas welcome! Library - Updates: - Starting work on setting up an exception logging service (Glitchtip) - Added a new partner in the Library Bundle - Starting a bug fix that will prevent being spammed by The Wikipedia Library Extension Vue.js - Updates: - Working on ES6 minification for ResourceLoader. - [Probably other updates James doesn't know about.] TechnologyAnalytics - Updates: - The machines for our backup Hadoop cluster are racked, we are setting them up, testing them, and moving backup data to them. After that, we'll be able to move forward with cluster migration to BigTop. - We continue with the EventLogging legacy schema migration to EventPlatform. - Working on calculation of session length metric. - We improved HDFS data security through better conventions on user groups and permits. - We're finalizing the corrections to canonical pageview dumps. Engineering ProductivityPerformance - Updates: - Kobiton has provided disappointing service for our mobile device lab, we're going to do a trial with Bitbar - Published two short videos on officewiki: - Why web performance matters at Wikimedia https://w.wiki/w7z - The role of the Performance Team https://w.wiki/w7$ - Speaking tomorrow in French and English at We Love Speed: https://www.welovespeed.com/en/2020/ Release Engineering - Updates: - [All] Deployments/Covid-19 https://wikitech.wikimedia.org/wiki/Deployments/Covid-19 - Train Health - Last week: 1.36.0-wmf.27 phab:T271341 <https://phabricator.wikimedia.org/T271341> - This week: 1.36.0-wmf.28 phab:T271342 <https://phabricator.wikimedia.org/T271342> - Next week: 1.36.0-wmf.29 phab:T271342 <https://phabricator.wikimedia.org/T271342> Search Platform - Blocked by: - [Analytics] merge of the side-output schema and wikimedia-event utilities https://phabricator.wikimedia.org/T270371 / https://phabricator.wikimedia.org/T269619 - Blocking: - N/A - Updates: - Failing HTTP check on WDQS servers after latest deployment - https://phabricator.wikimedia.org/T272713 - SPARQL-Query shows entries, which should be filter out; number of entries in result set might change when executed repeatedly (possible caching/indexing problem) - https://phabricator.wikimedia.org/T267175 - Extract a list of the 200 most viewed black historical figures from WDQS -https://phabricator.wikimedia.org/T272447 Site Reliability Engineering - Updates: - similar-users, linkrecommendation services exposing to internal services being finalized this week. Cross-cutting - Updates: - No significant movement on PHP 8.0 this week. - LibraryUpgrader is bumping all reports to eslint-config-wikimedia 0.18.0, which has proper JSON linting support - There are some speed issues on very long JSON files, so we worked with upstream to avoid this, and have released 0.18.1 with said fix just now. - If your repo doesn't get a patch automatically, it's probably because LibraryUpgrader errors on your repo. We'll fix these manually (or file Phab tasks), but if you want to do it ahead of us, we won't complain. You can see the dashboard of such repos here: https://libraryupgrader2.wmcloud.org/errors?branch=master -- deb tankersley (she/her) senior program manager, engineering Wikimedia Foundation

1 0

[MediaWiki-Docker] Splitting PHP-FPM and httpd containers
by Brennen Bearnes 26 Jan '21

26 Jan '21

Hey folks, tl;dr: We're splitting PHP & HTTP containers in the MediaWiki-Docker[0] development environment. If you're not currently using MediaWiki-Docker, you can safely ignore this message. --- We're planning to merge a change to the MediaWiki-Docker environment to split PHP-FPM into a separate container from Apache.[1] This should improve build efficiency for these images, and remove the need to duplicate so much stuff (Apache, etc.) between versions of PHP. It also unblocks support for PHP 7.3/7.4, and upgrades XDebug to the 3.x series. What you'll need to change in existing setups: In docker-compose-override.yaml: * Linux users should specify a MW_DOCKER_UID & MW_DOCKER_GID for all containers. In .env: * Set XDEBUG_ENABLE=true and XHPROF_ENABLE=true if you want the corresponding extensions turned on. * If you have an XDEBUG_CONFIG set, it may need updated to reflect new configuration value names.[2] Finally, I'd like to get it out the door this week, since it unblocks a number of requested improvements, but feedback is of course welcome in Gerrit.[1] [0]. https://www.mediawiki.org/wiki/MediaWiki-Docker [1]. https://gerrit.wikimedia.org/r/c/mediawiki/core/+/630988 [2]. https://xdebug.org/docs/upgrade_guide Thanks! -- Brennen Bearnes Release Engineering Wikimedia Foundation

1 1

Final TechCom Digest, 2020-01-20
by Daniel Kinzler 26 Jan '21

26 Jan '21

Hello all! This is the final TechCom digest. With the new Technical Decision Making Process <https://www.mediawiki.org/wiki/Technical_Decision_Making_Process> in place, we are spinning down the RFC process and shutting down the committee. A big Thank You to all committee members past and present for their time and dedication! On a closing note, two RFCs have been approved after Last Call, both of which I'm personally very happy to see: *Stable interface policy amendment <https://phabricator.wikimedia.org/T268326>*: The policy was amended to include a definition of the "MediaWiki Ecosystem" of extensions to be considered when deprecating obsolete code. The deprecation process was overhauled to allow for a clear timeline from soft deprecation via hard deprecation to removal. *Drop support for upgrading from old releases (pre 1.31) <https://phabricator.wikimedia.org/T259771>*: this frees up to remove about a thousand or so database patch files only needed for upgrading from very old systems. Upgrading from old versions of MediaWiki will still be possible, but have to be performed in multiple steps. -- Daniel Kinzler Principal Software Engineer, Core Platform Wikimedia Foundation

1 0

TechCom digest 2021-01-13
by Krinkle 26 Jan '21

26 Jan '21

The minutes from TechCom's triage meeting on 13 January 2020. Present: Dan A, Daniel K, Tim S., Tim T. RFC: Introduce PageIdentity [Approved] - https://phabricator.wikimedia.org/T208776 - Last Call is closed, approved. RFC: Normalize MediaWiki link tables - https://phabricator.wikimedia.org/T222224 - Amir shows that the enwiki-pagelinks table has surpassed the size of the revisions table. - Jaime shows that (post-MCR compression) the revision tables have gotten considerably smaller, and that on Commons the image/oldimage tables are even bigger than pagelinks and pose a bigger operational risk (since link data can be regenerated). He recommends executing the oldimage migration, which was approved in 2017 as part of RFC T28741 <https://phabricator.wikimedia.org/T28741>. RFC: Drop support for older database upgrades [Last Call] - https://phabricator.wikimedia.org/T259771 - Remains on Last Call until next week. Are people quietly excited? RFC: Stable interface policy, Nov 2020 amendment [Last Call] - https://phabricator.wikimedia.org/T268326 - Remains on Last Call for one more week. Next week IRC office hours No IRC discussion scheduled for next week. You can also find our meeting minutes at https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Minutes -- Timo

1 0

Upcoming cloud-vps project deletions
by Andrew Bogott 26 Jan '21

26 Jan '21

Hello! The 2020 project opt-in process wrapped up at the end of the year, and we've identified the following projects as abandoned: - asyncwiki - blog - commons-corruption-checker - fastcci - finding-glams - ign2commons - lizenzhinweisgenerator - lta-tracker - meza - ogvjs-integration - puppet - snuggle - wikidata-federation - wikidata-primary-sources-tool - wikidata-realtime-dumps - wikimania-scholarships At the end of this month (2020-01-31) those projects will be deleted along with all related data and VMs. If you know of anyone associated with those projects who is not on this list, please bring this to their attention. And, if you think any of this is in error, please notify me immediately. Thank you! -Andrew + the WMCS team

1 0

[REMINDER] Call for projects and mentors for Google Summer of Code 2021 and Outreachy Round 22 is ongoing!
by Pavithra Eswaramoorthy 26 Jan '21

26 Jan '21

Hello everyone, TLDR; Wikimedia will soon be applying as a mentoring organization to Google Summer of Code 2021 <https://summerofcode.withgoogle.com> [1] and Outreachy Round 22 <https://www.outreachy.org/> [2]. The application submission deadline for GSoC is February 20th, and Outreachy is March 7th. We are currently working on a list of interesting project ideas to include in the application. If you have some ideas for coding or non-coding (design, documentation, translation, outreach, research) projects, share them here: https://phabricator.wikimedia.org/T270429 [3]. Timeline As a mentor, you will be engaging potential candidates in the application period for GSoC between March 10th – April 13th, and for Outreachy between March 15th – April 16th. During this time, you will help candidates make small contributions to your project and answer any project related queries. You will be working more closely with the accepted candidates during the coding period between May-August. Project ideas We have started compiling a list of projects that you can take a look at here: https://www.mediawiki.org/wiki/Google_Summer_of_Code/2021 [4], https://www.mediawiki.org/wiki/Outreachy/Round_22 [5] If you don’t have an idea in mind and would like to pick one from an existing list, check out these projects: https://phabricator.wikimedia.org/tag/outreach-programs-projects/ [6] Through GSoC, you can mentor only coding but with Outreachy also non-coding projects (including design, documentation, translation, outreach, etc.). Note that GSoC has some changes this year, including: * Smaller project size ~175 hr project (previously 350 hr) * Shortened coding period T ~10 weeks long (previously 3 months) * Eligibility criteria redefined; the program is now open to candidates participating in a variety of academic programs (previously accredited university programs only) Please keep these changes in mind while sharing your project ideas. Some tips for proposing projects: * Follow this task description template when you propose a project in Phabricator: https://phabricator.wikimedia.org/tag/outreach-programs-projects [6]. Add #Google- Summer-of-Code (2021) or #Outreachy (Round 22) tag to it. * Remember, the project should require an experienced developer ~15 days to complete and a newcomer ~3 months for Outreachy and ~10 weeks for GSoC. * Each project should have at least 2 mentors, and one of them should hold a technical background. * When it comes to picking a project, you could propose one that is: * Relevant for your language community or brings impact to the Wikimedia ecosystem in the future. * Welcoming and newcomer-friendly and has a moderate learning curve. * A new idea you are passionate about, there are no deadlines attached to it; you always wanted to see it happen but couldn't due to lack of resources! * About developing a standalone tool (possibly hosted on Wikimedia Toolforge), with fewer dependencies on Wikimedia's core infrastructure, and doesn't necessarily require a specific programming language, etc. To learn more about the roles and responsibilities of a mentor, visit our resources on MediaWiki.org: https://www.mediawiki.org/wiki/Outreachy/Mentors [7], https://www.mediawiki.org/wiki/Google_Summer_of_Code/Mentors [8]. Cheers, GSoC 2021 and Outreachy Round 22 Administrators [1] https://summerofcode.withgoogle.com [2] https://www.outreachy.org/ [3] https://phabricator.wikimedia.org/T270429 [4] https://www.mediawiki.org/wiki/Google_Summer_of_Code/2021 [5] https://www.mediawiki.org/wiki/Outreachy/Round_22 [6] https://phabricator.wikimedia.org/tag/outreach-programs-projects/ [7] https://www.mediawiki.org/wiki/Outreachy/Mentors [8] https://www.mediawiki.org/wiki/Google_Summer_of_Code/Mentors

1 0

[Train] 1.36.0-wmf.27 status update (blocked at group0 by T272508)
by Brennen Bearnes 20 Jan '21

20 Jan '21

The 1.36.0-wmf.27 version of MediaWiki[0] is at group0[1], and is currently blocked. Thanks to James Forrester, RhinosF1, abi_, Nikerabbit, Urbanecm, Daimona, Ladsgroup, Marostegui, and probably others I'm forgetting for getting us this far. The new version cannot be deployed further until this issue is resolved: * PropertyInfoSnakUrlExpander: Bad value for parameter $snak->getDataValue(): must be a DataValues\StringValue - https://phabricator.wikimedia.org/T272508 Thanks for any help resolving this issue. If a fix is merged with reasonable time to deploy before the 15:00 Pacific deployment cutoff, the train will continue to group1 today.[2] Otherwise it will resume in the European morning window at earliest. -- Your temporary train tender [0]. <https://phabricator.wikimedia.org/T271341> [1]. <https://tools.wmflabs.org/versions/> [2]. https://wikitech.wikimedia.org/wiki/Deployments/Holding_the_train#What_happ…

1 0

2021-01-20 Scrum of Scrums meeting notes
by Grace Gellerman 20 Jan '21

20 Jan '21

https://www.mediawiki.org/wiki/Scrum_of_scrums/2021-01-20 2021-01-20 == Callouts == * Seats still available for the frontend web performance training. Doodle seems to be favoring West coast times currently: https://doodle.com/poll/ax9qak5mwb7rzvuh?utm_source=poll&utm_medium=link === No updates === Community Tech === No notes provided ===Product Infrastructure, Parsing, Inuka, Analytics, Cloud Services, Fundraising Tech, Quality and Test, Security == SoS Meeting Bookkeeping == * Updates: 9 responses to async retro. Will release responses today. == Product == === Anti-Harassment Tools === * Blocked by: * Blocking: * Updates: ** Working with DBAs to add a new table in SecurePoll extension: https://phabricator.wikimedia.org/T271268 === Editing === * Blocked by: * Blocking: * Updates: Parser cache thing from last week not done; doing this week. === Growth === * Blocked by: * Blocking: * Updates: ** working on link recommendations - https://wikitech.wikimedia.org/wiki/Add_Link ** planning image recommendations together with a number of other teams === iOS native app === * Blocked by: N/A * Blocking: N/A * Updates: No major updates. Refining what is going to hit 6.8. === Android native app === * Blocked by: N/A * Blocking: N/A * Updates: No major updates. Starting to refine image tagging structured editing task. === Web === * Blocked by: * Blocking: * Updates: ** https://phabricator.wikimedia.org/T260091: Rolling out max-width in VectorV2 to more places via configurable allow/deny lists === Structured Data === * Blocked by: * Blocking: * Updates: ** Resuming work on splitting MediaSearch UI out into a separate extension === Abstract Wikipedia === * Blocked by: * Blocking: * Updates: ** Working to implement a look-ahead search widget in our Vue code; with thanks again to Anne and Eric for their support and insight from their MediaSearch experience. ** Making initial commits to stand up the evaluator and orchestrator services' bases. ** Still working on programmatic type management. ** Community contest to select the logo concept for Wikifunctions is currently running; ideas welcome! === Language === * Blocked by: None * Blocking: None * Updates: No major updates ** Train blocker fixed: https://phabricator.wikimedia.org/T272428 === Library === * Blocked by: * Blocking: * Updates: ** Investigating behavior tracking software options. Will probably install Matomo in staging in the next few weeks ** Wikilink bug fixes ** Working on the Wikipedia Library Echo notification === Vue.js === * Blocked by: * Blocking: * Updates: ** We exist now! Hello world! ** "We" are Roan (full-time), Volker, Anne T and Eric G, Marcella (part-time) ** More to come later as we figure out priorities == Technology == === Platform === * Blocked by: We are blocked on deploying LVS changes due to https://phabricator.wikimedia.org/T272258 (Traffic, DCOps) * Blocking: * Updates: None === Engineering Productivity === ==== Performance ==== * Blocked by: Data Persistence with setup of x2 cluster * Blocking: none * Updates: ==== Release Engineering ==== * Blocked by: ** * Blocking: ** * Updates: ** [All] Deployments/Covid-19 https://wikitech.wikimedia.org/wiki/Deployments/Covid-19 ** Train Health *** Last week: 1.36.0-wmf.26 [[phab:T267419]]  *** This week: 1.36.0-wmf.27 [[phab:T271341]]  *** Next week: 1.36.0-wmf.28 [[phab:T271342]]  === Search Platform === * Blocked by: ** no one * Blocking ** no one * Updates: ** Show correct create page/page exists message on wikis with multiple writing systems -https://phabricator.wikimedia.org/T260957 ** Reduce the impact of the sanitizer on wikidata - https://phabricator.wikimedia.org/T239931 ** "ElasticSearch shard size check" icinga warnings on cloudelastic servers -https://phabricator.wikimedia.org/T265908 ** Support cloudelastic in spicerack elasticsearch - https://phabricator.wikimedia.org/T268779 ** 40-elasticsearch-readahead udev rule failing for cloudelastic100[5,6] - https://phabricator.wikimedia.org/T265699 ** Restart elasticsearch clusters to apply readahead changes - https://phabricator.wikimedia.org/T266492 === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** Growth on setting up LVS for linkrecommendation. * Updates: ** No updates == Cross-cutting == * Blocked by: * Blocking: * Updates: ** PHP 8.0 compatibility for REL1_35 of MediaWiki core is relatively close. (T266583) Once that's done, we'll work to enable it as voting for (at least) all tarball extensions and then announce functional compatibility in 1.35.2 or whatever release. ** PHP 8.0 compatibility for master/production looks like it is stalled on the replacement for ElasticSearch(!) as PHP 8.0 support may be held back until the new non-OSI licence is in place (T271777, T272111). [[Category:Scrum of scrums{{#translation:}}|*]]

1 0

[Train] 1.36.0-wmf.27 status update (rolled back from group0, blocked by T272428)
by Brennen Bearnes 19 Jan '21

19 Jan '21

The 1.36.0-wmf.27 version of MediaWiki[0] has been rolled back from group0[1], and is currently blocked. The new version cannot be deployed until this issue is resolved: * Error 1146: Table 'mediawikiwiki.translate_cache' doesn't exist - https://phabricator.wikimedia.org/T272428 Thanks for any help resolving this issue. Since we're near the 3pm Pacific cutoff point for the day, at earliest, the train can roll forward in the European morning.[2] -- Your loquacious locomotive laggard [0]. <https://phabricator.wikimedia.org/T271341> [1]. <https://tools.wmflabs.org/versions/> [2]. https://wikitech.wikimedia.org/wiki/Deployments/Holding_the_train#What_happ…

1 0

SWT South Asia 2021
by Krishna Chaitanya Velaga 19 Jan '21

19 Jan '21

Hello everyone, Hope you are all doing well, and keeping safe. I am happy to share with you all an update about the Small wiki toolkits <https://meta.wikimedia.org/wiki/Small_wiki_toolkits> initiative in India, and the larger South Asian region. As part of the Small wiki toolkits initiative, in the second half of 2020, Indic-TechCom, in collaboration with WMF's Developer Advocacy team, designed and conducted a technical workshop series: "Small wiki toolkits - Indic workshop series 2020.” A report of the same can be seen at https://w.wiki/Yqf. There has been positive response from community members, as the workshops have been helpful in understanding basic to intermediate level technical processes on wikis. Two more workshops have been conducted towards the end of last year. So, we are now planning to continue to run these workshops (one per month on average) and expand them to the Wikimedia communities in the South Asian region. The objective is to impart basic to intermediate level technical skills for volunteers working on Wikimedia projects of languages in the South Asian region. More information can be found at https://meta.wikimedia.org/wiki/SWT_South_Asia/Overview The first session will be a brainstorming session on technical challenges faced by contributions on their wikis, from 6 pm to 7:30 pm (IST) on 24 January 2021. It will be facilitated by Birgit Müller (Director of Technical Engagement at WMF). If you are interested, please register at https://meta.wikimedia.org/wiki/SWT_South_Asia/Registration. This discussion will be crucial to decide topics for future workshops. Community members are also welcome to suggest topics for future workshops anytime at https://w.wiki/t8Q. These workshops are being organized with the help of a handful of volunteers from India, Nepal and Bangladesh, and if anyone is interested to help in organizing, please let us know. Regards, Krishna Chaitanya on behalf of SWT South Asia organizers <https://meta.wikimedia.org/wiki/SWT_South_Asia/Organization>

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l January 2021