Wikitech-l February 2020

wikitech-l@lists.wikimedia.org

54 participants
70 discussions

Re: [Wikitech-l] A monthly purge
by bawolff 20 Feb '20

20 Feb '20

Accidentally replied directly instead of to list like i meant to On Thu, Feb 20, 2020 at 8:15 AM bawolff <bawolff+wn(a)gmail.com> wrote: > Some back of the napkin math > > If it takes 0.5 seconds to parse a page on average, it would take 289 days > to refresh all the pages on wikipedia (Assuming we aren't parallelizing the > task). It definitely seems like a non-trivial amount of computer work. > > See also discussion at https://phabricator.wikimedia.org/T157670 > (Basically the proposal is, instead of trying to purge everything in > alphabetical order, just start purging things that haven't been purged in > like a year or something. Discussion is fairly old, I don't think anyone is > working on it anymore). > > -- > bawolff > > On Thu, Feb 20, 2020 at 7:50 AM Amir E. Aharoni < > amir.aharoni(a)mail.huji.ac.il> wrote: > >> >> ‫בתאריך יום ה׳, 20 בפבר׳ 2020 ב-9:26 מאת ‪bawolff‬‏ <‪ >> bawolff+wn@gmail.com‬‏>:‬ >> >>> Pretty sure the answer is no (Although i don't know for a fact). >>> >>> However, parser cache only lasts for 30 days. So pages will get parsed at >>> least once every 30 days (if viewed). However that's separate from links >>> update (aka categories, linter, etc). >>> >>> I suspect that doing a linksupdate of every article once a month would >>> take >>> more than a month. >>> >> >> If it's true, and it may well be, is it conceivable that this will be >> some kind of a continuous process? >> >> >

1 0

A monthly purge
by יגאל חיטרון 20 Feb '20

20 Feb '20

Hello. Could you tell me, please, if there is a monthly purge of all pages, or at least all articles, in all wikis? I remember there were plans about this, to aid the Linter, but I don't know what happens now. I consider to suggest a new feature to our Village Pump, but it it can be turned on only with a purging of all articles. Thank you.

3 3

Scrum of scrums/2020-02-19
by Željko Filipin 19 Feb '20

19 Feb '20

Hi, for HTML version see https://www.mediawiki.org/wiki/Scrum_of_scrums/2020-02-19 Željko -- = 2020-02-19 = == Callouts == * SRE blocked by Releng on [[phab:T217924]] (Make canary wait time configurable) == SoS Meeting Bookkeeping == * Updates: ** Increase participation in Scrum of Scrums meeting [[phab:T245278]] == Product == === Community Tech === * Updates: ** The table for watchlist_expiry is created. ** We were having issues with Google approving our extension, and we think we've resolved it. === Anti-Harassment Tools === * Updates: ** Continuing development on CU. === Editing === * Updates: ** Bartosz created a proof of concept that enables us to test Gerrit patches with one click. This project came out of the hack week the team held at the end of December (2019). Demo instructions are here: [[phab:T76245#5788579]] ** Deployed version 1.0 of Replies for Discussionztools to Arabic, Dutch, French and Hungarian Wikipedias via a query string parameter === Growth === * Updates: ** Newcomer tasks 1.1 (topic matching): building support for loading ORES topic scores into the Search API === iOS native app === * Updates: **Continuing development on 6.6 release (mobile-html integration) [[phab:project/view/4273]] === Android native app === * Updates: **Almost done with offline conversion for mobile-html. **In the final design review stage for Suggested Edits V4 feature. === Web === * Updates: ** Summary: continuing desktop improvements (DIP). ** [[Reading/Web/Desktop_Improvements|Desktop Improvements Project (Vector / DIP)]]: *** [[phab:T245456|<nowiki>[Dev] Adopt template partials in Vector and revise sidebar component</nowiki>]] *** [[phab:T243281|<nowiki>Build opt-out button for logged-in users with new vector on</nowiki>]] *** [[phab:T242381|<nowiki>Add a Vector skin version preference</nowiki>]] *** [[phab:T232140|<nowiki>Separate out logo handling into square image logos and long text/wordmark banner logos</nowiki>]] *** [[phab:T239269|<nowiki>[SPIKE 16hrs] Add coding convention for new commits to Vector</nowiki>]] *** [[phab:T239258|<nowiki>Migrate JavaScript API documentation from jsduck to JSDoc</nowiki>]] ** Mobile website (MinervaNeue / MobileFrontend): *** [[phab:T245162|<nowiki>Job unable to create file page "Fatal: Call function inNamespace() on null" (via MinervaHooks)</nowiki>]] *** [[phab:T32405|<nowiki>[EPIC] MobileFrontend extension should stop special-casing main page</nowiki>]] *** [[phab:T244444|<nowiki>Replace all possible, non-product specific icons in MobileFrontend/MinervaNeue with WikimediaUI/OOUI ones</nowiki>]] *** [[phab:T244106|<nowiki>[M] Setup storybook from Minerva</nowiki>]] *** [[phab:T243650|<nowiki>Regression: Mobile talk page shows links in overlay header</nowiki>]] *** [[phab:T244181|<nowiki>[print] Article title hidden with mobile print styles</nowiki>]] === Product Infrastructure === * Blocked by: ** SRE on review of Helm charts for proton and mobileapps (no callout needed, they know) * Updates: ** Push notification service planning continues === Parsing === * Updates: ** Parsoid integration into core as a composer library underway ** Parsoid extension API drafting underway - hoping to have a first draft ready by end April === Language === * Blocked by: ** Core Platform: Requesting help to debug why some jobs are not being queued [[phab:T144780]] === UI Standardization === * Updates: ** Preparation of next minor release OOUI v0.37.0, planned for coming week, Tue 25th Feb == Technology == === Analytics === * Blocking: ** Release Engineering: The manual analytics-refinery-release and analytics-refinery-update-jars Jenkins jobs need to be ported from bare-metal Jessie into Docker (or just dropped) [[phab:T210271]] ** WMDE/Wikidata: waiting for updates on [[phab:T236895]] === Fundraising Tech === * Updates: ** More work on recurring donations with backup card processor ** Cleaning up useless data from the CiviCRM database ** CentralNotice work on banner templates and sub-national geotargeting ** Moving more payment processor API response normalization into backend library === Core Platform === * Blocking: ** Language: Requesting help to debug why some jobs are not being queued [[phab:T144780]] ** WMDE/Wikidata: would be happy to get some input/assistance on [[phab:T233520]] * Updates: ** Active development on job queuing ** API gateway and rate-limiting evolving [[Core_Platform_Team/Initiative/API_Gateway/Epics,_User_Stories,_and_Requirements#Epic_3:_Rate_limits]] ** Changeprop k8 to production === Engineering Productivity === ==== Performance ==== * Blocking: ** Release Engineering: Duplicate entry 'ext.uls.pt-vector|en' for key 'PRIMARY' [[phab:T245570]] ** WMDE/Wikidata: would be happy to get some input/assistance on [[phab:T233520]] ==== Quality and Test Engineering ==== * Updates: ** Quality and Test Engineering Office Hour *** Friday, February 21 17:00-17:50 UTC *** Topic: Software Testing Anti-patterns ==== Release Engineering ==== * Blocked by: ** Analytics: The manual analytics-refinery-release and analytics-refinery-update-jars Jenkins jobs need to be ported from bare-metal Jessie into Docker (or just dropped) [[phab:T210271]] ** Performance: Duplicate entry 'ext.uls.pt-vector|en' for key 'PRIMARY' [[phab:T245570]] * Blocking: ** SRE on on [[phab:T217924]] (Make canary wait time configurable) * Updates: ** Train Health *** Last week: 1.35.0-wmf.19 - [[phab:T233867]] *** This week: 1.35.0-wmf.20 - [[phab:T233868]] **** (Performance) Duplicate entry 'ext.uls.pt-vector|en' for key 'PRIMARY' [[phab:T245570]] *** Next week: 1.35.0-wmf.21 - [[phab:T233869] === Site Reliability Engineering === * Blocked by: ** Release Engineering on [[phab:T217924]] (Make canary wait time configurable) * Updates: ** ServiceOps has OKR reprioritization to deal with the aftermath/reasons of the outages. == Wikimedia DE == === Wikidata === * Blocked by: ** Analytics: We're waiting for updates on [[phab:T236895]] ** Core Platform and/or Performance: We would be happy to get some input/assistance on [[phab:T233520]]

1 0

[Train] 1.35.0-wmf.19 status update
by Dan Duvall 18 Feb '20

18 Feb '20

The 1.35.0-wmf.19 version of MediaWiki is blocked[0]. The new version is deployed to group1[1], but can proceed no further until these issues are resolved: * Varnish 5xx 2k/min spike - https://phabricator.wikimedia.org/T245202 Once these issues are resolved train can resume. If these issues are resolved on a Friday the train will resume Monday. Thank you for your help resolving these issues! -- Your humble train toiler [0]. https://phabricator.wikimedia.org/T233867 [1]. https://tools.wmflabs.org/versions -- Dan Duvall Senior Software Engineer, Release Engineering Wikimedia Foundation <http://wikimediafoundation.org>

1 1

Grant request for an AbuseFilter overhaul
by Daimona 18 Feb '20

18 Feb '20

Hello everyone! I have submitted a grant proposal on meta for overhauling and modernizing the AbuseFilter extension [1]. The goal of this proposal is to make the extension easier to maintain, with the side effects of simplifying future maintenance and easing feature additions. The grant will also cover a bugfix part and the addition of a community-requested feature. You can find the request here: https://meta.wikimedia.org/wiki/Grants:Project/Daimona_Eaytoy/AbuseFilter_o… If you're interested in this proposal, I invite you to endorse it on meta at the page linked above. If you have any questions, you can use the talk page on meta. Thank you for your time! [1] - https://www.mediawiki.org/wiki/Extension:AbuseFilter -- https://meta.wikimedia.org/wiki/User:Daimona_Eaytoy "Daimona" is not my real name -- he/him

1 0

`Parser::getTitle` may return `null`, even though it is not allowed to
by Robert Vogel 14 Feb '20

14 Feb '20

Hello! In version 1.34 creating a `Parser` object without explicitly setting the `mTitle` value was deprecated. But as it is just a deprecation yet, there are still cases where `mTitle` can be `null` (it seems like this is even the case when calling the proper service with `MediaWikiServices::getInstance()->getParser()`). Unfortunately the `Parser::getTitle` method got changed, in a way that it _must_ return an instance of `Title`. No `null` allowed. This leads to a `TypeError`. I have found two Phabricator tasks, each with a proper patch for this: - https://phabricator.wikimedia.org/T228881 / https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/533559/ - https://phabricator.wikimedia.org/T245129 / https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/571940/ It looks like a very small change. I'd very much appreciate if somebody could review and merge one of these. -- Robert Vogel

1 0

Scrum of scrums/2020-02-12
by Željko Filipin 13 Feb '20

13 Feb '20

Hi, for HTML version see https://www.mediawiki.org/wiki/Scrum_of_scrums/2020-02-12 Željko -- = 2020-02-12 = == Callouts == * SRE for Release engineering on Looking for scap reviewers for syslog support [[gerrit:563468]] * Parsing - Project retrospective of how Parsoid was ported to PHP: [[phab:phame/post/view/189/parsoid_in_php_or_there_and_back_again]] == Product == === Community Tech === * Updates: ** We anticipate new database table will be created this week (Watchlist Expiry). === Anti-Harassment Tools === * Blocking: ** Release Engineering - [Regression 1.35.0-wmf.19] i/p/IndexPager.php:* PHP Warning: implode(): Invalid arguments passed [[phab:T244937]]. This is part of adding more functionality (multicolumn sort). === Editing === * Updates: ** Deploying Reply v1.0 for talk pages next week ** For Reply v2.0, refining the approach to autcomplete @-mentions ([[phab:T232601]]) and developing a technical approach to editing comments ([[phab:T242562]]) === Growth === * Updates: ** Newcomer tasks 1.1 (topic matching): building the infrastructe to upgrade the topics to use the ORES models instead of our initial "morelike" algorithm. Big thanks to the Search and Scoring teams for their ongoing support. ** Newcomer tasks 1.2 (guidance): We are also beginning engineering work on this part of the newcomer tasks workflow ([[phab:T244431]]) === iOS native app === * Updates: ** Continuing development on 6.6 release (mobile-html integration) [[phab:project/view/4273]] === Android native app === * Updates: ** Finishing up conversion logic from old caching to new offline database scheme. (relating to mobile-html migration) ** Iterating on design refinements of Image Tagging interface. === Web === * Updates: ** Summary: continuing desktop improvements (DIP). Digging into logo handling, conventions, feature management, and legacy version preferences. ** [[Reading/Web/Desktop_Improvements|Desktop Improvements Project (Vector / DIP)]]: *** [[phab:T232140|<nowiki>Separate out logo handling into square image logos and long text/wordmark banner logos</nowiki>]] *** [[phab:T239269|<nowiki>[SPIKE 16hrs] Add coding convention for new commits to Vector</nowiki>]] *** [[phab:T242381|<nowiki>Add a Vector skin version preference</nowiki>]] *** [[phab:T236711|<nowiki>It should be possible to customise the implementation of mw.util.addPortletLink function on a skin basis</nowiki>]] ** Mobile website (MinervaNeue / MobileFrontend): *** [[phab:T244444|<nowiki>Replace all possible, non-product specific icons in MobileFrontend/MinervaNeue with WikimediaUI/OOUI ones</nowiki>]] *** [[phab:T244436|<nowiki>[Regression, Production] user contributions icon missing on AMC mode</nowiki>]] *** [[phab:T212465|<nowiki>[EPIC] None of our View's should exhibit 2 levels of inheritance</nowiki>]] *** [[phab:T32405|<nowiki>[EPIC] MobileFrontend extension should stop special-casing main page</nowiki>]] *** [[phab:T244106|<nowiki>Setup storybook from Minerva</nowiki>]] *** [[phab:T214641|<nowiki>[Technical] The LoadingOverlay and src/mobile.startup/rlModuleLoader.js pattern should be removed - it results in overlay flashes during switches</nowiki>]] *** [[phab:T139221|<nowiki>Better expose table of contents to assistive technology by using role=navigation</nowiki>]] === Product Infrastructure === * Updates: ** Push notifications technical planning in process === Structured Data === * Blocking: ** Search Platform: Data dumps for SDC: [[phab:T221917]] - honestly, we're not blocking this anymore * Updates: ** no update from last week - finishing off SDC (input types, constraints, novalue/somevalue) === Parsing === * Updates: ** Project retrospective of how Parsoid was ported to PHP: [[phab:phame/post/view/189/parsoid_in_php_or_there_and_back_again]] ** Working on a parsoid extension api using Parsoid's Cite implementation as a guinea pig. === UI Standardization === * Updates: ** OOUI v0.36.5 on the way, all icon related changes, added MF/MN icons ** Started OpenDesignKit integration discussions into design.wikimedia.org ** WikiEditor 2010 toolbar design aligned to standard components (thanks, Ed Sanders) == Technology == === Analytics === * Blocking: ** Release Engineering: The manual analytics-refinery-release and analytics-refinery-update-jars Jenkins jobs need to be ported from bare-metal Jessie into Docker (or just dropped) [[phab:T210271]] === Fundraising Tech === * Updates: ** Re-evaluating failure modes on paymentswiki - when we get networking errors making API calls donors often donate twice due to unclear errors. - [[phab:T243873]] ** More work towards recurring donations with backup card processor - [[phab:T243334]], ** Updating translated thank you letters and adding a button in the CiviCRM UI to send the multilingual versions - [[phab:T243098]] ** Better aligning our use of a few CiviCRM concepts with core - [[phab:T244326]] ** Upstreaming or discarding more local CiviCRM performance hacks - [[phab:T194313]], [[phab:T217600]] ** More work to finish CentralNotice sub-national geotargeting - [[phab:T243011]] === Core Platform === * Blocking: ** German Technical Wishlist (Release Engineering/SRE/Core Platform (?)) - Issues with logging into beta wikipedia cause failures of daily selenium tests of TwoColEditConflct extension: [[phab:T243123]] ** Search Platform: MW Job consumers sometimes pause for several minutes [[phab:T224425]] * Updates: ** API gateway plan in technical planning stages [[Core_Platform_Team/Initiatives/API_Gateway]] ** Search enhancement epic starting [[Core_Platform_Team/Initiatives/Core_REST_API_in_MediaWiki#Epic_1.5:_Search_enhancement]] === Engineering Productivity === ==== Quality and Test Engineering ==== * Updates: ** Started with series of internal testing strategy presentations. First in series - CheckUser. ==== Release Engineering ==== * Blocked by: ** Analytics: The manual analytics-refinery-release and analytics-refinery-update-jars Jenkins jobs need to be ported from bare-metal Jessie into Docker (or just dropped) [[phab:T210271]] ** Anti-Harassment Tools - [Regression 1.35.0-wmf.19] i/p/IndexPager.php:* PHP Warning: implode(): Invalid arguments passed [[phab:T244937]] * Blocking: ** SRE on Looking for scap reviewers for syslog support [[gerrit:563468]] ** German Technical Wishlist (Release Engineering/SRE (?)) - Issues with logging into beta wikipedia cause failures of daily selenium tests of TwoColEditConflct extension: [[phab:T243123]] * Updates: ** Train Health *** Last week: 1.35.0-wmf.18 - [[phab:T233866]] *** This week: 1.35.0-wmf.19 - [[phab:T233867]] **** Anti-Harassment Tools - [Regression 1.35.0-wmf.19] i/p/IndexPager.php:* PHP Warning: implode(): Invalid arguments passed [[phab:T244937]] *** Next week: 1.35.0-wmf.20 - [[phab:T233868]] === Scoring Platform === * Updates: ** Rolled back ORES deploy for memory issues. New deploy out on Monday. [[phab:T242705]] *** Memory opened up but issues still persist when doing logrotate. ** Coloring OOJSui icons using the WMF standard color palette. Is there a good way to do this? === Search Platform === * Blocked by: ** Structured Data: Data dumps for SDC: [[phab:T221917]] ** Core: MW Job consumers sometimes pause for several minutes [[phab:T224425]] * Updates: ** Load ORES articletopic data into ElasticSearch via the weekly bulk update [[phab:T240556]] === Security === * Updates: ** [[phab:T242124]] - EventStreamConfig Security Review = done ** [[phab:T240010]] - Wikipedia Previews = stalled on one or two questions for requesters ** [[phab:T242134]] - Talk Pages/Discussion Tools review = started ** [[phab:T243007]] - KaiOS review = waiting for code freeze or commit specification === Site Reliability Engineering === * Blocked by: ** Release engineering on Looking for scap reviewers for syslog support [[gerrit:563468]] * Blocking: ** German Technical Wishlist (Release Engineering/SRE/Core Platform (?)) - Issues with logging into beta wikipedia cause failures of daily selenium tests of TwoColEditConflct extension: [[phab:T243123]] == Wikimedia DE == === German Technical Wishlist === * Blocked by: ** Release Engineering/SRE/Core Platform (?): Issues with logging into beta wikipedia cause failures of daily selenium tests of TwoColEditConflct extension: [[phab:T243123]]

2 1

Reminder: Wikimedia Café online meeting for February 2020 with focus on the 2030 strategy
by Pine W 13 Feb '20

13 Feb '20

Hello colleagues, This is a reminder that this month's Wikimedia Café will start approximately 48 hours from now. -- Hello colleagues, The February 2020 Wikimedia Café meetup will occur on 15 February 2020 at 8:30 AM PST / 11:30 AM EST / 4:30 PM UTC / 10 PM IST. The date is earlier than usual in the month due to scheduling constraints. This month's meetup will focus on the 2030 strategy recommendations, and will be divided into two discussions of one hour each. More information regarding the agenda and links to strategy documents are available at https://meta.wikimedia.org/wiki/Wikimedia_Café. As usual, the meeting style for the Café will emphasize discussion rather than presentation. People are welcome to participate as listeners only if they prefer. Please see the page on Meta for more information about the Café. Please watch the page for any updates, particularly to the schedule or the agenda. Signing up for the meeting is optional, but is helpful to the organizers so that we can estimate how many people will attend. Signing up for the meeting also informs us who we should notify individually if there are significant changes. If there are any problems with connecting to the meeting or if you have any questions or comments then please write on the Meta talk page. Pine ( https://meta.wikimedia.org/wiki/User:Pine )

1 0

How to invalidate a ResourceLoader URL that depends on user-generated content?
by Máté Szabó 13 Feb '20

13 Feb '20

Hi all, I am working on a custom ResourceLoader module whose style contents are compiled from Sass stylesheets. These Sass stylesheets take as input user-defined theme variables that affect the output. I am looking for a way to invalidate ResourceLoader URLs that include such style modules, when these variables change as a result of an user action—primarily so that the user can see their own changes. ResourceLoader will cache such style URLs for 5 minutes on the CDN and in the user’s browser cache, so by default the user would have to wait until their cached copy of styles expire before they would receive the updated version. Accordingly, I have tried to explore if following the document steps to empty one’s browser cache[1] after making changes would cause the browser to fetch the updated resource from the backend. Unfortunately, both Chrome and Firefox appear to send a “Cache-Control: no-cache” request header for the style URL if the page is hard refreshed, which seems to cause most CDNs (including WMF’s) to act as the source of truth and serve the asset from cache if it is already there, without triggering a backend fetch or conditional revalidation. In the case of a simple refresh, Chromium and derivatives will not trigger a conditional revalidation of resources including on a page since 2017,[2] so they just continue to use the asset that was already in the browser’s cache. Thus, the standard instructions for emptying one’s cache do not seem to be sufficient to allow the user to see the effect of their styling changes. Given the above, the only option I see that would allow the user to see the updated styles after modifying input variables would be to append a version hash to the style URL included in the page HTML—something which ResourceLoader does not currently do. Do you know of a better approach to this problem? Are there any examples in MediaWiki core or extensions that accomplish something similar? I have consulted the code for ResourceLoaderSiteStylesModule, but it does not seem to have any specific invalidation behaviour, nor does it use a version hash in style URLs in the page HTML, so it seems that any updates to it take 5 minutes to propagate. Thanks in advance! — [1] https://en.wikipedia.org/wiki/Wikipedia:Bypass_your_cache [2] https://bugs.chromium.org/p/chromium/issues/detail?id=505048 Cheers, Máté Szabó SOFTWARE ENGINEER +36 30 947 5903 Fandom Poland sp. z o.o. z siedzibą w Poznaniu, ul. Abp. A. Baraniaka 6 Sąd Rejonowy Poznań – Nowe Miasto i Wilda w Poznaniu, VIII Wydział Gospodarczy Krajowego Rejestru Sądowego, KRS 0000254365 NIP: 5252358778 Kapitał zakładowy: 50.000,00 złotych

3 3

Phabricator database master restart - Thursday 13th 06:00AM UTC
by Manuel Arostegui 13 Feb '20

13 Feb '20

Hello, Thursday 13th at 06:00 AM UTC we'll be restarting phabricator database master for https://phabricator.wikimedia.org/T239791 and to get it upgraded as well (https://phabricator.wikimedia.org/T244566) Phabricator will be on read-only mode for around 1 minute. Sorry for the inconvenience, Manuel.

1 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l February 2020