Wikitech-l March 2020

wikitech-l@lists.wikimedia.org

58 participants
60 discussions

Scrum of scrums/2020-03-25
by Željko Filipin 26 Mar '20

26 Mar '20

Hi, for HTML version see https://www.mediawiki.org/wiki/Scrum_of_scrums/2020-03-25 Željko -- = 2020-03-25 = == Callouts == * Release Engineering ** [All] MediaWiki 1.35.0 will get cut on 7 April 2020. If your team has any proposed blockers/deadlines for that, please get them done: [[phab:tag/MW-1.35-release]] ** [All] Review guidance at [[wikitech:Deployments/Covid-19]] and Code Deployment Office Hour at 17:00UTC in #wikimedia-office == Product == === iOS native app === * Updates: ** Bug fixes & testing on 6.6 release (mobile-html integration) [[phab:project/view/4273]] ***Releasing to public beta in the next day or two === Android native app === * Updates: ** Released Suggested Edits v4 to Beta. ** Released integration with mobile-html to Beta. ** Gathering feedback and rounding out any remaining issues before releasing to production (likely next week). === Web === * Updates: ** Summary: evolving plans for Coronavirus, continuing desktop improvements project (DIP). ** [[Reading/Web/Desktop_Improvements|Desktop Improvements Project (Vector / DIP)]]: *** [[phab:T248010|<nowiki>Vector sidebar missing on test.wikipedia.org and weird footer</nowiki>]] *** [[phab:T247566|<nowiki>Broken section edit links styles on Vector</nowiki>]] *** [[phab:T247890|<nowiki>Default to feature branch in Vector</nowiki>]] *** [[phab:T245456|<nowiki>[Dev] Adopt template partials in Vector and revise sidebar component</nowiki>]] *** [[phab:T243281|<nowiki>Build opt-out link for logged-in users with new vector on</nowiki>]] *** [[phab:T113095|<nowiki>A cached server-side HTML template should update when you change a partial template which it includes</nowiki>]] *** [[phab:T247537|<nowiki>Regression: Border color of Vector action menu is black</nowiki>]] *** Hureo report ** Mobile website (MinervaNeue / MobileFrontend): *** [[phab:T248360|<nowiki>Phan warning in "includes/specials/SpecialMobileDiff.php" - working around by disabling</nowiki>]] *** [[phab:T248347|<nowiki>Allow customisations of Article::getSubstituteContent for different namespaces</nowiki>]] *** [[phab:T248141|<nowiki>Regression: MobileFrontend overrides non-existent user page handling in desktop view</nowiki>]] *** [[phab:T247274|<nowiki>Uncaught ReferenceError: mfTempOpenSection is not defined on main pages</nowiki>]] *** [[phab:T240622|<nowiki>[Technical debt payoff] Remove InlineDiffFormatter and InlineDifferenceEngine from MobileFrontend</nowiki>]] *** [[phab:T126825|<nowiki>Sections on mobile views jumps around after load (FOUC)</nowiki>]] *** [[phab:T237230|<nowiki>Long edit summary/titles/user breaking AMC special pages</nowiki>]] ** Miscellaneous *** [[phab:T248133|<nowiki>WebInstallerOutput should more elegantly load Vector stylesheet rather than hardcoding a reference</nowiki>]] ** Q4 planning == Technology == === Fundraising Tech === * Updates: ** About to test recurring iDEAL (NL bank transfer) via our backup payment processor: [[phab:T230837]] ** Testing migration of recurring card donations from main payment processor's old API to their new API: [[phab:T232017]] ** Fixing localization of error messages on payments-wiki in certain circumstances: [[phab:T247885]] ** About ready to merge sub-national targeting to CentralNotice master ** Reviewing CentralNotice clone campaign functionality ** Fixing matching gift CiviCRM import: [[phab:T246889]] === Core Platform === * Blocking: ** Wikimedia DE/Wikidata: We'd appreciate some input on [[phab:T245535]], and feedback on [[gerrit:580870]]. We got some comments from Krinkle (thanks), but would appreciate some more advice there. ** Search Platform: MW Job consumers sometimes pause for several minutes [[phab:T224425]] * Updates: ** Finishing minimal client epic for Core REST API [[phab:T229662]] ** Adding config flag for development endpoints in Core REST API [[phab:T247997]] === Engineering Productivity === ==== Release Engineering ==== * Updates: ** [All] Deployments/Covid-19 [[wikitech:Deployments/Covid-19]] ** [All] MediaWiki 1.35.0 will get cut on 7 April 2020. If your team has any proposed blockers/deadlines for that, please get them done: [[phab:tag/MW-1.35-release]] ** Train Health *** This week: 1.35.0-wmf.25 - [[phab:T233873]] *** Next week: 1.35.0-wmf.26 - [[phab:T233874]] === Search Platform === * Blocked by: ** Core: MW Job consumers sometimes pause for several minutes [[phab:T224425]] == Wikimedia DE == === Wikidata === * Blocked by: ** Not sure which team - Core Platform?: We'd appreciate some input on [[phab:T245535]], and feedback on [[gerrit:580870]]. We got some comments from Krinkle (thanks), but would appreciate some more advice there.

1 0

Security pre-release announcement: 1.31.7 / 1.33.3 / 1.34.1
by Sam Reed 25 Mar '20

25 Mar '20

Hi all, Tomorrow we will be issuing a security and maintenance release to all supported branches of MediaWiki. The new releases will be: - 1.34.1 - 1.33.3 - 1.31.7 This will resolve two minor issues in MediaWiki core, and also includes some fixes previously committed to git, including minor security and hardening patches along with bug fixes included for maintenance reasons. We've noted that these issues are minor, and as such you don't need to apply them as quickly as with other security releases, if you're unable to do so. We therefore decided to continue with getting the security (and maintenance) release out for this quarter as planned, even with the global situation as is. We will make the fixes available in these respective release branches, and also master. Tarballs will be available for the above mentioned point releases as well. A summary of some of the security fixes that have gone into non bundled MediaWiki extensions will also follow. Thanks! Sam

1 0

Production Excellence #19 (February 2020)
by Krinkle 24 Mar '20

24 Mar '20

📘 Read on Phabricator at https://phabricator.wikimedia.org/phame/post/view/192/ ------- How’d we do in our strive for operational excellence last month? Read on to find out! ## 📊 Month in numbers * 8 documented incidents. [1] * 27 new Wikimedia-prod-error reports. [2] * 26 Wikimedia-prod-error reports closed. [3] * 199 currently open Wikimedia-prod-error reports in total. [4] With a median of 4–5 documented incidents per month (over the last three years), there were a fairly large number of them this past month. To read more about these incidents and pending actionables; check < https://wikitech.wikimedia.org/wiki/Incident_documentation#2020>, or Explore Wikimedia incident stats at <https://codepen.io/Krinkle/full/wbYMZK> (interactive). ------- ## 📖 Unset vs array splice Our error monitor (Logstash) received numerous reports about an “Undefined offset” error from the OATHAuth extension. This extension powers the Two-factor auth (2FA) login interface on Wikipedia (< https://meta.wikimedia.org/wiki/Help:Two-factor_authentication>) ItSpiderman and Reedy investigated the problem. The error message: PHP Notice: Undefined offset: 8 at /srv/mediawiki/extensions/OATHAuth/src/Key/TOTPKey.php:188 This error means that the code was accessing item number 8 from a list (an array), but the item does not exist. Normally, when a “2FA scratch token” is used, we remove it from a list, and save the remaining list for next time. The code used the `count()` function to compute the length of the list, and used a for-loop to iterate through the list. When the code found the user’s token, it used the `unset( $list[$num] )` operation to remove token $num from the list, and then save $list for next time. The problem with removing a list item in this way is that it leaves a “gap”. Imagine a list with 4 items, like [ 1: …, 2: …, 3: … , 4: … ]. If we unset item 2, then the remaining list will be [ 1: …, 3: …, 4: … ]. The next time we check this list, the length of the list is now 3 (so far so good!), but the for-loop will access the items as 1-2-3. The code would not know that 3 comes after 1, causing an error because item 2 does not exist. And, the code would not even look at item 4! When a user used their first ever scratch token, everything worked fine. But from their second token onwards, the tokens could be rejected as “wrong” because the code was not able to find them. To avoid this bug, we changed the code to use `array_splice( $list, $num, 1 )` instead of `unset( $list[$num] )`. The important thing about array_splice is that it renumbers the items in the list, leaving no gaps. – https://phabricator.wikimedia.org/T244308 / https://gerrit.wikimedia.org/r/570253 ------- ## 📉 Outstanding reports Take a look at the workboard and look for tasks that might need your help. The workboard lists error reports, grouped by the month in which they were first observed. → https://phabricator.wikimedia.org/tag/wikimedia-production-error/ Breakdown of recent months: * March: 3 of 10 reports left (unchanged). ⚠️ * April: 4 of 14 left (unchanged). * May: (All clear!) * June: 4 of 11 left (unchanged). * July: 8 of 18 left (unchanged). * August: Two reports closed! 2 of 14 reports left. * September: One report closed, 7 of 12 left. * October: Two reports closed, 6 of 12 left. * November: 5 of 5 left (unchanged). * December: 6 of 9 left (unchanged). * January: One report closed, 6 of 7 reports left. * February: 7 new reports survived the month of February. Last month’s total over recent months was 57 open reports. Of those, 6 got closed, but with 7 new reports from February still open, the total is now up at 58 open reports. ------- ## 🎉 Thanks! Thank you to everyone who helped by reporting, investigating, or resolving problems in Wikimedia production. Together we’re getting there! Until next time, – Timo Tijhof Footnotes: [1] Incidents. – https://wikitech.wikimedia.org/wiki/Incident_documentation#2020 [2] Tasks created. – https://phabricator.wikimedia.org/maniphest/query/aT3iqdM0EJKW/#R [3] Tasks closed. – https://phabricator.wikimedia.org/maniphest/query/jVexIrtOPkcX/#R [4] Open tasks. – https://phabricator.wikimedia.org/maniphest/query/47MGY8BUDvRD/#R

1 0

Scrum of scrums/2020-03-18
by Željko Filipin 24 Mar '20

24 Mar '20

Hi, for HTML version see https://www.mediawiki.org/wiki/Scrum_of_scrums/2020-03-18 This was the last in-person Scrum of scrums meeting. Going forward we will just update the etherpad. Željko -- = 2020-03-18 = == Callouts == * Release Engineering ** Deployments/Covid-19 [[wikitech:Deployments/Covid-19]] ** [All] MediaWiki 1.35.0 will get cut on 7 April 2020. If your team has any proposed blockers/deadlines for that, please get them done: [[phab:tag/MW-1.35-release]] * Structured Data needs review from Security on proposed new data dump for MachineVision: [[phab:T236431#5900726]] == SoS Meeting Bookkeeping == * Updates: ** This was the last in-person SoS meeting. Going forward we will just update the etherpad. == Product == === Editing === * Updates: ** No Updates, we are shifting or focus but hoping to turn on Replying 1.0 on Beta === iOS native app === * Updates: ** Wrapping up development on 6.6 release (mobile-html integration)...Beta soon[[phab:project/view/4273]] === Android native app === * Updates: ** Release of mobile_html ready for Beta, pending final testing. ** Release of Suggested Edits v4 (image tagging) pending user testing / polishing updates. === Web === * Updates: ** Working on Logo and Header for Desktop Refresh === Structured Data === * Blocked by: ** Security on review of proposed new data dump: [[phab:T236431#5900726]] * Updates: ** will miss the meeting because of a clash - back next week ** units for quantities, improved geo-coordinates, date, and monolingual text input types all live ** constraints violations code merged should be live when train rolls ** some CAT improvements on the way == Technology == === Fundraising Tech === * Updates: ** Deployed NL bank transfer capability for backup processor, working on making it possible to accept recurring bank donations there ** Investigating new ways to manage PayPal recurring donations ** Testing migrating recurring donations from main card processor's old API integration to their new integration ** Getting started making an autocomplete dropdown of employers offering matching gifts === Core Platform === * Blocking: ** Wikimedia DE/Wikidata: would appreciate someone advising on [[phab:T225814]]. ** Search Platform: MW Job consumers sometimes pause for several minutes [[phab:T224425]] * Updates: ** Continued work on jobrunner, changeprops ** Core REST API next version coming soon ** API Gateway ** Developer Portal prototype [[Core_Platform_Team/Initiatives/API_Gateway/Documentation_Plan]] ** Process of adopting or co-parenting CentralNotice === Engineering Productivity === ==== Release Engineering ==== * Blocking: ** Wikimedia DE/Wikidata: would appreciate someone chiming in on [[phab:T245826]] * Updates: ** [All] MediaWiki 1.35.0 will get cut on 7 April 2020. If your team has any proposed blockers/deadlines for that, please get them done: [[phab:tag/MW-1.35-release]] ** Train Health *** This week: 1.35.0-wmf.24 - [[phab:T233872]] *** Next week: 1.35.0-wmf.25 - [[phab:T233873]] === Scoring Platform === * Updates: ** Pile of volunteers: chtnnh, haksoat, clemons, nikhil, (+2 more on the way to IRC/phab) ** Jade *** Did first round of user-testing on Beta. *** We got the Jade Diff view to work. https://deployment.wikimedia.beta.wmflabs.org/wiki/Jade:Diff/4 **** Considered native OOUI element for Diff. Needs PHP. Crusty. **** Struggling to get date format strings to the client side ** Released revscoring 2.6.9 which includes section-based features (e.g. text complexity measures) ** Implemented improved feature extraction for idioms and images (re. articlequality) === Search Platform === * Blocked by: ** Core: MW Job consumers sometimes pause for several minutes [[phab:T224425]] * Updates: ** Copy English Wikipedia drafttopic scores to other wikis somewhere in the CirrusSearch pipeline [[phab:T241015]] ** Once the ORES articletopic - ElasticSearch pipeline is set up, update data about all articles [[phab:T243357]] === Security === * Blocking: ** Structured Data: review of proposed new data dump: [[phab:T236431#5900726]] === Site Reliability Engineering === * Blocking: ** Product Infrastructure on creations of k8s namespaces/tokens for proton, mobileapps. Working on it ** Research on creation of k8s namespaces/tokens for recommendation-api. Working on it. == Wikimedia DE == === Wikidata === * Blocked by: ** Release Engineering: We would appreciate someone chiming in on [[phab:T245826]] ** Not sure who - Core Platform: We would appreciate someone advising on [[phab:T225814]]. * Updates: ** wb_terms table (the old term store) is not being read or updated anymore. [[phab:T208425]]

4 3

Unable to run MobileFrontEnd Extension
by Egbe Eugene 24 Mar '20

24 Mar '20

Hi All, After Installing the MobileFrontEnd extension, I am unable to run it due to some error. *Fatal error: Uncaught Error: Call to undefined method MediaWiki\MediaWikiServices::getContentHandlerFactory() in /Library/WebServer/Documents/myweb/otherprojects/mw/core/extensions/MobileFrontend/includes/MobileFrontendEditorHooks.php* I am not sure what is missing and how can I make this run. Thanks

2 2

MediaWiki API page move/redirect data
by James Gardner 23 Mar '20

23 Mar '20

Hi all, We're still working on a project with the MediaWiki API, and we've ran into a different issue regarding page moves/redirects. We're trying to pull revision and redirect data from the "Killing/Death of Luo Changqing" page and talk page. Unfortunately, this page wasn't found when pulling it through the MediaWiki API when we filtered using our date range from 2009-2019. Either Death or Killing worked prior to the page move, but now we found that we can no longer access the revisions that occurred during the old time frame. Regarding pages that have been moved/redirected, what would you recommend us to do pull this data that was previously available? Thanks, Jackie, James, Junyi, Kirby

1 0

[Blog post] Tale of redesigning the huge Wikidata wb_terms table
by Léa Lacroix 23 Mar '20

23 Mar '20

Hello all, For almost a year, the Wikidata development team has been working on the task of redesigning and migrating the wb_terms table, which had become too big and unsustainable over the years. You can read the tale of our journey on this blog post: Come to Terms with Changes <https://phabricator.wikimedia.org/phame/post/view/195/coming_to_terms_with_…> If you’re a tool maintainer and your tool queries directly the Labs database replicas, you can read more details <https://lists.wikimedia.org/pipermail/wikidata/2020-March/013901.html> about the next steps and how to update your code. Congratulations to all the developers involved in this big project! Cheers, -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

[JavaScript] The best documentation automation can buy
by Stephen Niedzielski 23 Mar '20

23 Mar '20

Yo! I've written a short post on improving JSDocs with typing. It's called "the best documentation automation can buy." If you write JavaScript, you may find it informative. If you're already using this new tooling in your projects, I'd be interested to hear your thoughts as well. In Grant's words, "Write the docs!" https://phabricator.wikimedia.org/phame/post/view/194/the_best_documentatio… Stephen

2 1

Subject=Re: Announcing Citation Detective, a public dataset of sentences missing citations
by Ai-Jou Chou 23 Mar '20

23 Mar '20

Hi Emufarmers! Thanks for your attention! :) Yes, the tool might detect the sentences that already referenced, because Citation Detective actually feeds *every single sentence* in an article to Citation Need model and extract sentences with high scores. Highlighting a sourced sentence doesn't mean the source used is unreliable as Citation Detective has no idea of whether a sentence has a reference and the content of the source. I would say it's just like double confirm that the sentence needs a citation, and yes there is a citation already. You might have questions why the tool didn't exclude those sentences already have a reference. The reason is a reference doesn't necessarily apply just to the sentence right before it. They could apply to more than one sentence or a whole paragraph, and there's no way to determine that from the Wikitext. So that's why the tool was designed (at least for the initial version) to feed every sentence to the model and detect citation need score. There's the same case with { citation needed } tags, so you would also find sentences with {cn} tag highlighted in the prototype. That means both human and machine think the statement needs a citation to a reliable source. I hope this clarifies things for you. :) Aiko

1 0

Questions - GSoC
by Stud 20160227 22 Mar '20

22 Mar '20

Dear, Hope This mail finds you well. First of all I would like to introduce myself to you... I'm Mahmoud Ahmed studying computer science at Cairo university. I am currently trying to join GSoC 2020 (Add 'Reverted' filter to RC Filters) But I have some questions, please. Can you give me more details clearly about this project (Add 'Reverted' filter to RC Filters) Because I am very excited about it. Finally... Thank you very much in advance. Regards, Mahmoud Ahmed. --

3 7

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l March 2020