Wikitech-l June 2019

wikitech-l@lists.wikimedia.org

54 participants
64 discussions

Scrum of scrums/2019-06-19
by Željko Filipin 20 Jun '19

20 Jun '19

Hi, for HTML version see https://www.mediawiki.org/wiki/Scrum_of_scrums/2019-06-19 I'm not sure if I'm reading meeting notes correctly, but looks like Performance is blocked on the entire Audiences? === Performance === * Blocked by: ** Audiences, code review of: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Graph/+/493628/ Also, is Site Reliability Engineering is blocking WMDE or not? === Site Reliability Engineering === * Blocking: ** WMDE has been unblocked on termbox Željko -- = 2019-06-19 = == Callouts == * SRE DBAs point out that: ** 19th June Wed 05:00 UTC - s4 primary db master failover - read only required ** 25th June Wed 06:00 UTC - change the last parsercache key https://phabricator.wikimedia.org/T210725 * SRE Service Operations require assistance (from Release Engineering) on https://phabricator.wikimedia.org/T224857 * Release Engineering: Train Health This week: 1.34.0-wmf.10 - https://phabricator.wikimedia.org/T220735 ** Core Platform: Jobs not being executed on 1.34.0-wmf.10 https://phabricator.wikimedia.org/T226109 ** Multimedia: BadMethodCallException on Commons: /includes/specials/pagers/ImageListPager.php: Call to a member function getUrl() on a non-object (boolean) https://phabricator.wikimedia.org/T226102 == Audiences == === Contributors === ==== Editing ==== * Updates: ** After qualitative and quantitave tests with positive results, we completed a full rollout of section editing on all wikis ** Mobile Edit Cards: released v1.0 of mobile VE Edit Cards to Bengali, Hebrew and Persian wikis ** Design on Edit Cards v2.0 is ongoing. That work is happening in task:T225725 ==== Growth ==== * Blocking: ** Performance: code review of: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/508965/ === Readers === ==== iOS native app ==== * Updates: ** Finishing up 6.3 work (talk pages) ( https://phabricator.wikimedia.org/tag/ios-app-v6.3-lobster-on-a-motorcycle/) ** Preparing for a bug fix release 6.3.1 ( https://phabricator.wikimedia.org/tag/ios-app-v6.3.1_lobster-on-an-orv/) ==== Android native app ==== * Updates: ** Completed development of structured image caption editing (with suggested edit feed). Continuing to test; will soon release to beta. ==== Readers Web ==== * Updates: ** Summary: started tasking out next year's technical work for MobileFrontend and continuing ongoing development for the advanced mobile contributions project. ** Responsive website (MinervaNeue / MobileFrontend / Portals): *** Advanced mobile contributions https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions **** [https://phabricator.wikimedia.org/T222630 <nowiki>Add wikidata item link to AMC overflow menu</nowiki>] **** [https://phabricator.wikimedia.org/T221792 <nowiki>PageActions Menu should use Builder pattern and reuse existing Menu elements</nowiki>] *** Invest in the MobileFrontend & MinervaNeue frontend architecture https://www.mediawiki.org/wiki/Reading/Web/Projects/Invest_in_the_MobileFro… **** [https://phabricator.wikimedia.org/T225447 Tasking for next year] *** Browser tests **** [https://phabricator.wikimedia.org/T224605 <nowiki>MinervaNeue wdio build fails with PHP error from Wikibase "Failed to map interlanguage prefix es to a global site ID."</nowiki>] ** QuickSurveys support **** [https://phabricator.wikimedia.org/T218304 <nowiki>Allow quicksurveys to target based on registration date</nowiki>] ** Planning *** Desktop refresh design and planning *** Wikimania proposal planning ==== Readers Infrastructure ==== * Updates: ** PCS/MCS split *** Wikifeeds patches need +2s: https://gerrit.wikimedia.org/r/#/q/project:mediawiki/services/wikifeeds++st… ** Page-library *** New version released but had some issues *** Work on interaction handling continues *** Adjust text size patch complete *** TemplateStyles CSS appears in quick facts header *** PR for mobile-html section offsets ( https://phabricator.wikimedia.org/T219909) ** Kubernetes migration *** Mobileapps, Wikifeeds, and Recommendation-API all running CI tests on service pipeline, ready to migrate at SRE's convenience ==== Multimedia ==== * Blocking: ** Release Engineering: BadMethodCallException on Commons: /includes/specials/pagers/ImageListPager.php: Call to a member function getUrl() on a non-object (boolean) https://phabricator.wikimedia.org/T226102 * Updates: ** qualifiers for depicts statements finally going out tomorrow ** offsite next week ** other statements should get onto test during offsite == Technology == === Core Platform === * Blocking: ** Performance: code review of https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/414968/ and https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/505468/ ** Release Engineering: *** Jobs not being executed on 1.34.0-wmf.10 https://phabricator.wikimedia.org/T226109 *** (low priority): https://phabricator.wikimedia.org/T205361 is blocking undeployment of CodeReview. === Performance === * Blocked by: ** CPT, code review of https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/414968/ and https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/505468/ ** Audiences, code review of: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Graph/+/493628/ ** Growth, code review of: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/508965/ * Updates: ** Recent blog posts about performance perception findings: https://phabricator.wikimedia.org/phame/blog/view/7/ ** Timo working on making Mediawiki CI gate job faster: https://phabricator.wikimedia.org/T225730 === Release Engineering === * Blocked by: ** Core Platform Team (low priority): https://phabricator.wikimedia.org/T205361 is blocking undeployment of CodeReview. ** SRE: *** Traffic Team (low priority): https://phabricator.wikimedia.org/T213769 is blocking undeployment of Wikipedia Zero. *** ServiceOps Team: Scap 3.10.0: https://phabricator.wikimedia.org/T224915 ** Wikidata: We need to update wikiba.se hosting to PHP7 so we can drop php56 from CI. https://phabricator.wikimedia.org/T224905 * Blocking: ** SRE Service Operations requires assistance on https://phabricator.wikimedia.org/T224857 * Updates: ** Train Health *** Last week: 1.34.0-wmf.9 - NO TRAIN OR ANY OTHER DEPLOYS due to SRE Off-site *** This week: 1.34.0-wmf.10 - https://phabricator.wikimedia.org/T220735 **** Core Platform: Jobs not being executed on 1.34.0-wmf.10 https://phabricator.wikimedia.org/T226109 **** Multimedia: BadMethodCallException on Commons: /includes/specials/pagers/ImageListPager.php: Call to a member function getUrl() on a non-object (boolean) https://phabricator.wikimedia.org/T226102 *** Next week: 1.34.0-wmf.11 - https://phabricator.wikimedia.org/T220736 === Search Platform === * Blocked by: ** Wikidata: RDF for Structured Data on Commons https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/507910 and https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/489447 (fixes in the making) === Security === * Updates: ** Backlog *** Brian still performing code review, this is a tracking ticket for follow-ups for library/vendor code, probably need to reassign not started - https://phabricator.wikimedia.org/T223307 *** Exploring outsourced reviews for OIT things, rewrites, JB to follow up after new vendor budgets set stalled - https://phabricator.wikimedia.org/T155537, https://phabricator.wikimedia.org/T156960, https://phabricator.wikimedia.org/T148246, https://phabricator.wikimedia.org/T187846 *** DoubleWiki, Brian to deploy as sec release, then push through gerrit, stalled - https://phabricator.wikimedia.org/T131199 *** Audiences user-testing concept review, stalled - https://phabricator.wikimedia.org/T220043 ** Active *** Waiting on update from Yurik/Vega folks (upstream) to proceed stalled - https://phabricator.wikimedia.org/T222806 *** Planet wikimedia - assigned to Michal Anna, stalled - https://phabricator.wikimedia.org/T207246 *** Wikisource ext, starting - https://phabricator.wikimedia.org/T217289 *** FormWizard extension, Brian to write response, tag releng stalled - https://phabricator.wikimedia.org/T201492 ** Waiting *** Wikibase Termbox, completed for now, John to follow up with Raz re: risk ownership, in-progress - https://phabricator.wikimedia.org/T216419 *** Labs db/sanitarium and maintain-views.yaml audits, in-progress - https://phabricator.wikimedia.org/T169097, https://phabricator.wikimedia.org/T103011 *** Audiences growth team emails concept review, stalled - https://phabricator.wikimedia.org/T220242 === Site Reliability Engineering === * Blocked by: ** Release Engineering: SRE Service Operations requires assistance on https://phabricator.wikimedia.org/T224857 * Blocking: ** WMDE has been unblocked on termbox ** Release Engineering: *** Traffic Team (low priority): https://phabricator.wikimedia.org/T213769 is blocking undeployment of Wikipedia Zero. *** ServiceOps Team: Scap 3.10.0: https://phabricator.wikimedia.org/T224915 * Updates: * SRE DBAs point out that: ** 19th June Wed 05:00 UTC - s4 primary db master failover - read only required ** 25th June Wed 06:00 UTC - change the last parsercache key https://phabricator.wikimedia.org/T210725 == Wikidata == * Blocking: ** Search Platform: RDF for Structured Data on Commons https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/507910 and https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/489447 (fixes in the making) ** Release Engineering: We need to update wikiba.se tests to PHP7 so we can drop php56 from CI. https://phabricator.wikimedia.org/T224905 * Updates: ** Started the migration of Wikidata secondary storage (wb_terms) to a better designed tables to start next ** Started working on the Wikidata bridge (editing Wikidata without leaving Wikipedia) ** Work on the mobile termbox editing progresses

3 2

Fwd: Feedback regarding Hashtags tool's statistics page
by Aditya Jain 20 Jun '19

20 Jun '19

---------- Forwarded message --------- From: Aditya Jain <adi2007jain(a)gmail.com> Date: Tue 18 Jun, 2019, 4:52 PM Subject: Feedback regarding Hashtags tool's statistics page To: <libraries(a)lists.wikimedia.org>, <1lib1ref(a)lists.wikimedia.org>, < wikitech-l(a)lists.wikimedia.org> Hashtag Search <https://hashtags.wmflabs.org/> is a tool that allows users to search for uses of hashtags in Wikimedia edit summaries. As of now, the functionalities of the tool are quite basic - Few simple search options are provided and the results are listed with the option to download them as CSV. Many Wikimedia campaigns such as 1Lib1Ref use this tool to track edits and users might want to get into more details for a particular search. Along with my mentor Sam Walton, I have been working on a GSoC project “Create a subpage for statistics and charts related to hashtag search’’ <https://phabricator.wikimedia.org/T186706>. The aim of this project is to create a subpage for helpful statistics and graphs related to the results of a search. After making some progress we have displayed the three primary graphs that we were looking for - Top projects, Top users and edits over time. Users of the tool can also download each of these statistics as CSV. They can also view a complete list of editors and projects. We have setup a test instance of the tool which you can find here: https://hashtags-staging.wmflabs.org/ After clicking on the above link a search form is provided where you can search for a hashtag. Fill the form and make a search (You can also click on one of the trending tags). Click on the “Show statistics” button in the stats box. You will now arrive at the statistics page. Scroll the page to see the graphs. You can also download the results as CSV. Search form is also provided on the statistics page so that you could modify the search directly. Now we are trying to collect feedback from users of the tool so that we can enhance the current functionality of the tool. Also we would love to know what more statistics we could show on the subpage. If you have any suggestions for us please let us know by commenting on my Pull Request: https://github.com/WikipediaLibrary/hashtags/pull/31 or commenting on the Phabricator for the project: https://phabricator.wikimedia.org/T218986. Thanks, Aditya

1 0

Redis job runner
by Aran 20 Jun '19

20 Jun '19

Hello, I've set up the redisJobRunnerService and redisJobChronService from here: https://github.com/wikimedia/mediawiki-services-jobrunner They're working well and processing jobs as they should be, but I was wondering if anyone knows of any documentation about the configuration to fine-tune the prioritisation of various job types differently. All I've really been able to find about the configuration is the README in the repo. Thanks, Aran

1 0

The revisions watching database is blocked for ten hours
by יגאל חיטרון 19 Jun '19

19 Jun '19

Hello. At least ten hours the revisions watching database is blocked. Reproducing steps: 1) Open the history of some page you have non-visited new revisions in your watchlist. 2) Make sure you see green marks "updated since my last visit". 3) Open the page itself. 4) Go back to the history. 5) The green marks removed as expected. 6) Wait five minutes. 7) Refresh the history. 8) The marks are back. The problem exists on some wikis, not all of them. For example, I can see it on hewiki and can't on ruwiki. There is a possibility that hewiki is the only one. Before I file a phab task, was there some anticipated database blocking? Thank you. Igal (User:IKhitron)

3 4

Phabricator monthly statistics - 2019-05
by communitymetrics＠wikimedia.org 19 Jun '19

19 Jun '19

Hi Community Metrics team, This is your automatic monthly Phabricator statistics mail. Accounts created in (2019-05): 272 Active Maniphest users (any activity) in (2019-05): 1036 Task authors in (2019-05): 563 Users who have closed tasks in (2019-05): 328 Projects which had at least one task moved from one column to another on their workboard in (2019-05): 292 Tasks created in (2019-05): 2548 Tasks closed in (2019-05): 2149 Open and stalled tasks in total: 42273 Median age in days of open tasks by priority: Unbreak now: 12 Needs Triage: 495 High: 867 Normal: 1091 Low: 1499 Lowest: 1464 (How long tasks have been open, not how long they have had that priority) Active Differential users (any activity) in (2019-05): 10 TODO: Numbers which refer to closed tasks might not be correct, as described in https://phabricator.wikimedia.org/T1003 . Yours sincerely, Fab Rick Aytor (via community_metrics.sh on phab1003 at Wed Jun 19 08:27:43 UTC 2019)

2 1

Commonswiki db primary master failover (read-only required) 19th June 05:00 AM UTC
by Manuel Arostegui 19 Jun '19

19 Jun '19

Hello, We have requested a 30 minutes read-only window for s4 (commonswiki) for the 19th June from 05:00-05:30 UTC to switchover our primary commonswiki database master - https://phabricator.wikimedia.org/T224516 db1068 is an old host that will be decommissioned (T217396) and which also suffers from intermittent memory problems (T213664). We are going to do this Wednesday 19th June from 05:00 to 05:30 AM UTC (we do not expect to use the 30 minutes window, if everything goes as expected). Impact: Writes will be blocked, reads will remain unaffected. Communication will happen at #wikimedia-operations If you are around at that time and want to help with the monitoring, please join us! Thanks Manuel.

1 2

Technical Conference 2019 - Call for participant nominations
by Greg Grossmeier 19 Jun '19

19 Jun '19

Hello all, As you may have seen, the next Wikimedia Technical Conference[0] is coming up in November 2019. It will take place November 12-15th in Atlanta, GA (USA). As announced at the Hackathon and documented on-wiki[1] this year's event will focus on the topic of "Developer Productivity". Like last year, we are looking for diverse stakeholders, perspectives, and experiences that will help us to make informed decisions. We need people who can create and architect solutions, as well as those who will make funding and prioritization decisions for the projects. See the FAQ for (hopefully) any questions you have: <https://www.mediawiki.org/wiki/Wikimedia_Technical_Conference/2019/FAQ> Please fill out the survey using this link to nominate yourself or someone else to attend: <https://forms.gle/CLeGFSMiEasJgEU27> This survey is conducted via a third-party service, which may make it subject to additional terms. For more information on privacy and data-handling, see this survey privacy statement: <https://foundation.wikimedia.org/wiki/Wikimedia_Technical_Conference_Survey…> This nomination form will remain open between May 29 and June 17, 2018. If you have any questions, please post them on the event's talk page <https://www.mediawiki.org/wiki/Talk:Wikimedia_Technical_Conference/2019>. Thanks! Greg and the Technical Conference 2019 Program Committee [0] <https://www.mediawiki.org/wiki/Wikimedia_Technical_Conference/2019> [1] <https://www.mediawiki.org/wiki/Wikimedia_Technical_Conference/2019#Vision_S…> -- Greg Grossmeier Release Team Manager

1 2

Write expectations exception
by Aran 19 Jun '19

19 Jun '19

|In a MediaWiki-based project I'm working on I'm getting many of these kinds of exceptions: [DBPerformance] Expectation (writes <= 0) by MediaWiki::main not met (actual: 8) | |I've read up on the Database transactions article in mediawiki.org and can see that to remove the exceptions we'd need to conform to some very specific criteria for all our db write, which is quite a problem for this particular project. My question is, are these criteria ever likely to be enforced in future MW versions, or will they always just be warnings to help guide performance improvements? |||

2 2

Wednesday: Technical Advice IRC Meeting
by Raz Shuty 18 Jun '19

18 Jun '19

Sorry for cross-posting! Reminder: Technical Advice IRC meeting this week **Wednesday 3-4 pm UTC** on #wikimedia-tech. Questions can be asked in English and Persian! The Technical Advice IRC Meeting (TAIM) is a weekly support event for volunteer developers. Every Wednesday, two full-time developers are available to help you with all your questions about Mediawiki, gadgets, tools and more! This can be anything from "how to get started" over "who would be the best contact for X" to specific questions on your project. If you know already what you would like to discuss or ask, please add your topic to the next meeting: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting Hope to see you there! -- Raz Shuty Engineering Manager Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30 219 158 26-0 https://wikimedia.de Imagine a world, in which every single human being can freely share in the sum of all knowledge. That‘s our commitment. Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1 0

CI jobs run time
by Amir Sarabadani 17 Jun '19

17 Jun '19

Hey, In the past couple of weeks, jenkins jobs became considerably slow. There has been some actions to improve it. * There's the list of slow phpunit tests [0] ** One test in CirrusSearch was taking 1 minutes and 30 seconds, that's fixed and now it takes around 20 seconds [1] ** There was an innocent test that was taking 8.6 seconds because it was sleeping 8.5 seconds of it to make sure the cache is expired, that is fixed as well [2] ** There was a rather slow test in Babel extension. It's cleaned up [3] * Mediawiki caching in browser tests has changed from database-based cache to APC cache, dropping ~1200 queries in every jenkins test job. Saving 30 seconds from core CI jobs and more in extensions [4] (It was a really big low-hanging fruit) * There's a work to separate unit tests and integration tests that beside other benefits, it saves around 10 seconds of each phpunit job. It got merged but it had to be reverted due to breaking Postgres tests, hopefully will get it fixed and re-merged soon [5] * There has been several discussions on how to improve performance of browser tests [6] 🎉🎉🎉This chart sums up the result of the work done so far. Hopefully it'll get even better [7] 🎉🎉🎉 == What you can do? == * Please take a look at [0] and see if the extension you are maintaining or developing has strangely long tests, if so, please fix it. * Let's get [5] re-merged! * Please participate at the discussions at [6] about browser tests [0]: https://integration.wikimedia.org/ci/job/wmf-quibble-core-vendor-mysql-hhvm… [1]: https://phabricator.wikimedia.org/T225184 [2]: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/515023 [3]: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Babel/+/514735 [4]: https://phabricator.wikimedia.org/T225496 [5]: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/513106 [6]: https://phabricator.wikimedia.org/T225248 [7]: https://phabricator.wikimedia.org/F29585158 Have a lovely day -- Amir Sarabadani (he/him) Software engineer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Tel. (030) 219 158 26-0 https://wikimedia.de Unsere Vision ist eine Welt, in der alle Menschen am Wissen der Menschheit teilhaben, es nutzen und mehren können. Helfen Sie uns dabei! https://spenden.wikimedia.de Wikimedia Deutschland — Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

4 3

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l June 2019