[Wikitech-l] 2018-05-02 Scrum of Scrums meeting notes

https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-05-02 ==2018-05-02 == == Callouts == * Fundraising campaigns https://meta.wikimedia.org/wiki/CentralNotice/Calendar - next up: Netherlands 2018-04-10through 2018-05-08 * Services: If someone maintains/knows JobQueue Job code that posts jobs cross-wiki, please tell us. * Language -> Platform: Need help in https://phabricator.wikimedia.org/T193008(MediaWiki\MediaWikiServices::rese… doesn't reset database connection state) * If you have a WMF service that uses WDQS, please document at https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Usage and consider whether it should be using internal cluster or public endpoint. * Research: If you speak English, French, or Spanish, you're welcome to label some unsourced statements at http://labels.wmflabs.org. Look for the "Labeling Unsourced Statements" (or its equivalent in other languages) campaign. * Research: If you speak two languages from the list of Arabic, English, Spanish, Japanese, French, and Russian, you're also welcome to help us with mapping section titles. https://docs.google.com/spreadsheets/d/1KjUiMERifNvoqxaXYo3aymjELHrIkyX9wgC… * Research: We would love to hear about a robust way of sending out user talk page messages across wikis, not just enwiki. == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: **Point release should be code complete this week; mostly small bug fixes + Wikidata magic word override (https://phabricator.wikimedia.org/T191347) ==== Android native app ==== * Blocked by: * Blocking: * Updates: **Working on multilingual extensions. Planning to complete before Hackathon, and perform user testing there. ==== Readers Web ==== * Blocked by: * Blocking: * Updates: **Some rather involved improvements to infobox alignment https://phabricator.wikimedia.org/T189688 **Refactoring jQuery collapsible elements to use CSS instead of JavaScript https://phabricator.wikimedia.org/T42812 **Improvements, bug fixes, and analyses to page previews; better RTL support https://phabricator.wikimedia.org/T192928 https://phabricator.wikimedia.org/T190831 https://phabricator.wikimedia.org/T193058 https://phabricator.wikimedia.org/T192773 https://phabricator.wikimedia.org/T188734 https://phabricator.wikimedia.org/T193053 **Miscellaneous watchstar bug fixes wrapping up https://phabricator.wikimedia.org/T191454 **Reviewing applicants for full stack engineer backfill **Page issues instrumentation planning https://phabricator.wikimedia.org/T191532 **Little slow this week due to holidays and vacations **Ongoing minor improvements to MobileFrontend and MinervaNeue *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome 1, Objective 4]]: Continue improving the ways that users can download articles of interest for later consumption *** Reading Web depends on SRE, RelEng, Reading Infra ==== Readers Infrastructure ==== * Blocked by: * Blocking: * Updates: ** Updated summary output to use the correct description if it was overridden. Same for mobile-sections plus also added the description_source so the apps can see where the description came from. (deployed as of Monday, going to roll out to pages as they are edited) ** ReadingLists response times went up 10x, might be related to ChronologyProtector change (there is a fix, thanks Performance!) https://phabricator.wikimedia.org/T193417 *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome 1, Objective 4]]: Continue improving the ways that users can download articles of interest for later consumption *** Reading Web depends on SRE, RelEng, Reading Infra **[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of client apps by coalescing and moving more logic to the server]] ***Reading Infra depends on Parsing, Services ===== Maps ===== * Blocked by: * Blocking: * Updates: ** ==== Multimedia ==== * Updates ** Continuing work on prototype for file-page integration for MediaInfo ** Search work progressing well - moving on from properties to qualifiers ** Still considering instrumentation possibilities *Quarterly goal dependency update: **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 3.1]] Prepare for launch of the first Structured Data on Commons feature (multilingual file captions) ***SDC depends on Multimedia,SRE, WMDE, Search Platform, MediaWiki Platform, Research ** [[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 2.1]] Integrate structured file captions into search *** SDC depends on Search Platform, Multimedia **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Structured_Data#Segment_4:_Programs|Segment 4, Outcome 2]]: Develop a better understanding of existing needs for Structured Commons- T171252 ***Research depends on Multimedia === Contributors === ==== Community Tech ==== * Blocked by: * Blocking: * Updates: ** Starting a new project, AFC improvement ==== Anti-Harassment Tools ==== * Blocked by: * Blocking: * Updates: ** ==== Editing ==== * Blocked by: * Blocking: ** Updates: ** ==== Parsing ==== * Blocked by: Operations on https://phabricator.wikimedia.org/T193414 -- Production Debian stretch servers are running a different version of Tidy compared to Debian Jessie servers * Blocking: * Updates: ** *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.6]] Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team *** Parsing depends on MediaWiki Platform, Services **** Work Ongoing. (1) Tidy will be gone by end of quarter, and in touch with Platform team about it (2) Language variants work is apace. Services & Parsing are yet to meet to work about API spec for what Services wants Parsing to implement. **[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of client apps by coalescing and moving more logic to the server]] Increase code sharing of client apps by coalescing and moving more logic to the server. *** Reading Infra depends on Parsing, Services **** Not aware of anything that Parsing needs to do (unless it is language variants, which is already being addressed as part of our own goal above). [[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better data|Outcome 2: Objective 1]]: Revision storage scaling *** Services depends on SRE, Parsing **** Services, Parsing, Editing need to meet and resolve if data-mw blob separation is happening this quarter or not. ==== Collaboration ==== * Blocked by: * Blocking: * Updates: ** ==== Language ==== * Blocked by: See callouts * Blocking: None * Updates: ** ContentTranslation Version 2 work continue. *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.1]] Improve and consolidate our unified editing platform so that it's great on all devices ***Language depends on Editing === Audiences Design === * Blocked by: * Blocking: * Updates: ** ==== UI Standardization ==== * Blocked by: * Blocking: * Updates: ** == Technology == === Analytics === * Blocked by: * Blocking: * Updates: ** Druid has been upgraded to 0.10 (on the road to 0.11, allowing to move to java8 and therefore stretch instead of jessie) ** http://pivot.wikimedia.org is having problems for some datasources (banner-impressions) - We ask people to start using Superset ( superset.wikimedia.org - wikitech account + ping to analytics needed) ** Geowiki (editors per country) is now named GeoEditors. Data has been vetted against old data (see https://wikitech.wikimedia.org */wiki/Analytics/Systems/GeoEditors) ** Mirror Maker still has issues. We are upgrading jumbo to Kafka 1.1 before rolling it to the main cluster, with ops collaboration ** EventBus logs are pushed to logstash ** On the way to upgrade ua-parser, the library we use to classify user agents ** Dashiki can now pivot data (example of configuration change: https://meta.wikimedia.org/w/index.php?title=Config%3ADashiki%3AInterlangua…did=17894735, example of dashboard using it: https://language-reportcard.wmflabs.org/interlanguage/#desktop) === Cloud Services === * Blocked by: * Blocking: * Updates: ** === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Infrastructure for transition to EventLogging for data on FR cluster ** Fixing up some errors in the donations queue ** Setting up BitPay ** Ongoing fixes and improvements in existing systems: CiviCRM, Ingenico, Grafana dashboard === MediaWiki Platform === * Blocked by: * Blocking: * Updates: ** MCR *** Task wrangling *** Working to resolve architectural disagreements *** Migration script to populate the new MCR tables is done, waiting for MCR people to review. *** ar_text/ar_flags removal is blocked on Cloud Services: https://phabricator.wikimedia.org/T189141 *** Wrote a maintenance script to deduplicate ar_rev_id, needs review: https://gerrit.wikimedia.org/r/q/topic:%2522bug%252FT193180-cleanup-archive… *** API updates for query modules are in Gerrit: https://gerrit.wikimedia.org/r/#/c/413223/ *** Started working on the API "templated parameters" patch again. Almost ready to put it in Gerrit. Described in https://phabricator.wikimedia.org/T174032#3990488. ** MediaWiki 1.31 blocker work ** Usual code review and bug work *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.6]] Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the first Structured Data on Commons feature]] (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research ** [[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 1.1]] Assist with deploying MultiContent Revisions on Commons [[phab:T174022|T174022]] Implement multi-content revisions, [[phab:T174023|T174023]] Implement MCR storage layer, [[phab:T174045|T174045]] DB schema migration for MCR, [[phab:T174044|T174044]] Deploy MCR storage layer, [[phab:T174043|T174043]]Deploy Multi-Content Revisions ***SDC depends on MediaWiki Platform, WMDE === Performance === * Blocked by: ** * Blocking: ** * Updates: ** === Release Engineering === * Blocked by: none * Blocking: none? * Updates: ** Announcing Quibble! https://phabricator.wikimedia.org/phame/post/view/99/introducing_quibble/ ** MediaWiki core and extensions 1.32.0-wmf.2 going out this week [ https://phabricator.wikimedia.org/T191048 blocker task] if you spot blockers ** New Scap Release coming Soon™ *** git-lfs updates for ORES *** better canary stuff for MediaWiki deploys *Quarterly goal dependency update: ** no updates **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome 1, Objective 4]]: Continue improving the ways that users can download articles of interest for later consumption *** Reading Web depends on SRE, RelEng, Reading Infra === Research === * Blocked by: None * Blocking: None * Updates: ** Sending out invitations to multilingual Wikipedia users and asking their help in mapping article section titles into different languages (ar, en, es, fr, ja, ru). Would love to hear if anyone knows about a robust way of sending talk page messages across wikis, not just enwiki. You're welcome to contribute too: https://docs.google.com/spreadsheets/d/1KjUiMERifNvoqxaXYo3aymjELHrIkyX9wgC… ** Running a Wikilabels campaign for unsourced statements (en, fr, it). So far we have 25 independent annotators and more than 300 labels. Your contributions are also welcome: http://labels.wmflabs.org *Quarterly goal dependency update: Jonathan is driving these initiatives. **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the first Structured Data on Commons feature]] (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Structured_Data#Segment_4:_Programs|Segment 4, Outcome 2]]: Develop a better understanding of existing needs for Structured Commons- [[phab:T171252|T171252]] ***Research depends on Multimedia === Scoring Platform === * Blocked by: None * Blocking: None * Updates: **git-lfs is almost finished **wp10 data is available on beta cluster and will go to prod soon === Search Platform === * Blocked by: * Blocking: * Updates: ** WDQS internal cluster is up and serving traffic. ** Fixed error in negating keywords: https://phabricator.wikimedia.org/T193208 ** Working on Polish analyzer: https://phabricator.wikimedia.org/T186046 ** Evaluating feature generation by `query_explorer` functionality of LTR plugin: https://phabricator.wikimedia.org/T187148 ** Working on improving sharding of Elastic cluster: https://phabricator.wikimedia.org/T192972 ** Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108 ** Working on indexing more statements in Wikidata: https://phabricator.wikimedia.org/T99899 ** WDQS GUI auto-build working, thanks again RelEng! *Quarterly goal dependency update: **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the first Structured Data on Commons feature]] (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|*Objective 2.1]] Integrate structured file captions into search *** SDC depends on Search Platform, Multimedia === Security === * Blocked by: * Blocking: * Updates: ** *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.6]]: Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services === Services === * Blocked by: none * Blocking: none * Updates: ** RESTBase storage clenups ongoing ** Job Queue migration continues *** The job type config must be global for all the jobs that are posted cross-wiki, so our normal switchover process doesnt work for them. We've identifie most of cases, but we need help if someone knows a job we might have missed. *Quarterly goal dependency update: **[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of client apps by coalescing and moving more logic to the server.]] ***Reading Infra/Parsing, Services **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.6]] Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services [[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better data|Outcome 2: Objective 1]]: Revision storage scaling *** Services depends on SRE, Parsing === Site Reliability Engineering === * Blocked by: * Blocking: * Updates: ** *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome 1, Objective 4]]: Continue improving the ways that users can download articles of interest for later consumption *** Reading Web depends on SRE, RelEng, Reading Infra **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the first Structured Data on Commons feature]] (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research [[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better data|Outcome 2: Objective 1]]: Revision storage scaling *** Services depends on SRE, Parsing == Wikidata == * Blocked by: ** MediaWiki core's selenium tests flaky when run as part of mwext-mw-selenium-node-composer-jessie job - https://phabricator.wikimedia.org/T191537 **CI infrastructure Issues *** What level of service is expected, what to do when the service level "drops" too much? *** What amount of contribution can/should be expected from our team, how much can/should we get involved? * Blocking: * Updates: **Release of Lexeme is still on schedule - May 23rd. ** *Quarterly goal dependency update: **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the first Structured Data on Commons feature]] (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 1.1]] Assist with deploying MultiContent Revisions on Commons [[phab:T174022|T174022]] Implement multi-content revisions, [[phab:T174023|T174023]] Implement MCR storage layer, [[phab:T174045|T174045]] DB schema migration for MCR, [[phab:T174044|T174044]] Deploy MCR storage layer, [[phab:T174043|T174043]]Deploy Multi-Content Revisions ***SDC depends on MediaWiki Platform, WMDE == German Technical Wishlist == * Blocked by: * Blocking: * Updates: ** == SoS Meeting Bookkeeping == * Updates: **