[Wikitech-l] 2018-04-11 Scrum of Scrums meeting notes

https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-04-11 = 2018-04-11 = *Q4 FY1718* == Callouts == * Fundraising campaigns https://meta.wikimedia.org/wiki/CentralNotice/Calendar - Netherlands 2018-04-03 through 2018-05-01 * OpenStack upgrades on Friday 2018-04-13 -- Horizon and CI (Zuul/Jenkins/Nodepool) will be disabled for a while beginning around 14:00 UTC * Still need help from RelEng with WDQS GUI build automation: https://phabricator.wikimedia.org/T189811 == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: ** ==== Android native app ==== * Blocked by: * Blocking: * Updates: ** Reading list sync rolled out to 10% of production (~800K users) ** On track to roll out to full production next week. ==== Readers Web ==== * Blocked by: [performance] https://phabricator.wikimedia.org/T42812 - nice collaboration going on here, to improve mw-collapsible behaviour on desktop - [releng] https://phabricator.wikimedia.org/T190710 - getting browser tests ported to Node for Minerva - Thanks Željko for being so responsive! * Blocking: * Updates: ** Updating clients to reflect new wikidata description changes ( https://phabricator.wikimedia.org/T190953) **Deploying Page previews to German Wikipedia (right now!!) *Quarterly goal dependency update: **Outcome 1, Objective 4: Continue improving the ways that users can download articles of interest for later consumption *** Reading Web depends on SRE, RelEng, Reading Infra ==== Readers Infrastructure ==== * Blocked by: * Blocking: * Updates: ** CSS endpoint going to be deployed in RESTBase after MediaWiki train completes ** Working on very simplistic browser extension to add a single page to a reading list ** TemplateStyles was deployed on ruwiki *Quarterly goal dependency update: **Outcome 1, Objective 4: Continue improving the ways that users can download articles of interest for later consumption *** Reading Web depends on SRE, RelEng, Reading Infra **Increase code sharing of client apps by coalescing and moving more logic to the server. ***Reading Infra depends on Parsing, Services ===== Maps ===== * Blocked by: None * Blocking: None * Updates: ** Working on internationalization and enwiki rollout ==== Multimedia ==== * Updates ** Wikibase patches for file page prototype going through ** Search work continuing ** File captions - currently working through changing the name of the field for MediaInfo - it's a bear *Quarterly goal dependency update: **Objective 3.1 Prepare for launch of the first Structured Data on Commons feature (multilingual file captions) ***SDC depends on Multimedia,SRE, WMDE, Search Platform, MediaWiki Platform, Research ** Objective 2.1 Integrate structured file captions into search *** SDC depends on Search Platform, Multimedia **Segment 4, Outcome 2: Develop a better understanding of existing needs for Structured Commons- T171252 ***Research depends on Multimedia === Contributors === ==== Community Tech ==== * Blocked by: * Blocking: * Updates: ** CodeMirror is getting ready to promotion from beta ** Test deployment of GlobalPreferences today ==== Anti-Harassment Tools ==== * Blocked by: * Blocking: * Updates: Blocking Tool Improvments ==== Editing ==== * Blocked by: * Blocking: ** Updates: ** Goal 3.1 Improve and consolidate our unified editing platform so that it's great on all devices ***Editing depends on Parsing ==== Parsing ==== * Blocked by: * Blocking: * Updates: ** Tidy replaced with RemexHtml on 40 more wikis today (see https://phabricator.wikimedia.org/T190731 ) Currently have about 650 running RemexHTML, another 250 yet to convert. ** Parsoid is exploring the use of Finite State Transducer formalism for language variants ( see https://phabricator.wikimedia.org/T191925 && gerrit - https://gerrit.wikimedia.org/r/#/c/423197/ ) ** TTO and us had an IRC meeting last week about the Poem extension and TTO summarized the decisions in the phabricator task ( see https://phabricator.wikimedia.org/T54061#4110198 ) ** No quarterly goal dependency updates at this time. Services, Editing, Parsing teams will be syncing up this/next week to identify any blockers. *Quarterly goal dependency update: **Goal 3.6 Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services **Increase code sharing of client apps by coalescing and moving more logic to the server. ***Reading Infra depends on Parsing, Services ** Goal 3.1 Improve and consolidate our unified editing platform so that it's great on all devices ***Editing depends on Parsing Outcome 2: Objective 1: Revision storage scaling - Services depends on SRE, Parsing ==== Collaboration ==== * Blocked by: * Blocking: * Updates: ** Hoping to dark-deploy maps localization this week; had some false starts and reverted deploys ==== Language ==== * Blocked by: * Blocking: * Updates: ** *Quarterly goal dependency update: **Goal 3.1 Improve and consolidate our unified editing platform so that it's great on all devices ***Language depends on Editing === Audiences Design === * Blocked by: * Blocking: * Updates: ** ==== UI Standardization ==== * Blocked by: * Blocking: * Updates: ** OOUI – v0.26.3 released: https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v… *** Apex theme now uses WikimediaUI icon, this was long overdue in regards of maintainability cost and that there were under 10% Apex specific icons left *** 4 new icons for 'tableMoveColumn*' & 'tableMoveRow*' *** CheckboxInput-/MenuOptionWidget/MenuToolGroup: Don't specify icon in CSS, reduces CSS output (Bartosz Dziewoński) ** Work on size unification aftermath in different products (Notifications, Flow, RevisionSlider) ** Style Guide technical and contents improvements ongoing with 5 open PRs currently *** PR with big impact: Using system fonts as sans-serif choice – https://phabricator.wikimedia.org/T188799 ** Transparency Report: Wrapping up accessiblity measurements (continuation from last week) ** == Technology == === Analytics === * Blocked by: traffic in a bit of issues * Blocking: * Updates: ** ** Major change on hadoop cluster to spark2, we try to avoid non backwards compatible changes ** but this actually non backwards compatible so it will be a stop-the-world change for some hours TODAY: https://phabricator.wikimedia.org/T159962 ** Mystery of the week around geo-location of pageviews for opera mini, almost there, talking to opera team this week: https://phabricator.wikimedia.org/T187014 ** Continue working on measuring popups on all sites (now feature is enabled everywhere minus dewiki and enwiki). ** Completed first release of geowiki data (editors per country), available now in superset internally: https://superset.wikimedia.org/superset/dashboard/9/ ** Issues with mirrormarker and newer versions of kafka producers not solved, job topics blocked from jumbo cluster === Cloud Services === * Blocked by: * Blocking: * Updates: ** OpenStack upgrades on Friday -- Horizon and CI will be disabled for a while beginning around 14:00 UTC ** Short blog post published on Q3 FY17/18 highlights for WMCS < https://phabricator.wikimedia.org/phame/post/view/91/cloud_services_team_q3… ** Andrew fixed a privledge escalation security issue with Horizon's puppet managment screens (T191433) ** Brooke is working on Wiki Replica view changes and needs some instruction from the DBA team on how to depool servers for maintenance there. ** Chase is working out Neutron gateway HA configuration in labtestn ** Madhu & Ariel migrated the last user-facing Dumps endpoints to the new Cloud Services managed servers === Fundraising Tech === * CentralNotice: deploy CSP headers and violation alert on forced banner previews ** Need to determine exact header to add ** Curious about the state of the larger CSP work * Custom data field changes for our CiviCRM instance * Custom reporting for CiviCRM to help find fraudsters faster * More API update work for main CC processor, lately focused on the recurring donation bits === MediaWiki Platform === * Blocked by: * Blocking: * Updates: ** MW 1.31 release planning *** Pencils down on April 16 *** considering additional bundled extensions (T178349 and subtasks) *** Making Remex default tidy implementation w/ James_F (T185753) *** Running phan for extensions that are proposed to be bundled - https://gerrit.wikimedia.org/r/#/q/status:open+topic:phan *** Run tests with PHPUnit 6 - https://gerrit.wikimedia.org/r/#/q/topic:phpunit-6 *** Handle extension dependencies in the installer (for new bundled extensions, still WIP) - https://gerrit.wikimedia.org/r/#/c/424967/ *** Also planning to do license reviews for new bundled extensions - e.g. https://gerrit.wikimedia.org/r/#/c/424726/ *** Implement PlatformSettings.php RfC - https://gerrit.wikimedia.org/r/#/c/423577/ ** MCR: *** ar_rev_id population patch merged! *** maintenance/populateArchiveRevId.php run on all wikis! *** Preparation schema change for ar_text dropping is still in progress *** Working on MCR schema migration script *** API updates for query modules are in Gerrit: https://gerrit.wikimedia.org/r/#/c/413223/ *** Started working on the API "templated parameters" described in https://phabricator.wikimedia.org/T174032#3990488. Not in Gerrit yet. *** Code review ** Actor table: *** Further deployment is waiting on the schema change (T188299) *** 17 extension updates still need review: https://gerrit.wikimedia.org/r/#/q/status:open+topic:bug/T167246-actor-migr… ** Comment table: *** Follow up schema change is still in progress https://gerrit.wikimedia.org/r/#/c/417039/ *** Toolforge view updating patch is merged and pending updates *** The image_comment_temp merging patches are in Gerrit *** All except one are blocked on the schema change ** Improving API test coverage: *** 8 changes outstanding: improved test coverage for ApiMain, ApiStructureTest, ApiBase, ApiParse, ApiMove, and ApiUserrights; two improvements to ApiTestCase (getting rid of manual session mangling and doLogin); and API usage errors in one more extension. *** This is the last week for Aryeh ** Code review and bug fixes ** Quarterly goal dependency update: **Goal 3.6 Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services **** MWPT working on relevant planning for Platform Evolution CDP to prepare for next FY and will coordinate with Parsing as this progresses **Prepare for launch of the first Structured Data on Commons feature (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research **** MWPT meets weekly with SDC and WMDE ** Objective 1.1 Assist with deploying MultiContent Revisions on Commons T174022 Implement multi-content revisions, T174023 Implement MCR storage layer, T174045 DB schema migration for MCR, T174044 Deploy MCR storage layer,T174043 Deploy Multi-Content Revisions ***SDC depends on MediaWiki Platform, WMDE **** MWPT meets weekly with SDC and WMDE === Performance === * Blocked by: None * Blocking: None * Updates: ** Aaron/Timo are helping with train delay **Moving our apps out of puppet, so that we don't need to bug SRE in order to deploy changes ** Implemented WebP lossless compression support in thumbor on Vagrant for testing, and it looks good. Gilles will be reachingout to folks on SRE and Traffic to discuss some of the potential impacts of implementing in prod. ** bunches of code review === Release Engineering === * Blocked by: * Blocking: ** scap and git-lfs work is blocking ores deployment. Mukunda is working on it. Should be unblocked by the end of this week. * Updates: ** Train is delayed, see task for updates: https://phabricator.wikimedia.org/T183968 **Quarterly goal dependency update: ***Outcome 1, Objective 4: Continue improving the ways that users can download articles of interest for later consumption **** Reading Web depends on SRE, RelEng, Reading Infra === Research === * Blocked by: * Blocking: * Updates: *Quarterly goal dependency update: **Prepare for launch of the first Structured Data on Commons feature (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research **Segment 4, Outcome 2: Develop a better understanding of existing needs for Structured Commons- T171252 ***Research depends on Multimedia === Scoring Platform === * Blocked by: ** Still blocked by git-lfs. This is urgent for us, Adam is happy to work on it synchronously if this might be helpful for RelEng? * Blocking: * Updates: === Search Platform === * Blocked by: * Blocking: * Updates: **Need help with WDQS GUI build automation: https://phabricator.wikimedia.org/T189811 **Evaluating feature generation by `query_explorer` functionality of LTR plugin: https://phabricator.wikimedia.org/T187148 **Working on some job queue issues: https://phabricator.wikimedia.org/T191024 **Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108 **Working on Lexeme completion search: https://phabricator.wikimedia.org/T190970 **Some new docs about running non-wikidata WDQS setups: https://www.mediawiki.org/wiki/Wikidata_query_service/Implementation/Standa… *Quarterly goal dependency update: **Prepare for launch of the first Structured Data on Commons feature (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research **Objective 2.1 Integrate structured file captions into search *** SDC depends on Search Platform, Multimedia === Security === * Blocked by: * Blocking: * Updates: ** *Quarterly goal dependency update: **Goal 3.6: Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services === Services === * Blocked by: Search on CirrusSearchElasticaWrite job serialization T191024 * Blocking: none * Updates: ** Switched a big bulk of jobs (9) for test wikis to the kafka queue ** restbase-dev cluster work continues, so ignore any alert for it *Quarterly goal dependency update: **Increase code sharing of client apps by coalescing and moving more logic to the server ***Reading Infra/Parsing, Services **** working with Reading Infra on new CSS endpoints rollout **Goal 3.6 Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services Outcome 2: Objective 1: Revision storage scaling ***Services depends on SRE, Parsing - === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** None * Updates: ** ICU migration current ongoing https://phabricator.wikimedia.org/T189295 ** mcrouter 0.37 released and packaged already for Debian stretch. Jessie is a bit more difficult, WIP ** Various countries routed to eqsin (Singapore PoP) once more (Japan, India, Pakistan, Australia is being experimented with still) ** image scalers no longer received thumb.php traffic ** ping offloading to a dedicated host is happening https://phabricator.wikimedia.org/T190090. Keep in mind that pings to LVS IPs from now on will not be replied to by the hosts they used to be replied from *Quarterly goal dependency update: **Outcome 1, Objective 4: Continue improving the ways that users can download articles of interest for later consumption *** Reading Web depends on SRE, RelEng, Reading Infra *** services/SRE will schedule a meeting to discuss this **Prepare for launch of the first Structured Data on Commons feature (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research Outcome 2: Objective 1: Revision storage scaling - Services depends on SRE, Parsing == Wikidata == * Blocked by: * Blocking: * Updates: ** *Quarterly goal dependency update: **Prepare for launch of the first Structured Data on Commons feature (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research ** Objective 1.1 Assist with deploying MultiContent Revisions on Commons T174022 Implement multi-content revisions, T174023 Implement MCR storage layer, T174045 DB schema migration for MCR, T174044 Deploy MCR storage layer,T174043 Deploy Multi-Content Revisions ***SDC depends on MediaWiki Platform, WMDE == German Technical Wishlist == * Blocked by: * Blocking: * Updates: ** Raising FileImporter test coverage, currently at 48%: https://phabricator.wikimedia.org/T190290 ** Looking into MediaWiki CodeSniffer performance: https://gerrit.wikimedia.org/r/#/projects/mediawiki/tools/codesniffer,dashb… ** Documenting FileImporter for performance review: https://phabricator.wikimedia.org/T191283 ** Currently deploying new Wikidiff2 to beta: https://phabricator.wikimedia.org/T190717 ** Blogging about the learnings on Wikidiff2: https://phabricator.wikimedia.org/T190176 == SoS Meeting Bookkeeping == * Updates: **please add links to Phab tickets in the goals update section