[Wikitech-l] 2018-04-04 Scrum of Scrums meeting notes

https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-04-04 = 2018-04-04 = == Callouts == * Fundraising campaigns https://meta.wikimedia.org/wiki/CentralNotice/Calendar - next up: Netherlands 2018-04-03 through 2018-05-01 * Language -> VE/Editing https://phabricator.wikimedia.org/T188072 and review of: https://gerrit.wikimedia.org/r/#/c/419402/ * Need help from RelEng with WDQS GUI build automation: https://phabricator.wikimedia.org/T189811 * Wikidata selenium test stability problems: https://phabricator.wikimedia.org/T189762 == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: ** Reading Lists sync in beta since 03/29, finishing up 5.8 ( https://phabricator.wikimedia.org/tag/ios-app-v5.8.0-manatee-on-a-skateboar… ) ==== Android native app ==== * Blocked by: * Blocking: * Updates: ** Reading List syncing in Beta for a couple of weeks, currently running survey to gather feedback from Beta users. ** Will begin initial rollout to production (10%) this week. ==== Readers Web ==== * Blocked by: ** Cached page previews not shown on refresh (services+Brandon Black) https://phabricator.wikimedia.org/T184534. We'd like to fix this but it's outside the scope of projects we usually work on. How can we move this along? * Blocking: * Updates: ** *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome 1, Objective 4]]: Continue improving the ways that users can download articles of interest for later consumption * Preparing for deploy. Deploy of VirtualPageViews is being finished today, switching focus to working out a site config deploy task. Thank you Analytics! *** Reading Web depends on SRE, RelEng, Reading Infra ==== Readers Infrastructure ==== * Blocked by: * Blocking: * Updates: ** TemplateStyles will be enabled on German Wikipedia on Wednesday *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome 1, Objective 4]]: Continue improving the ways that users can download articles of interest for later consumption *** Reading Web depends on SRE, RelEng, Reading Infra **[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of client apps by coalescing and moving more logic to the server.]] ***Reading Infra depends on Parsing, Services ===== Maps ===== * Blocked by: * Blocking: * Updates: ** Unblocked on ops today ** Gehel back from vacation this week ** Planning release to enwiki ==== Multimedia ==== * Updates ** File page MediaInfo display prototype: Slow progress ** Search prototype: Up on wmflabs instance, some kinks to work out ** UploadWizard multi-lingual captions: Patch up, working through review *Quarterly goal dependency update: **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 3.1]] Prepare for launch of the first Structured Data on Commons feature (multilingual file captions) ***SDC depends on Multimedia,SRE, WMDE, Search Platform, MediaWiki Platform, Research *** Most dependencies fulfilled, prototypes moving quickly through review ** [[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 2.1]] Integrate structured file captions into search *** SDC depends on Search Platform, Multimedia *** Dependencies pretty much fulfilled, prototype up for perusal on wmflabs instance **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Structured_Data#Segment_4:_Programs|Segment 4, Outcome 2]]: Develop a better understanding of existing needs for Structured Commons- https://phabricator.wikimedia.org/T171252 *** Research depends on Multimedia *** Haven't heard anything, happy to help === Contributors === ==== Community Tech ==== * Blocked by: * Blocking: * Updates: ** Still working on GlobalPreferences ** Discussing the outcome of ACTRIAL with the community ==== Anti-Harassment Tools ==== * Blocked by: * Blocking: * Updates: ** ==== Editing ==== * Blocked by: * Blocking: ** Updates: None *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.1]] Improve and consolidate our unified editing platform so that it's great on all devices ***Editing depends on Parsing. ==== Parsing ==== * Blocked by: * Blocking: * Updates: ** Tidy replaced on a bunch more wikis today (all wikiquotes except frwikiquote, all wikimedia wikis). More being replaced next week (ref. announcement on wikitech-l & ambassadors-l) ** Today, at noon PT, we are going to be discussing with TTO about the <poem> extension in #mediawiki-parsoid. We are trying to figure out whether to merge <poem> into core ( https://phabricator.wikimedia.org/T54061 ) with a different tag, or make other fixes to the poem extension. If you have an interest in this extension, please show up in #mediawiki-parsoid. *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.6]] Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services **[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing]] of client apps by coalescing and moving more logic to the server. ***Reading Infra depends on Parsing, Services **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.1]] Improve and consolidate our unified editing platform so that it's great on all devices ***Editing depends on Parsing ** Program 7, Outcome 2: Objective 1: Revision storage scaling *** Services depends on SRE, Parsing **[[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better data|Outcome 2: Objective 1]]: Revision storage scaling ***Services depends on SRE, Parsing ==== Collaboration ==== * Blocked by: * Blocking: * Updates: ** Still dealing with fallout from the OOUI icon changes, need more follow-ups from Volker (will talk to him) ** Deployed maps i18n code to maps-test, still working out issues before prod deployment ==== Language ==== * Blocked by: ** Requesting review of https://gerrit.wikimedia.org/r/#/c/419402/ from VE/Editing. * Blocking: * Updates: ** ContentTranslation version 2 work continue ** apertium upgrade done except fra-cat pair *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.1]] Improve and consolidate our unified editing platform so that it's great on all devices ***Language depends on Editing === Audiences Design === * Blocked by: * Blocking: * Updates: ** ==== UI Standardization ==== * Blocked by: * Blocking: * Updates: ** OOUI – v0.26.2 released: https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v… *** Two icon deprecations: 'userActive'/'userInactive' & 'find' icon *** 10 more icon and sizing patches as follow-ups to https://phabricator.wikimedia.org/T97631 & https://phabricator.wikimedia.org/T177432 rel in 0.26.0 ** Work on size unification aftermath in different products (Notifications, Flow, RevisionSlider) ** Style Guide technical and contents improvements ongoing with 4 open PRs currently ** Transparency Report: Wrapping up accessiblity measurments == Technology == === Analytics === * Blocked by: * Blocking: * Updates: ** Improved MediaWiki history reconstruction performance, less than half (7 hours now with spark 2) ** Preparing the migration of all Spark jobs to Spark2 ** Lots of small-medium Wikistats2 front-end bug-fixes ** Ongoing: Aggregation of new Audiences EL events VirtualPageViews (Popups) into pageview_hourly format. === Cloud Services === * Blocked by: ** Labvirt1021/22 still having initial install issues, working with DC Ops: https://phabricator.wikimedia.org/T183937 ** Labvirt1019/1020 still have RAID issues, working with DC Ops & HP ticket open: https://phabricator.wikimedia.org/T187373 * Blocking: * Updates: ** Madhu moved Dumps content hosting to new servers managed by Cloud Services team. *** Toolforge, Cloud VPS, Analytics NFS moved Monday 2018-04-02 with no incident and minimal outage *** dumps.wikimedia.org web interface & rsync moving on Wednesday 2018-04-04 *** More details at <https://etherpad.wikimedia.org/p/dumps-migration>. ** Tool catalog project led by James Hare has been branded "Toolhub" and initial use cases published on meta <https://meta.wikimedia.org/wiki/Toolhub ** Chase got a functioning high availability OpenStack Neutron layer 3 network up in labtestn! There is more work to do, but this is a big Neutron project milestone. ** Q4 goals posted on mediawiki.org === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** EventLogging for fundraising landing page views: https://gerrit.wikimedia.org/r/423952 ** Short-term response to banner that loaded 3rd party script: add CSP only on forced banner previews: https://gerrit.wikimedia.org/r/420754 ** Looking at new bitcoin payment processor, as our current one is stopping USD cashouts at the end of the month ** More work on new API for main CC processor ** CiviCRM custom data enhancements === MediaWiki Platform === * Blocked by: * Blocking: * Updates: ** MW 1.31 release planning *** considering additional bundled extensions ( https://phabricator.wikimedia.org/T178349 ) ** Pingback heartbeat patch merged ** MCR: *** API updates for query modules are in Gerrit: https://gerrit.wikimedia.org/r/#/c/413223/ *** Code review ** Actor table: *** Further deployment is waiting on the schema change ( https://phabricator.wikimedia.org/T188299 ) *** 17 extension updates still need review: https://gerrit.wikimedia.org/r/#/q/status:open+topic:bug/T167246-actor-migr… ** Comment table: *** Follow up schema change is still in progress https://gerrit.wikimedia.org/r/#/c/417039/ *** Toolforge view updating patch is merged and pending updates *** The image_comment_temp merging patches are in Gerrit **** All except one are blocked on the schema change ** Improving API test coverage: *** 6 changes merged and 6 changes outstanding ** Code review and bug fixes *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.6]] Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services **** MWPT working on relevant planning for Platform Evolution CDP to prepare for next FY and will coordinate with Parsing as this progresses **Prepare for launch of the first Structured Data on Commons feature (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research **** MWPT meets weekly with SDC and WMDE ** [[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 1.1]] Assist with deploying MultiContent Revisions on Commons [[phab:T174022|T174022]] Implement multi-content revisions, [[phab:T174023|T174023]] Implement MCR storage layer, [[phab:T174045|T174045]] DB schema migration for MCR, [[phab:T174044|T174044]] Deploy MCR storage layer,[[phab:T174043|T174043]] Deploy Multi-Content Revisions * SDC depends on MediaWiki Platform, WMDE ** MWPT meets weekly with SDC and WMDE === Performance === * Blocked by: ** * Blocking: **The train :-( * Updates: ** Continuing to monitor SG datacenter rollout, looking great. Working on a blog post. ** Thumbor patches, adding WebP support ** Safemode support for Visual Editor is out for review ** Coal has been updated to use Kafka ** Waiting on SRE for a rebuild of the mcrouter package (they're aware, just haven't had time yet) ** Tons of ongoing CR === Release Engineering === * Blocked by: * Blocking: * Updates: ** *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome 1, Objective 4]]: Continue improving the ways that users can download articles of interest for later consumption *** Reading Web depends on SRE, RelEng, Reading Infra === Research === * Blocked by: * Blocking: * Updates: *Quarterly goal dependency update: **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the first Structured Data on Commons feature]] (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Structured_Data#Segment_4:_Programs|Segment 4, Outcome 2]]: Develop a better understanding of existing needs for Structured Commons- T171252 ***Research depends on Multimedia === Scoring Platform === * Blocked by: ** git-lfs support, we've tried deploying a git-lfs submodule and it doesn't work. git-lfs is still lacking on the deployment node, but that shouldn't affect the deployment itself. See task https://phabricator.wikimedia.org/T180627 * Blocking: * Updates: === Search Platform === * Blocked by: * Blocking: https://phabricator.wikimedia.org/T190958 needs re-deploy * Updates: ** Wikidata fulltext search is fully enabled, so there should be improvements there ** Slovak stemmer analysis done: https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Slovak_Stemmer_Analy… ** Next step - creating proper analysis plugin: https://phabricator.wikimedia.org/T190815 ** Fixed an issue with Wikidata RDF dumps: https://phabricator.wikimedia.org/T185589 ** Evaluating feature generation by `query_explorer` functionality of LTR plugin: https://phabricator.wikimedia.org/T187148 ** Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108 ** Working on Lexeme completion search: https://phabricator.wikimedia.org/T189745 ** Need help with WDQS GUI build automation: https://phabricator.wikimedia.org/T189811 ** Wikidata selenium test stability problems: https://phabricator.wikimedia.org/T189762 *Quarterly goal dependency update: **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the first Structured Data on Commons feature]] (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 2.1]] Integrate structured file captions into search *** SDC depends on Search Platform, Multimedia === Security === * Blocked by: * Blocking: * Updates: ** *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.6]]: Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services === Services === * Blocked by: search on updating CirrusSearch jobs * Blocking: none * Updates: ** Moved a big bulk of jobs to kafka ** Resetting the restbase-dev environment *Quarterly goal dependency update: **<nowiki>*</nowiki>[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of client apps by coalescing and moving more logic to the server.]] ***Reading Infra/Parsing, Services **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal 3.6]] Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team ***Parsing depends on MediaWiki Platform, Services ** [[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better data|Outcome 2: Objective 1:]] Revision storage scaling *** Services depends on SRE, Parsing === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** None * Updates: ** Holiday week, really slow stuff not much to report ** HHVM CVE-2018-6334 patch rolled out this week ** *Quarterly goal dependency update: **[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome 1, Objective 4]]: Continue improving the ways that users can download articles of interest for later consumption *** Reading Web depends on SRE, RelEng, Reading Infra **[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the first Structured Data on Commons feature]] (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research ** [[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better data|Outcome 2: Objective 1]]: Revision storage scaling *** Services depends on SRE, Parsing == Wikidata == * Blocked by: * Blocking: * Updates: ** *Quarterly goal dependency update: **Prepare for launch of the first Structured Data on Commons feature (multilingual file captions) ***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki Platform, Research ** Objective 1.1 Assist with deploying MultiContent Revisions on Commons T174022 Implement multi-content revisions, T174023 Implement MCR storage layer, T174045 DB schema migration for MCR, T174044 Deploy MCR storage layer,T174043 Deploy Multi-Content Revisions ***SDC depends on MediaWiki Platform, WMDE == German Technical Wishlist == * Blocked by: * Blocking: * Updates: ** Following up on OOUI changes influencing AdvancedSearch ** Going to focus on FileImporter (a.k.a. MoveToCommons) again == SoS Meeting Bookkeeping == * Updates: **