[Wikitech-l] 2018-03-21 Scrum of Scrums meeting notes

https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-03-21#Multimedia = 2018-03-21 = == Callouts == * Fundraising campaigns https://meta.wikimedia.org/wiki/CentralNotice/Calendar - next up: Netherlands 2018-04-03 through 2018-05-01 * WMDE needs a Wikidiff2 review: https://gerrit.wikimedia.org/r/404293 == Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: ==== Android native app ==== * Blocked by: * Blocking: * Updates: ** Increased rollout of Reading List sync to 50% of Beta audience. ** Blocked on a few minor client-side issues, but should continue full rollout by end of week. ==== Readers Web ==== * Blocked by: * Blocking: * Updates: *Interrupt work for Hindi Wikipedia campaign ( https://phabricator.wikimedia.org/T190101) *Discussion around DNT (https://phabricator.wikimedia.org/T187277) *Improvements and some refactoring to Popups extension *Quarterly goal dependency update: ** Increase learning by lowering the cost of exploration (Page previews) *** Turning on virtual page view logging for all wikis ( https://phabricator.wikimedia.org/T189906) *** Pushing for full-deploy in next 3 weeks. ( https://phabricator.wikimedia.org/T154635) *** This goal was impacted by interrupt work. ** Continue improving the ways that users can download articles of interest for later consumption *** Proton work was delayed until next quarter due to external dependencies in standing up a production endpoint ==== Readers Infrastructure ==== * Blocked by: * Blocking: * Updates: ** New /page/media endpoint is exposed via RESTBase, so apps can test it and provide feedback. ** /page/metadata and /page/references to follow this week. ===== Maps ===== * Blocked by: * Blocking: * Updates: ==== Multimedia ==== * Blocked by: N/A * Blocking: N/A * Updates ** Search: Patch for MediaInfo caption indexing nearly complete, WMDE and Cormac working diligently to get things merged, Search supporting (thanks, both) ** File page prototyping: Progressing, slowly. Mark to meet with Thiemo from WMDE to clear some things up. Quarterly goal dependency update ** Prepare backend infrastructure for structured data search *** SDC: Search PF/Multimedia/Wikidata ** Wikimedia Technology/Goals/2017-18 Q3#Segment 2: Search integration and exposure *** SDC: Research/Multimedia ** Prepare backend infrastructure for structured data search *** SDC: Search PF/Multimedia/Wikidata === Contributors === ==== Community Tech ==== * Blocked by: * Blocking: * Updates: * GlobalPreferences going live today * Work on TemplateWizard progressing well ==== Anti-Harassment Tools ==== * Blocked by: None * Blocking: None * Updates: * Work continues on Blocking tools * Bug fixes to Interaction Timeline * Research on Harassment Reporting Tool ==== Editing ==== * Blocked by: * Blocking: * Updates: ==== Parsing ==== * Blocked by: * Blocking: * Updates: ** Tidy has been replaced on all wikiversities today. About 300 more to go before the end June deadline. ** Whitespace trimming in headings, list items, table cells, headings and captions (wikitext versions only) has been pushed to production. Found a bug for which there is a fix in gerrit ** Arlo has been working on the heredocs syntax RFC ( https://phabricator.wikimedia.org/T114432 ) and has a WIP patch @ https://gerrit.wikimedia.org/r/#/c/418198/ if someone wants to test and play with it on your local dev wikis. *Quarterly goal dependency update: ** Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team *** Parsing:Mediawiki PF/Services *** No new updates or blockers. ==== Collaboration ==== * Blocked by: * Blocking: * Updates: ** Flow uninstalled from wikis where it was unused; can still be requested by communities, will be re-enabled with local consensus ** Almost done setting up maps test servers in labs ==== Language ==== * Blocked by: None * Blocking:None * Updates: ** CX2 work continue; ** Migration to recommend service from Production is planned; WIP. === Audiences Design === * Blocked by: * Blocking: * Updates: ==== UI Standardization ==== * Blocked by: * Blocking: * Updates: ** OOUI – v0.26.0 breaking change release https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v… *** 7 breaking changes, mostly already deprecated icons got removed *** WikimediaUI theme's new icon set in place https://phabricator.wikimedia.org/T177432 ! *** Unified VE in Vector/WikimediaUI interface base font-size to usage of OOUI in core/extensions elsewhere https://phabricator.wikimedia.org/T97631 ! ** Style Guide *** Continued work on v1 goals https://phabricator.wikimedia.org/tag/wikimediaui_style_guide/ == Technology == === Analytics === * Blocked by: * Blocking: * Updates: ** 504s in Pageview API RESTBase layer, problem with RESTBase running out of TCP ports, applying some fixes now to prevent the problem ** notebook1003 is replacing notebook1001, so we need to migrate folks using Jupyter, watch for communication on the lists ** responsive Wikistats work is ongoing ** porting geowiki to Hadoop is done, building a dashboard on top of it now ** more analytics processes migrated to Kafka Jumbo cluster (webrequest and EventLogging are fully migrated now) ** improvements on automatic refining of EventLogging topics to Hadoop * Quarterly goal dependency update: **Improve, adjust, or create features geared at the needs identified in New Editors research project. *** New Editors Experience:Analytics === Cloud Services === * Blocked by: * Blocking: * Updates: === Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Upgrading CiviCRM install to 5.0 ** More work on main CC processor new API, including redesigning recurring contribution charges to fit better with Civi ** Making sure all the unintentional recurring donor refunds happened as they were supposed to ** More work on using EventLogging to get banner impression and fundraiser landing page stats ** Looking into applying CSP to banner preview pages, but need to decide how to make it fit with pending core patch: https://gerrit.wikimedia.org/r/253969 === MediaWiki Platform === * Blocked by: * Blocking: * Updates: *Quarterly goal dependency update: ** Support work towards unifying MediaWiki's parser implementations, in liaison with Technology's MediaWiki team *** Parsing:Mediawiki PF/Services * Reduce product and technical debt to modernise our tools and technologies, and to make future changes more effective and efficient *** Parsing/Mediawiki PF **1.1 It is possible to store structured data within wiki pages, in particular on media file pages on Commons. We will enable the MediaWiki storage layer to correctly store and process structured data elements within wiki pages. *** SDC: Mediawiki PF/Wikidata === Performance === * Blocked by: None * Blocking: None * Updates: ** Collecting a bunch of oversample data from countries in Asia/Oceania, prep for Singapore to go live. ** Thumbor 100% done ** Expecting to spend a bit of time helping with reviews or ResourceLoader changes related to Hindi wiki front page stuff ** auto_prepend_file being used to load profiler ** Re-did coal changes to use python-kafka, confluent_kafka doesn't work ** Deploying RUMSpeedIndex this week, essentially emulates Speed Index calculation using the browser resource timing API === Release Engineering === * Blocking ** * Blocked ** * Updates ** Minor Gerrit upgrade planned for this week (2.14.6 -> 2.14.7) ** Incident analysis started last week of the last year’s worth of incidents reports ** Scap 3.7.7 should be rolled out to production this week *Quarterly goal dependency update: ** Continue improving the ways that users can download articles of interest for later consumption *** Reading Web: Tech Ops/RelEng (work is currently blocked on https://phabricator.wikimedia.org/T187821 which is part of a larger epic https://phabricator.wikimedia.org/T181084) ** Talked about in team meeting Monday ** is there a task? === Research === * Blocked by: * Blocking: * Updates: *Quarterly goal dependency update: ** Wikimedia Technology/Goals/2017-18 Q3#Segment 2: Search integration and exposure *** SDC: Research/Multimedia === Scoring Platform === * Blocked by: ** Releng is helping us with a git-lfs pilot deployment. ** Waiting for Scap 3.7.7 for a fast-rollback process. * Blocking: * Updates: ** The JADE extension is in beta. *** https://www.mediawiki.org/wiki/Extension:JADE *** This is for auditing ORES. It adds "Jade" and "Jade_talk" namespaces to MediaWiki. We're using JSON content with schema validation. Prateek is contributing some volunteer design time. === Search Platform === * Blocked by: * Blocking: * Updates: *Quarterly goal dependency update: ** Prepare backend infrastructure for structured data search *** SDC: Search PF/Multimedia/Wikidata ** Update: *** Trey published new blog about supporting languages with multiple writing systems: https://blog.wikimedia.org/2018/03/12/supporting-languages-multiple-writing… *** AB test for query explorer feature for LTR finished & analyzed: https://phabricator.wikimedia.org/T189843 Results are looking pretty good, seems to be improving things. *** Wikidata search patches merged. The new search not enabled yet, but improved search result format is. *** Fixed performance issue with LTR cache: https://phabricator.wikimedia.org/T188015 *** TermLookup implementation for ElasticSearch merged: https://phabricator.wikimedia.org/T143706 *** Fixed deepcategory keyword to not return results if keyword has failed, e.g. because there are too many categories: https://phabricator.wikimedia.org/T189331 *** Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108 *** Working on Slovak analysis: https://phabricator.wikimedia.org/T178929 *** Started implementation work on Lexeme search: https://www.wikidata.org/wiki/User:Smalyshev_(WMF)/Lexeme_search === Security === * Blocked by: * Blocking: * Updates: === Services === * Blocked by: none * Blocking: none * Updates: ** New REST endpoint to get lint errors for page/revision *** https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_page_lint_title_re… ** New REST endpoint to get page media *** https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_page_media_title_r… ** refreshLinks topic is now partitioned in kafka according to MySQL shard, should decrease connection spikiness === Site Reliability Engineering === * Blocked by: ** None * Blocking: ** None * Updates: ** Firewalling of misc database hosts successful. Some delays in OTRS email delivery experienced ** Consistent logical backups in codfw goal completed successfully ** mathoid deployed in kubernetes. Production traffic is partly served (~50%) from the new kubernetes cluster in both eqiad+codfw ** eqsin turn-up planning procedure meetings starting. ** EtcdConfig in production goal almost completed ** Some varnish issues ongoing https://phabricator.wikimedia.org/T189892 *Quarterly goal dependency update: ** Continue improving the ways that users can download articles of interest for later consumption *** Reading Web: Tech Ops/RelEng ** Update: No updates, was postponed for early next quarter ** Audiences DesignStandardise our user interfaces to match user expectation of quality from our products *** Audiences Design: Ops ** Update: Done AFAIK. == Wikidata == * Looking into current Lua usage to see where we can improve the Lua functions we provide: https://phabricator.wikimedia.org/T189506 * Optimizing a heavily used database table (wb_terms): https://phabricator.wikimedia.org/T188279 * Polishing a lot of things for lexicographical data first deployment *Quarterly goal dependency update: **1.1 It is possible to store structured data within wiki pages, in particular on media file pages on Commons. We will enable the MediaWiki storage layer to correctly store and process structured data elements within wiki pages. *** SDC: Mediawiki PF/Wikidata ** Prepare backend infrastructure for structured data search *** SDC: Search PF/Multimedia/Wikidata == German Technical Wishlist == * Blocked on external review for the most recent Wikidiff2 optimizations: https://gerrit.wikimedia.org/r/404293 * Working towards an MVP of the FileImporter extension: https://phabricator.wikimedia.org/tag/move-files-to-commons/ * Finishing touches on AdvancedSearch: https://phabricator.wikimedia.org/tag/advanced-search/ == SoS Meeting Bookkeeping == * Updates: