[Wikitech-l] 2017-09-20 Scrum of Scrums meeting notes

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-09-20 *= 2017-09-20= * Grace out of office, please self-organize contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering == callouts == * Ops => Collaboration. Flow dumps speed issue still https://phabricator.wikimedia.org/T172025 and (T164262) == Audiences == === Readers === ==== Reading Web ==== * Actively working on OCG replacement * Refactoring & adding electron support to Collection extension * Working on Marvin * Got a green light to enable Popups on en wiki ==== Multimedia ==== * Pushing 3D to Test/Test2 sometime next week barring any further blockers * Would still like feedback from Performance on https://phabricator.wikimedia.org/T166699 but since we asked and didn't get any last week, I no longer consider it a blocker ==== Discovery ==== * Blocked by: none * Blocking: none * Updates: ** (Maps) Reimaged test-servers, updating to Node 6.11 ** (Maps) enwiki looking at maps ** (Front-end) Mirgating CirrusSearch Selenium tests from Ruby to Node ** (Front-end) Running AB test on Special:Search ==== iOS native app ==== * Blocked by: none * Blocking: none * Updates: ** Still releasing 5.6.1 with minor bug fixes ( https://phabricator.wikimedia.org/project/view/2898/ ) ** Continuing work on 5.7.0 ( https://phabricator.wikimedia.org/project/view/2899/ ) - Onboarding updates, Improved analytics ==== Reading Infrastructure ==== * Blocked by: Security (ReadingLists review https://phabricator.wikimedia.org/T174126 ) * Blocking: * Updates: ** fixed some bugs with Electron multipage rendering, testing again this week ** continuing work on reading lists ** MCS/PCS: Reverted to old sectioning code ** PCS: Comparison tables of old and new implementation of text extracts for various languages === Community Tech === * Blocked by: none * Blocking: none * Updates: ** ACTRIAL started ** Getting database population script for range contributions fixed ** Working on GlobalPreferences == Technology == === Analytics === * Blocked by: * Blocking: * Updates: Working on Wikistats2 back-end: loading easy-to-query data into Druid, implementing AQS endpoints, vetting metrics data - adding proxy to Druid cluster for authentication Working on Wikistats2 front-end: UI improvements, bug fixes Ongoing EL purging: improvement to script, that now works fine, not sure if the purging will finish before end of quarter === Fundraising Tech === * Blocked by: * Blocking: * Updates: Mostly freaking out about our database replication lag Can once again A/B test payments form variations with query string parameters More work on new API integration for main CC processor Fixing CN bug where campaigns that haven't started yet can still pre-empt existing campaigns: https://phabricator.wikimedia.org/T175358 === Release Engineering === * Blocked by: * Blocking: * Updates: * it's happening: All trebuchet-deployed services have been ported to scap. https://phabricator.wikimedia.org/T129290 === Search Platform === * Blocked by: none * Blocking: none * Updates: * Machine-learning driven ranking is deployed as ranking algorithm on enwiki: https://phabricator.wikimedia.org/T175772 * Running A/B test for machine-learning driven ranking on 18 other wikis: https://phabricator.wikimedia.org/T175771 * Running A/B test on displaying other language links under search result: https://phabricator.wikimedia.org/T175647 * Analyzed language fallback usage in Mediawiki search: https://phabricator.wikimedia.org/T147959 ** Conclusion: a lot of misuse, needs work to clear it up * Published blog post by Trey explaining human-graded relevance test: https://blog.wikimedia.org/2017/09/19/search-relevance-survey/ * Categories are now exported into RDF: https://lists.wikimedia.org/pipermail/wikitech-l/2017-September/088799.html ** Weekly for now, daily updates coming soon * New logstash servers set up: https://phabricator.wikimedia.org/T175045 * New stats dashboard: How long Wikipedia searchers stay on the search result pages: https://discovery.wmflabs.org/metrics/#spr_surv * Results of A/B test swapping 2nd and 3rd search result analyzed: https://commons.wikimedia.org/wiki/File:Swap2and3_Search_Test_Analysis.pdf === Security === * Blocked by: * Blocking: * Updates: ** Reviews: *** ReadingLists *** vue.js (sorry for the delay) *** wikiba.se === Services === * Blocked by: none * Blocking: none * Updates: ** Cassandra 3 and new storage model rollout to production is happening right now *** Beginning with mobile tables at first, but still in test mode, serving from old storage ** EventBus based job queue double-processes the first job successfully === Technical Operations === * Apologies, Alex/Fillippo won't be able to make it today. Riccardo will attend * Blocked by: ** Collab Team on Flow dumps speed issue still T172025 and T164262 * Blocking: ** None * Updates: ** We had some spikes of 503s at the cache layer, under control now, see T175803, T174932 and T145661 if interested ** luasandbox 2.0.14 rollout completed ** New appservers will be put in production in the next days ** Reminder: Salt (the foundation upon Trebuchet was built) will be removed before the end of next week, replaced by Cumin. All projects that were deployed with Trebuchet, have already been migrated to scap3 or Debian packages. === Contributors === ==== Global Collaboration ==== ===== Language ===== * Blocked: none * Blocking: none * Updates: * cxserver deployment is unblocked; Registry refactoring targetted in next deployment. Request for review: https://gerrit.wikimedia.org/r/#/c/377713/ * CX2 is in progress. Lots of work done here. ===== Collaboration ===== * Updates ** RCFilters - Rolled out new Recent Changes Filters to be on by default for all users on he.wiki, ca.wiki, fr.wiki, but you can opt out. Watchlist Filters are now part of the Beta feature. Various new features and bug fixes. *** {{git|85df9a3a}} - Move New Filters opt-out preference to own section ({{phabricator|T175765}}) *** {{git|ceb02fbf}} - RCFilters: make live update part of the beta feature ({{phabricator|T175766}}) *** {{git|004b72e3}} - RCFilters: Add an initialization hook *** {{git|f5d587d3}} - Hide RC/WL related preferences as appropriate ({{phabricator|T175611}}) ** {{git|210946c3}} - (GuidedTour) Allow directly launching tour from server without ?tour= or cookies ({{phabricator|T167262}}) ==== UI Standardization ==== * Blocked: none / none * Updates: ** Working on extending WikimediaUI Base to include vars already needed in Marvin, especially `font` specifics ** OOUI v0.23.1 released *** Code hygiene fixes and icon deprecation * Ongoing: ** OOUI: *** preparation work on responsive toolbars design part *** icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide's guidelines ** Continuation on WikimediaUI Style Guide, continuing updating imagery on color section https://wikimedia.github.io/WikimediaUI-Style-Guide/visual-style_colors.html ** Aligning arbitrary, historically grown colors to WikimediaUI color palette https://phabricator.wikimedia.org/T148708 – now for the remaining blues ** Make TransparencyReport fully accessible – https://gerrit.wikimedia.org/r/#/c/376875/ merged, now for the build update ==== Parsing ==== * Blocked by: https://gerrit.wikimedia.org/r/378774 (html5 ids in Cite extension) * Blocking: * Updates: ** C.Scott is porting over the html5 section id support into Parsoid (Parsoid had html5 section ids till last year - switched to html4 ids last year to match core -- now updating to html5 with legacy support again) *** Requires some fixes to Cite and LanguageConverter :( ** Kunal (legoktm) is running a script to lint all pages on all wikis so that linter errors are initialized on all wikis to get baseline information in place ** Last week, we switched MediaWiki and TestWiki to use Remex instead of Tidy. *** Individual wikis can file subtasks of https://phabricator.wikimedia.org/T175706 to switch their wikis over. ** Heads up for Parsoid clients (VE, CX, Flow, MCS) *** <section> wrapping code is now out of WIP and in review ( https://gerrit.wikimedia.org/r/#/c/364933/) -- please test your code to make sure you can handle <section> wrappers. If necessary, you can pre-process the DOM to strip out <section> tags. Parsoid's output is guaranteed to preserve template wrapping semantics with / without <section> tags. Parsoid can also accept DOMs with / without <section> tags (for serializing back to wikitext). *** We plan to switch Parsoid DOM output to use <figure-inline> tags instead of <span> for inline images ( https://gerrit.wikimedia.org/r/#/c/370227/ ) -- please test your code to make sure you can handle the new markup. VE can handle this.