https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-11-08

= 2017-11-08 = == Callouts == * No MW Train the week of the 20th due to Thanksgiving, SWATs will be open on Mon and Tuesday (Wednesday is “Friday” that week). * RelEng: Please port your browser tests to the nodejs framework. 7 repositories have not started yet (still in ruby, which is no longer maintained). [https://phabricator.wikimedia.org/T139740 task] * Tidy replacement: if you are an editor on some wiki, please get the word out to your wikis to fix the high-priority linter issues. We want to get Tidy removed from the cluster sooner than later.A bunch of wikis (including mediawiki) no longer have Tidy, and a few 100 small wikis can get Tidy replaced based on stats about linter issues ( https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy/Linter/Stats/2017-11-0... ). But would be good to get some largish wikis moved over so we can identify any lingering problems sooner than later.

== Audiences == === Readers === ==== iOS native app ==== * Blocked by: * Blocking: * Updates: ** 5.7.1 in the app store today (minor bug fixes) https://phabricator.wikimedia.org/project/view/3047/ ** Continuing work on 5.8 - https://phabricator.wikimedia.org/project/view/2913/ (faster article loads, synced reading lists)

==== Android native app ==== * Blocked by: * Blocking: * Updates: ** Released version 2.6.206 to production. ** Continuing to test / integrate with Reading List service.

==== Reading Web ==== * Blocked by: ** Ops&Release engineering - How to use Chromium in Puppeteer [ https://phabricator.wikimedia.org/T178570%5D+ https://phabricator.wikimedia.org/T178189 *** Currently investigating on path forward in https://phabricator.wikimedia.org/T180037. * Blocking: * Updates: ** Working on the Chromium-based PDF renderer service. *** Starting performance testing within two weeks. ** Researching behaviour around the "Download to PDF" button [ https://phabricator.wikimedia.org/T177969, https://phabricator.wikimedia.org/T177969#3742056] ** Working product and technical documentation for Marvin

==== Reading Infrastructure ==== * Blocked by: * Blocking: * Updates: ** Continuing work on MCS summary, media (gallery) endpoints. ** Continuing with Reading Lists

==== Multimedia ==== * Blocked by: * Blocking: * Updates:

==== Discovery ==== * Blocked by: * Blocking: * Updates: ** continuing work on portal automation. going well.

===== Maps ===== * Blocked by: N/A * Blocking: N/A * Updates: None

=== Contributors === ==== Editing ==== * Blocked by: * Blocking: * Updates:

==== Parsing ==== * section + figure-inline + html5-ids changes aren't going out this week after all -- reviews not yet finalized + we need thorough testing in beta cluster. We are still hoping to get it out next week. * Tidy replacement: if you are an editor on some wiki, please get the word out to your wikis to fix the high-priority linter issues. We want to get Tidy removed from the cluster sooner than later. A bunch of wikis (including mediawiki) no longer have Tidy, and a few 100 small wikis can get Tidy replaced based on stats about linter issues

==== Global Collaboration ==== * Blocked by: nobody * Blocking: ops maybe? There's the Flow dumps issue but not sure how much of a blocker that is * Updates: ** Going to roll out RCFilters to remaining (FlaggedRevs) wikis on Nov 15 ** Early work on using wikitext instead of HTML for storing Flow content, need something like pcache except 1) it's Parsoid HTML not PHP HTML 2) Flow comments aren't pages

==== UI Standardization ==== ** OOUI v0.24.1 & v0.24.2 released (missed last week's Scrum of Scrums), https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v0...; among changes: *** UI design **** WikimediaUI theme with 3 changes: SVG markup (see below), MenuSectionOptionWidget clearer on long lists and ComboBoxInputWidget providing stronger user feedback on open * Ongoing: ** OOUI & relying products: *** icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide's guidelines https://phabricator.wikimedia.org/T177432 ** Unify SVG markup across Foundation products https://phabricator.wikimedia.org/T178867

=== Community Tech === * Not blocking * Not blocked * Our annual survey is underway: https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey * Fixing issues uncovered by Unicode sections rollout to Russian projects

== Technology == === Analytics === * New kafka Jumbo cluster is up, for now mirror-ing topics from analytics cluster. * New DB hosts for eventlogging back in place, purging data out of compliance at a faster rate due to better hardware. * We have spark2 in cluster! * Still troubleshooting metric difference wikistats 1.0 on wikistats 2.0. Take a look edit metrics available now (some of them have a 5% difference, so not final yet): https://stats.wikimedia.org/v2/#/am.wikipedia.org * Work on refining eventlogging data in hadoop continues so hadoop can be a easy to query backend for eventlogging data

=== Cloud Services === * Blocked by: * Blocking: * Updates:

=== Fundraising Tech === * Blocked by: * Blocking: * Updates: ** Deploying new English thank you letter ** Payments form validation improvements ( https://phabricator.wikimedia.org/T176501) ** Better API error handling for backup card processor ( https://phabricator.wikimedia.org/T177893) ** More stats reporting to prometheus ** CiviCRM UI improvements (https://phabricator.wikimedia.org/T142549, https://phabricator.wikimedia.org/T178003, https://phabricator.wikimedia.org/T177873) ** CentralNotice: allow delaying impression reporting ( https://phabricator.wikimedia.org/T176334)

=== MediaWiki Platform === * Blocked by: * Blocking: * Updates:

=== Performance === * Blocked by: * Blocking: * Updates:

=== Release Engineering === * Blocking ** ??? * Blocked ** Please port your browser tests to the nodejs framework. 7 repositories have not started yet (still in ruby, which is no longer maintained). *** See: https://phabricator.wikimedia.org/T139740 *** Notably: Global Collaboration Team, Fundraising Tech, Wikibase, and Mulitmedia * Updates ** No MW Train the week of the 20th due to Thanksgiving, SWATs will be open on Mon and Tuesday (Wednesday is “Friday” that week). ** [TechDebt program] First pass of the service levels for component ownership was shared with the Code Health Group last week; feedback on-going. ** [TechDebt program] The next blog post should be posted Real Soon Now™ (done on our side). ** [SSD Program] Working on getting the mathoid tests running on submit. ** [SSD Program] A new release of Blubber is on the horizon. *** [SSD Program] Aside: what it is: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Deployment... *** [SSD Program] Bubber: https://phabricator.wikimedia.org/source/blubber/ ** [scap tech debt] Working to support both pre git-2.11 and post git 2.11 hosts (namely Trusty and Jessie (with backports)/Stretch, respectively) to let us use newer functionality (notably `--jobs`). ** [CI] Most tox jobs are moved to Docker containers ** [CI] Investigating why many docker containers are left behind after a SIGTERM

=== Research === * Blocked by: * Blocking: * Updates:

=== Scoring Platform === * Blocked by: * Blocking: * Updates: **frwiki -- ongoing issue. https://phabricator.wikimedia.org/T179711 (should be fixed by SOS) **Draft quality data is coming to English Wikipedia (ping Commtech and Global Collab) ***You can use this to help patrollers sort new pages by (Spam, Attach, Vandalism/Hoax, or OK). OK == not immediately concerning but not necessarily good. ***[Reponse from Roan: we don't plan to use draftquality directly at this point] **Wikilabels getting some improvements. ***Links to on-wiki documentation for each campaign ("info" link) ***Statistics about labelers will have usernames **Amir is working on Icelandic language support.

=== Search Platform === * Blocked by: * Blocking: * Updates: ** Added hiragana<->katakana translations for English wiki (not for Japanese one) https://phabricator.wikimedia.org/T176197 ** Implemented demoting disambig pages for Wikidata prefix search https://phabricator.wikimedia.org/T148411 ** Improved reusability of Searcher code https://phabricator.wikimedia.org/T178906 ** Improving completion suggester https://phabricator.wikimedia.org/T178474 ** Working on porting Selenium tests from Ruby to JS ** Working on upgrade to Elastic 5.5 ** Working on indexing Wikidata descriptions and adding them to fulltext search

=== Security === * Blocked by: None * Blocking: Everyone waiting for security reviews * Updates: ** Reviews: *** BeautifulSoup (done, comments will be added shortly) *** vue.js (done, same) *** mediawiki-services-chromium-render ** Preparing for MW security release

=== Services === * Blocked by: none * Blocking: none * Updates: ** Parsoid storage switched to new storage model for all but wikipedias ** Definitions storage switched to new model ** Planning on switching RecordLintJob to kafka-based JobQueue

=== Technical Operations === * Blocked by: ** Flow isAllowed gets actual revision text before it is needed https://phabricator.wikimedia.org/T172025 * Blocking: * Updates: ** labsdb1001 storage crashed; Manuel got the dbs back up, but we are leaving in read-only - https://phabricator.wikimedia.org/T179464 ** Turning up s8 databases for wikidata https://phabricator.wikimedia.org/T177208 ** wikidiff2 1.5.1 fully rolled out, enabled for group0 wikis, additional will be enabled by Adam Shoreland over the next weeks

== Wikidata == * Blocked by: * Blocking: * Updates: ** getting rid of the Wikidata build (to deploy Wikibase etc. as normal extensions): https://phabricator.wikimedia.org/T173818 *** changed how JS libraries are included (as submodules, instead of via composer) * work on Lexeme (persistence) * work on multi-content revisions * Coordinates in RDF exports are going to use xsd:double: https://phabricator.wikimedia.org/T179228

== German Technical Wishlist == * Blocked by: * Blocking: * Updates: