[Wikitech-l] 2017-10-11 Scrum of Scrums meeting notes

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-10-11 = 2017-10-11= contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering == Callouts == * Are there any special steps to wire a MediaWiki LoggerFactory into logstash-beta? AWight is unable to find some logs, which are present on fluorine-beta. * CommTech would appreciate help reaching consensus regarding the way preferences are handlled: https://lists.wikimedia.org/pipermail/wikitech-l/2017-October/088980.html * DBAs have disabled Wikidata RC syndication on Commons and ruwiki, and are working on purging existing WD RC rows on those wikis. This is because of performance and table size reasons. https://phabricator.wikimedia.org/T171027 * MediaWiki 1.31.0-wmf.3 didn't go out yesterday, we are getting caught up today. https://phabricator.wikimedia.org/T174359 == Audiences == === Readers === ==== iOS native app ==== * Blocked by: none * Blocking: none * Updates: ** Continuing work on 5.7.0 ( https://phabricator.wikimedia.org/project/view/2899/ ) - Visual updates to onboarding, iOS 11 support, iPhone X support ==== Android native app ==== * Blocked by: none * Blocking: none * Updates: ** On schedule to release beta of E-release this week ( https://phabricator.wikimedia.org/project/view/2839/ ) ** Will start (local) testing of Reading List service. === Discovery === * Continuing work on portal automation, resolving dependencies that fail to build on some systems and cleaning up old code. ==== Maps ==== * Blocked by: none * Blocking: none * Updates: * Got some help debugging nodejs issues * Community is activating <mapframe> on Spanish Wikipedia ==== Reading Infrastructure ==== * ReadingList RESTBase patch under review; hoping to deploy both RESTBase and MediaWiki part to beta soon * MCS: accounting for Parsoid changes: <figure-inline> instead of <span> for inline figures (https://phabricator.wikimedia.org/T177301) ==== Web ==== * Turning off OCG [cc. ops] * Working on replacement for OCG === Contributors === ==== Parsing ==== * cscott's ping to wikidata team: https://gerrit.wikimedia.org/r/#/c/379669/ -- Parsing is not blocked on this per se, but this is something that cscott has been contributing to as part of a longer term project. Using SoS to ping wikidata on a stalled review for him. * Updates ** Parsing team offsite next week -- will be absent at next week's SoS. ** Parsoid code to use the updated Templatedata format now merged (suggested by Thiemo at Wikimania 16). It is in the beta cluster and needs testing before being deployed. ** We are likely to merge both the <section> wrapper code as well as the <figure-inline> code at the same time so that we can bump the HTML version number once. *** Looks like cxserver/CX has patches to handle section output. Is MCS ready to handle Parsoid's section output? C.Scott said he is going to add a patch for VE to handle this, but, if VE folks want to tackle that on your own, please do so. ==== Global Collaboration ==== ===== Language ===== * Compact Language Links out of Beta deployment being scheduled next week for German Wikipedia. * Work continue with VE-CX integration and adpatation. * cxserver has been updated to latest master, debugging issues with Beta cxserver. ===== Collaboration ===== * New filters on RC now the default experience on all wikis except some FlaggedRevs wikis * Looking into Watchlist/ORES slow query issues, but DBAs are also working on that ==== UI Standardization ==== * Blocked: icon feedback appreciated, see below / none * Updates: ** Improved whitespace usage in OOUI HTMLForms https://phabricator.wikimedia.org/T177668 ** OOUI v0.23.4 release in preparation *** Generalized icon/indicator padding/positioning https://phabricator.wikimedia.org/T161177 & streamlined icon/indicator visibility and layout https://phabricator.wikimedia.org/T160593 * Ongoing: ** OOUI: *** icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide's guidelines https://phabricator.wikimedia.org/T177432 ** Aligning arbitrary, historically grown colors across products to WikimediaUI color palette https://phabricator.wikimedia.org/T148708 === Community Tech === * Blocked: not blocked, but see our callout * Blocking: none * Report: ** Working on GlobalPreferences ** Parser caches are almost populated with Unicode IDs == Technology == === Analytics === * No blocker (AFAIK) * Not blocked *Updates: ** Wikistats 2.0 UI is alpha-level with reading metrics, still missing edit metrics because of missing backend ** Setting up a production data cluster for druid with public data to be the edit-data public APIs backend. We have split our Druid cluster in two, with 3 hosts each. ** Getting Druid Grafana dashboards in place. ** Working on processing eventlogging data in hadoop instead of MySQL ** Still purging data in MySQL eventlogging hosts, slow going ** Work to move metrics to prometheus continues. === Scoring Platform === * Blocked by: none * Blocking: ** Still working on file handle exhaustion, which prevents us from using the new ORES cluster. * Updates: ** Deploying "revscoring 2.0", which could cause some instability in Extension:ORES and in the ORES service. === Search Platform === * Blocked by: none * Blocking: none * Updates: * Messaging fallbacks usage for search disabled ( https://phabricator.wikimedia.org/T147959) - actual switch happens when we reindex * MLR assisted results test analysis published: https://wikimedia-research.github.io/Discovery-Search-Test-InterleavedLTR/, results generally positive * Patch for indexing wikidata statements merged, will index P31 and P279 initially after deployment * Working on porting Selenium tests from Ruby to JS * Working on upgrade to Elastic 5.5 * Working on indexing Wikidata descriptions and adding them to fulltext search === Services === * Blocked: none * Blocking: none ** Updates: *** Deploying cassandra 3 in beta cluster *** Starting testing Parsoid with Cassandra 3 === Technical Operations === * '''Blocked''': ** none * '''Blocking''': ** none * Updates ** Singapore caching DC ongoing ** Quite a bit of puppet refactoring for varnishes ** ganglia slowly being killed (nobody will probably even notice) === Fundraising Tech === * Still trying to figure out db lag, have some leads: https://phabricator.wikimedia.org/T173472 * Still trying to figure our missing data from kafkatee, getting some help: https://phabricator.wikimedia.org/T177331 * More CiviCRM contact de-duplication improvements * Fixing Civi import bugs: https://phabricator.wikimedia.org/T177806, https://phabricator.wikimedia.org/T177803 * Cleaned up contact groups in Civi to improve performance * Diagnosing apparent missing banner impressions (big chunks seem to be bot-related): https://phabricator.wikimedia.org/T176802 == Wikidata == * (Hopefully) unblocked QUnit 2 update: https://phabricator.wikimedia.org/T177764 * We had to turn of RecentChanges integration on big wikis for performance reasons, looking into workarounds: https://phabricator.wikimedia.org/T177707 * Fixed a regression we believe happened because of a jQuery update: https://phabricator.wikimedia.org/T177485 * We turn JS-only components into pure NPM libraries: https://phabricator.wikimedia.org/T177087 * Constraint checks also on qualifiers and references: https://phabricator.wikimedia.org/T176863 * Added new Lua functions to the mw.wikibase library to avoid pulling entire entities: https://phabricator.wikimedia.org/T142903, https://phabricator.wikimedia.org/T173262 * Full URIs for external identifiers will soon be in the RDF export: https://phabricator.wikimedia.org/T121274 * Optimizing runtime of the wikidata.org dump generation: https://phabricator.wikimedia.org/T177486 * wb_entity_per_page table is not updated any more: https://phabricator.wikimedia.org/T140890 * wb_terms table on labs does have a term_full_entity_id column now: https://phabricator.wikimedia.org/T167114 * As usual, we continue working on the Lexeme extension for Wiktionary: https://phabricator.wikimedia.org/T168263 == RelEng == * Blocked: none * Blocking: none? * Updates: ** wmf.3 cut this morning, back on track today group0 and group1 == Security == * Reviews: ** Ex:FileImporter (T160982) ** Ex:WikipediaExtracts (T149424) ** pdfrw