https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-04-11
= 2018-04-11 =
*Q4 FY1718*
== Callouts ==
* Fundraising campaigns
https://meta.wikimedia.org/wiki/CentralNotice/Calendar - Netherlands
2018-04-03 through 2018-05-01
* OpenStack upgrades on Friday 2018-04-13 -- Horizon and CI
(Zuul/Jenkins/Nodepool) will be disabled for a while beginning around 14:00
UTC
* Still need help from RelEng with WDQS GUI build automation:
https://phabricator.wikimedia.org/T189811
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
**
==== Android native app ====
* Blocked by:
* Blocking:
* Updates:
** Reading list sync rolled out to 10% of production (~800K users)
** On track to roll out to full production next week.
==== Readers Web ====
* Blocked by:
[performance]
https://phabricator.wikimedia.org/T42812 - nice
collaboration going on here, to improve mw-collapsible behaviour on desktop
- [releng]
https://phabricator.wikimedia.org/T190710 - getting browser
tests ported to Node for Minerva - Thanks Željko for being so responsive!
* Blocking:
* Updates:
** Updating clients to reflect new wikidata description changes (
https://phabricator.wikimedia.org/T190953)
**Deploying Page previews to German Wikipedia (right now!!)
*Quarterly goal dependency update:
**Outcome 1, Objective 4: Continue improving the ways that users can
download articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
==== Readers Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** CSS endpoint going to be deployed in RESTBase after MediaWiki train
completes
** Working on very simplistic browser extension to add a single page to a
reading list
** TemplateStyles was deployed on ruwiki
*Quarterly goal dependency update:
**Outcome 1, Objective 4: Continue improving the ways that users can
download articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
**Increase code sharing of client apps by coalescing and moving more logic
to the server.
***Reading Infra depends on Parsing, Services
===== Maps =====
* Blocked by: None
* Blocking: None
* Updates:
** Working on internationalization and enwiki rollout
==== Multimedia ====
* Updates
** Wikibase patches for file page prototype going through
** Search work continuing
** File captions - currently working through changing the name of the field
for MediaInfo - it's a bear
*Quarterly goal dependency update:
**Objective 3.1 Prepare for launch of the first Structured Data on Commons
feature (multilingual file captions)
***SDC depends on Multimedia,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
** Objective 2.1 Integrate structured file captions into search
*** SDC depends on Search Platform, Multimedia
**Segment 4, Outcome 2: Develop a better understanding of existing needs
for Structured Commons- T171252
***Research depends on Multimedia
=== Contributors ===
==== Community Tech ====
* Blocked by:
* Blocking:
* Updates:
** CodeMirror is getting ready to promotion from beta
** Test deployment of GlobalPreferences today
==== Anti-Harassment Tools ====
* Blocked by:
* Blocking:
* Updates:
Blocking Tool Improvments
==== Editing ====
* Blocked by:
* Blocking:
** Updates:
** Goal 3.1 Improve and consolidate our unified editing platform so that
it's great on all devices
***Editing depends on Parsing
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
** Tidy replaced with RemexHtml on 40 more wikis today (see
https://phabricator.wikimedia.org/T190731 ) Currently have about 650
running RemexHTML, another 250 yet to convert.
** Parsoid is exploring the use of Finite State Transducer formalism for
language variants ( see
https://phabricator.wikimedia.org/T191925 && gerrit
-
https://gerrit.wikimedia.org/r/#/c/423197/ )
** TTO and us had an IRC meeting last week about the Poem extension and TTO
summarized the decisions in the phabricator task ( see
https://phabricator.wikimedia.org/T54061#4110198 )
** No quarterly goal dependency updates at this time. Services, Editing,
Parsing teams will be syncing up this/next week to identify any blockers.
*Quarterly goal dependency update:
**Goal 3.6 Support work towards unifying MediaWiki's parser
implementations, in liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
**Increase code sharing of client apps by coalescing and moving more logic
to the server.
***Reading Infra depends on Parsing, Services
** Goal 3.1 Improve and consolidate our unified editing platform so that
it's great on all devices
***Editing depends on Parsing
Outcome 2: Objective 1: Revision storage scaling
- Services depends on SRE, Parsing
==== Collaboration ====
* Blocked by:
* Blocking:
* Updates:
** Hoping to dark-deploy maps localization this week; had some false starts
and reverted deploys
==== Language ====
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**Goal 3.1 Improve and consolidate our unified editing platform so that
it's great on all devices
***Language depends on Editing
=== Audiences Design ===
* Blocked by:
* Blocking:
* Updates:
**
==== UI Standardization ====
* Blocked by:
* Blocking:
* Updates:
** OOUI – v0.26.3 released:
https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v…
*** Apex theme now uses WikimediaUI icon, this was long overdue in regards
of maintainability cost and that there were under 10% Apex specific icons
left
*** 4 new icons for 'tableMoveColumn*' & 'tableMoveRow*'
*** CheckboxInput-/MenuOptionWidget/MenuToolGroup: Don't specify icon in
CSS, reduces CSS output (Bartosz Dziewoński)
** Work on size unification aftermath in different products (Notifications,
Flow, RevisionSlider)
** Style Guide technical and contents improvements ongoing with 5 open PRs
currently
*** PR with big impact: Using system fonts as sans-serif choice –
https://phabricator.wikimedia.org/T188799
** Transparency Report: Wrapping up accessiblity measurements (continuation
from last week)
**
== Technology ==
=== Analytics ===
* Blocked by: traffic in a bit of issues
* Blocking:
* Updates:
**
** Major change on hadoop cluster to spark2, we try to avoid non backwards
compatible changes
** but this actually non backwards compatible so it will be a
stop-the-world change for some hours TODAY:
https://phabricator.wikimedia.org/T159962
** Mystery of the week around geo-location of pageviews for opera mini,
almost there, talking to opera team this week:
https://phabricator.wikimedia.org/T187014
** Continue working on measuring popups on all sites (now feature is
enabled everywhere minus dewiki and enwiki).
** Completed first release of geowiki data (editors per country), available
now in superset internally:
https://superset.wikimedia.org/superset/dashboard/9/
** Issues with mirrormarker and newer versions of kafka producers not
solved, job topics blocked from jumbo cluster
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
** OpenStack upgrades on Friday -- Horizon and CI will be disabled for a
while beginning around 14:00 UTC
** Short blog post published on Q3 FY17/18 highlights for WMCS <
https://phabricator.wikimedia.org/phame/post/view/91/cloud_services_team_q3…
** Andrew fixed a privledge escalation security
issue with Horizon's puppet
managment screens (T191433)
** Brooke is working on Wiki Replica view changes and needs some
instruction from the DBA team on how to depool servers for maintenance
there.
** Chase is working out Neutron gateway HA configuration in labtestn
** Madhu & Ariel migrated the last user-facing Dumps endpoints to the new
Cloud Services managed servers
=== Fundraising Tech ===
* CentralNotice: deploy CSP headers and violation alert on forced banner
previews
** Need to determine exact header to add
** Curious about the state of the larger CSP work
* Custom data field changes for our CiviCRM instance
* Custom reporting for CiviCRM to help find fraudsters faster
* More API update work for main CC processor, lately focused on the
recurring donation bits
=== MediaWiki Platform ===
* Blocked by:
* Blocking:
* Updates:
** MW 1.31 release planning
*** Pencils down on April 16
*** considering additional bundled extensions (T178349 and subtasks)
*** Making Remex default tidy implementation w/ James_F (T185753)
*** Running phan for extensions that are proposed to be bundled -
https://gerrit.wikimedia.org/r/#/q/status:open+topic:phan
*** Run tests with PHPUnit 6 -
https://gerrit.wikimedia.org/r/#/q/topic:phpunit-6
*** Handle extension dependencies in the installer (for new bundled
extensions, still WIP) -
https://gerrit.wikimedia.org/r/#/c/424967/
*** Also planning to do license reviews for new bundled extensions - e.g.
https://gerrit.wikimedia.org/r/#/c/424726/
*** Implement PlatformSettings.php RfC -
https://gerrit.wikimedia.org/r/#/c/423577/
** MCR:
*** ar_rev_id population patch merged!
*** maintenance/populateArchiveRevId.php run on all wikis!
*** Preparation schema change for ar_text dropping is still in progress
*** Working on MCR schema migration script
*** API updates for query modules are in Gerrit:
https://gerrit.wikimedia.org/r/#/c/413223/
*** Started working on the API "templated parameters" described in
https://phabricator.wikimedia.org/T174032#3990488. Not in Gerrit yet.
*** Code review
** Actor table:
*** Further deployment is waiting on the schema change (T188299)
*** 17 extension updates still need review:
https://gerrit.wikimedia.org/r/#/q/status:open+topic:bug/T167246-actor-migr…
** Comment table:
*** Follow up schema change is still in progress
https://gerrit.wikimedia.org/r/#/c/417039/
*** Toolforge view updating patch is merged and pending updates
*** The image_comment_temp merging patches are in Gerrit
*** All except one are blocked on the schema change
** Improving API test coverage:
*** 8 changes outstanding: improved test coverage for ApiMain,
ApiStructureTest, ApiBase, ApiParse, ApiMove, and ApiUserrights; two
improvements to ApiTestCase (getting rid of manual session mangling and
doLogin); and API usage errors in one more extension.
*** This is the last week for Aryeh
** Code review and bug fixes
** Quarterly goal dependency update:
**Goal 3.6 Support work towards unifying MediaWiki's parser
implementations, in liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
**** MWPT working on relevant planning for Platform Evolution CDP to
prepare for next FY and will coordinate with Parsing as this progresses
**Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**** MWPT meets weekly with SDC and WMDE
** Objective 1.1 Assist with deploying MultiContent Revisions on Commons
T174022 Implement multi-content revisions, T174023 Implement MCR storage
layer, T174045 DB schema migration for MCR, T174044 Deploy MCR storage
layer,T174043 Deploy Multi-Content Revisions
***SDC depends on MediaWiki Platform, WMDE
**** MWPT meets weekly with SDC and WMDE
=== Performance ===
* Blocked by: None
* Blocking: None
* Updates:
** Aaron/Timo are helping with train delay
**Moving our apps out of puppet, so that we don't need to bug SRE in order
to deploy changes
** Implemented WebP lossless compression support in thumbor on Vagrant for
testing, and it looks good. Gilles will be reachingout to folks on SRE and
Traffic to discuss some of the potential impacts of implementing in prod.
** bunches of code review
=== Release Engineering ===
* Blocked by:
* Blocking:
** scap and git-lfs work is blocking ores deployment. Mukunda is working on
it. Should be unblocked by the end of this week.
* Updates:
** Train is delayed, see task for updates:
https://phabricator.wikimedia.org/T183968
**Quarterly goal dependency update:
***Outcome 1, Objective 4: Continue improving the ways that users can
download articles of interest for later consumption
**** Reading Web depends on SRE, RelEng, Reading Infra
=== Research ===
* Blocked by:
* Blocking:
* Updates:
*Quarterly goal dependency update:
**Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**Segment 4, Outcome 2: Develop a better understanding of existing needs
for Structured Commons- T171252
***Research depends on Multimedia
=== Scoring Platform ===
* Blocked by:
** Still blocked by git-lfs. This is urgent for us, Adam is happy to work
on it synchronously if this might be helpful for RelEng?
* Blocking:
* Updates:
=== Search Platform ===
* Blocked by:
* Blocking:
* Updates:
**Need help with WDQS GUI build automation:
https://phabricator.wikimedia.org/T189811
**Evaluating feature generation by `query_explorer` functionality of LTR
plugin:
https://phabricator.wikimedia.org/T187148
**Working on some job queue issues:
https://phabricator.wikimedia.org/T191024
**Working on query parsing refactoring:
https://phabricator.wikimedia.org/T185108
**Working on Lexeme completion search:
https://phabricator.wikimedia.org/T190970
**Some new docs about running non-wikidata WDQS setups:
https://www.mediawiki.org/wiki/Wikidata_query_service/Implementation/Standa…
*Quarterly goal dependency update:
**Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**Objective 2.1 Integrate structured file captions into search
*** SDC depends on Search Platform, Multimedia
=== Security ===
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**Goal 3.6: Support work towards unifying MediaWiki's parser
implementations, in liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
=== Services ===
* Blocked by: Search on CirrusSearchElasticaWrite job serialization T191024
* Blocking: none
* Updates:
** Switched a big bulk of jobs (9) for test wikis to the kafka queue
** restbase-dev cluster work continues, so ignore any alert for it
*Quarterly goal dependency update:
**Increase code sharing of client apps by coalescing and moving more logic
to the server
***Reading Infra/Parsing, Services
**** working with Reading Infra on new CSS endpoints rollout
**Goal 3.6 Support work towards unifying MediaWiki's parser
implementations, in liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
Outcome 2: Objective 1: Revision storage scaling
***Services depends on SRE, Parsing
-
=== Site Reliability Engineering ===
* Blocked by:
** None
* Blocking:
** None
* Updates:
** ICU migration current ongoing
https://phabricator.wikimedia.org/T189295
** mcrouter 0.37 released and packaged already for Debian stretch. Jessie
is a bit more difficult, WIP
** Various countries routed to eqsin (Singapore PoP) once more (Japan,
India, Pakistan, Australia is being experimented with still)
** image scalers no longer received thumb.php traffic
** ping offloading to a dedicated host is happening
https://phabricator.wikimedia.org/T190090. Keep in mind that pings to LVS
IPs from now on will not be replied to by the hosts they used to be replied
from
*Quarterly goal dependency update:
**Outcome 1, Objective 4: Continue improving the ways that users can
download articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
*** services/SRE will schedule a meeting to discuss this
**Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
Outcome 2: Objective 1: Revision storage scaling
- Services depends on SRE, Parsing
== Wikidata ==
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
** Objective 1.1 Assist with deploying MultiContent Revisions on Commons
T174022 Implement multi-content revisions, T174023 Implement MCR storage
layer, T174045 DB schema migration for MCR, T174044 Deploy MCR storage
layer,T174043 Deploy Multi-Content Revisions
***SDC depends on MediaWiki Platform, WMDE
== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
** Raising FileImporter test coverage, currently at 48%:
https://phabricator.wikimedia.org/T190290
** Looking into MediaWiki CodeSniffer performance:
https://gerrit.wikimedia.org/r/#/projects/mediawiki/tools/codesniffer,dashb…
** Documenting FileImporter for performance review:
https://phabricator.wikimedia.org/T191283
** Currently deploying new Wikidiff2 to beta:
https://phabricator.wikimedia.org/T190717
** Blogging about the learnings on Wikidiff2:
https://phabricator.wikimedia.org/T190176
== SoS Meeting Bookkeeping ==
* Updates:
**please add links to Phab tickets in the goals update section