https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-08-22
= *2018-08-22* =
== Callouts ==
* SRE:
Switchover/Switchback dates finalized
Services: Tuesday, September 11th 2018 14:30 UTC
Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC
Traffic: Tuesday, September 11th 2018 19:00 UTC
MediaWiki: Wednesday, September 12th 2018: 14:00 UTC
Switchback:
Traffic: Wednesday, October 10th 2018 09:00 UTC
MediaWiki: Wednesday, October 10th 2018: 14:00 UTC
Services: Thursday, October 11th 2018 14:30 UTC
Media storage/Swift: Thursday, October 11th 2018 15:00 UTC
IMPORTANT: For the duration of the switchover (1 month), please refrain
from large db schema changes and please avoid deploying any kind of new
feature that requires creation of tables.
IMPORTANT: There will be a train freeze in the week of Sept 10th and Oct
8th.
* Release Engineering blocked by: Feedback needed (*how problems could have
been prevented*) from many people/teams on a recent MediaWiki train related
incident report.
** 1.32.0-wmf.13, 9 blockers, feedback needed for 8 of them:
https://wikitech.wikimedia.org/wiki/Incident_documentation/20180717-Train
** Aaron Schulz (Performance), Adam Wight (Scoring Platform), Bartosz
Dziewoński (Contributors), Brad Jorsch (MediaWiki Platform), C. Scott
Ananian (Contributors), Daniel Kinzler (Wikimedia Deutschland), Timo Tijhof
(Performance), Prateek Saxena (Audiences Design)
* Release Engineering looking for more participants to join the Code Health
Metrics working group. This group's purpose is to define and later
implement a set of core metrics that we will use to asses the health of our
code base. More info:
https://www.mediawiki.org/wiki/Code_Health_Group/projects/Code_Health_Metri…
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by: -
* Blocking: -
* Updates:
**6.0.0 released (
https://phabricator.wikimedia.org/tag/ios-app-v6.0-walrus-on-a-unicycle/ )
** 6.0.1 to be released next week (
https://phabricator.wikimedia.org/tag/ios-app-v6.0.1-walrus-on-a-golf-cart/
)
** 6.1 up next, release date TBD (
https://phabricator.wikimedia.org/tag/ios-app-v6.1-narwhal-on-a-bumper-car/
)
==== Android native app ====
* Blocked by:
* Blocking:
* Updates:
** Finishing up polling Echo notifications (limited to certain types of
notifications initially)
==== Readers Web ====
* Blocked by:
* Blocking:
* Updates:
** Mobile website (MinervaNeue / MobileFrontend):
*** Page issues UI and instrumentation: T191528 T201124 T197932 T196150
*** Invest in the MobileFrontend & MinervaNeue frontend architecture:
T188937
*** Avoid abrupt and jarring white flash in media viewer: T197110
*** Other fixes and hygiene: T197499 T199000 T197497 T202021 T201956
==== Readers Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** PCS:
*** Separating pagelib CSS from base CSS this week
*** Adding simple header to mobile-html (title, description, page image)
** Maps:
*** Nothing to report.
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
** Nothing to report
==== Multimedia ====
* Blocked by:
* Blocking:
* Updates
** SDoC: Discussion with search team about searching for related statements
** SDoC: prototyping depicts search
** SDoC: design artwork
** No longer blocked by MCR
=== Contributors ===
==== Community Tech ====
* Blocked by:
* Blocking:
* Updates:
** No update.
==== Anti-Harassment Tools ====
* Blocked by:
* Blocking:
* Updates:
** Continuing work on Partial Blocks:
https://phabricator.wikimedia.org/T2674
** Updated the TechComm RFC to include Multiblocks:
https://phabricator.wikimedia.org/T199917
==== Editing ====
* Blocked by:
* Blocking:
** Updates:
** Improved/bug fixes: collapsed tables in visual diffs; insertion of
horizontal rules; pages with lots of tables in WTE 2017; switching between
visual editor and source mode; VE linking in IE11; regressions with icon
displays; etc.
** Compact multiple transactions into one:
==== Growth ====
* Blocked by:
** SRE: Query about plausibility of measuring e-mail traffic to our
no-reply addresses https://phabricator.wikimedia.org/T202329
* Blocking:
* Updates:
** No update.
==== Language ====
* Blocked by:
* Blocking:
* Updates:
** No update.
=== Audiences Design ===
* Blocked by:
* Blocking:
* Updates:
** No update.
==== UI Standardization ====
* Blocked by:
* Blocking:
* Updates:
** No update.
== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
** Changes to hash and salt (every 90 days) appinstallIDs are on, these
will be retained (hashed) forever
**Working with performance in making eventlogging javascript module lighter
so it can be loaded on 1st page load.
**Working on computing “top contributors” metric per project so it is
available in wikistats2, this is one of the most coveted reports by our
community.
**Spark available now in Jupyter notebooks.
**Meetings around modern event data platform to define what parts will we
start developing next quarter.
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
** No update.
=== Fundraising Tech ===
* Blocked by:
* Blocking:
* Updates:
** Put off upgrade of mediawiki on payments servers
** testing payment notification listener on debian stretch and php 7.0
** Bugfixes and UI tweaks for new card processor API integration
** Nearly done new system for ingesting EventLogging data about banners and
FR landing pages
*** Activated EventLogging for all banner campaigns, sampled at 1% on the
front end.
** Working on email Opt-In page
** CiviCRM:
*** reviewing final group of patches for donor data export and deletion
*** bugfixes on imports
*** Looking at performance of donation imports
=== Core Platform ===
* Blocked by:
* Blocking:
* Updates:
** LoadBalancer overload investigation (T180918)
** TitleValue performance (T201801)
** Awaiting review: Security review for Guzzle 6.3.3 (T202143)
** In testing: Create GuzzleHttpRequest class as new default for
HttpRequestFactory (T202110)
** Ready for testing: Convert MultiHttpClient to use Guzzle (T202352)
** Platform Evolution: developing sessions for TechConf
** REST mobile endpoints removed from projects other then wikipedia and
wikivoyage
** New event type coming to EventBus on this train:
mediawiki.revision-tags-change
=== Performance ===
* Blocked by:
* Blocking:
* Updates:
** Deferred jQuery loading is rolling out to all wikis on this week's train
** All wikis using mcrouter
** nutcracker writes have been disabled
** Mcrouter broadcast operations enabled for all wikis
** patched a bug in Html::inlineScript
** Removed use of StartProfiler.php from mw-init
=== Release Engineering ===
* Blocked by:
** Feedback needed (*how problems could have been prevented*) from many
people/teams on a recent MediaWiki train related incident report.
*** 1.32.0-wmf.13, 9 blockers, feedback needed for 8 of them:
https://wikitech.wikimedia.org/wiki/Incident_documentation/20180717-Train
*** Aaron Schulz (Performance), Adam Wight (Scoring Platform), Bartosz
Dziewoński (Contributors), Brad Jorsch (MediaWiki Platform), C. Scott
Ananian (Contributors), Daniel Kinzler (Wikimedia Deutschland), Timo Tijhof
(Performance), Prateek Saxena (Audiences Design)
* Blocking:
** MediaWiki 1.29 final release and EOL; was due in June:
https://phabricator.wikimedia.org/T197669 (w/ Security)
* Updates
** New general purpose CI job that builds and runs test containers via
Blubber/Docker based on config provided in each project (think
`.travis.yml` file)
*** Read more about Blubber here:
https://wikitech.wikimedia.org/wiki/Blubber
*** See recent builds at
https://integration.wikimedia.org/ci/blue/organizations/jenkins/blubber-tes…
*** Gives developers one major benefit of the CD pipeline work now, having
control over their pre-merge and gating tests without having to mess with
integration/config
*** Only scheduled to run for a few repos at the moment, but will
eventually be expanded to many more projects (we need to tune CI infra
around it first)
** Looking for more participants to join the Code Health Metrics working
group. This group's purpose is to define and later implement a set of core
metrics that we will use to asses the health of our code base. More info:
https://www.mediawiki.org/wiki/Code_Health_Group/projects/Code_Health_Metri…
=== Research ===
* Blocked by: None
* Blocking: None
* Updates:
** Deployed a new UnsourcedStatements campaign (en, fr, it):
https://labels.wmflabs.org
** Finished the first phase of the "Elicit new editor interests" research.
Starting the second phase soon.
** Looking into using Cassandra as a storage engine of Recommendation JS
API.
=== Scoring Platform ===
* Blocked by: SRE, RelEng
* Blocking: None
* Updates:
** All SP repos have been moved to wikimedia organization in github.
** Git LFS support for ores models is going live, some help from RelEng
would be great: https://phabricator.wikimedia.org/T192042
** JADE scalability discussion continues, TechCom meeting is scheduled for
today.
** PoolCounter support to limit number of external connections to ORES has
started. SRE input would be appericiated:
https://phabricator.wikimedia.org/T201824
** Wikilabels robustness is being worked on, some puppet changes will need
review. The node is being reimaged to stretch
=== Search Platform ===
* Blocked by:
* Blocking:
* Updates:
** Published blog post about search tokenization:
https://wikimediafoundation.org/2018/08/07/anatomy-search-token-affection/
** Added support for haswbstatement without value (any statement with given
property) https://phabricator.wikimedia.org/T199886
** Re-indexed wikis in Malay, Indonesian, and Polish, enabling Malay
stemming and fixing errors in Polish stemming (
https://phabricator.wikimedia.org/T196780,
https://phabricator.wikimedia.org/T186046)
** Created ElasticSearch plugin for Esperanto analysis:
https://phabricator.wikimedia.org/T200099
** Migrated ElasticSearch cluster to RAID0 and upgraded to Debian stretch:
https://phabricator.wikimedia.org/T198391
** Working on migrating communications from analytics to search cluster to
Kafka: https://phabricator.wikimedia.org/T198490
** Working on running multiple Elastic instances on the same hardware:
https://phabricator.wikimedia.org/T193654
** Working on ES 6.3 upgrade: https://phabricator.wikimedia.org/T197960
** Working on query parsing refactoring:
https://phabricator.wikimedia.org/T185108
=== Security ===
* Blocked by:
* Blocking:
** MediaWiki 1.29 final release and EOL; was due in June:
https://phabricator.wikimedia.org/T197669 (w/ RelEng)
* Updates:
**
=== Site Reliability Engineering ===
* Blocked by:
* Blocking:
* Updates:
** No update.
== Wikidata ==
* Blocked by:
** none
* Blocking:
** none
* Updates:
** Work in Lexicographical data continues:
*** adding senses
*** improving the editor workflow
*** https://phabricator.wikimedia.org/project/view/2292/
** Experiment about not using secondary SQL table for formatting item links
being gradually deployed starting Thu Aug, 23rd
*** https://phabricator.wikimedia.org/T201831
== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
** No update.
== Multi-Content Revisions ==
* Blocked by:
* Blocking:
* Updates:
** MCR schema migrations are in progress
*** Finished running populateContentTables.php except for a few small wikis
*** Filed T202032: duplicate ar_rev_id values causing
populateContentTables.php to fail
** Phase 2 (file caption support) development tasks nearing completion
** Phase 3 (statement support) development tasks commencing
== SoS Meeting Bookkeeping ==
* Updates:
**
Hey,
ORES repositories used to exist in wiki-ai organization in github.com. For
example you could reach them at github.com/wiki-ai/ores or
github.com/wiki-ai/wikilabels
They all are now moved to wikimedia organization. So they exist in
github.com/wikimedia/ores or github.com/wikimedia/wikilabels. You can get
list of all of scoring platform team repos in [1]
This is needed for better organizational hierarchy and maintenance and also
better support for git LFS [2]. The old URLs redirect to the new ones so
you should not face any issues in developments but at one point in really
far future these might stop working, so please make sure you update the
urls. Also, if you encounter any issues, let me know.
Sorry for any inconvenience and thank you for your contribution.
[1]: https://github.com/orgs/wikimedia/teams/scoring-platform/repositories
[2]: https://git-lfs.github.com/
Best
--
Amir Sarabadani
Software Engineer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
Hello everyone,
I'm pleased to announce that we've released OOUI v0.28.0 this Tuesday.
Key highlights of this release:
- New OOUI “Tutorials” section[0]
Special thanks to GSOC student Hagar Shilo, who has provided us an excellent
addition to our documentation.
The tutorials simplify stepping into development with the library.
- Final code cleanup step after major icon refinement[1], see below.
Possible breaking changes:
The following changes might cause breaking of your interface, please carefully
consider if they affect your code.
- Drop 'find' icon, deprecated in v0.26.2
- Drop 'settings' icon, deprecated in v0.27.0
- Drop cite icons, renamed and deprecated in v0.27.0
- Remove 'clip' & 'unClip', deprecated in v0.26.1
- Remove 'comment', deprecated in v0.26.1
- Remove deprecated 'userActive'/'userInactive', deprecated in v0.26.2
All six icon changes above have been deprecated and announced in former
versions. With this release we've removed them completely.
You can find details on additional new features, code-level and accessibility
changes, styling and interaction design amendments, and all
improvements since v0.27.0 in the full changelog[2].
I'd also like to give a shoutout to Alangi Derick and Tim Eulitz for
their contributions.
If you have any further queries or need help dealing with breaking
changes, please
let me know.
As always, library documentation is available on mediawiki.org[3],
there is comprehensive generated code-level documentation and
interactive demos and tutorials hosted on doc.wikimedia.org[4].
OOUI version: 0.28.0
MediaWiki version: 1.32.0-wmf.18
Date of deployment to production: Regular train, starting Thursday 16 August
[0] - https://doc.wikimedia.org/oojs-ui/master/demos/tutorials/index.html
[1] - https://phabricator.wikimedia.org/T177432
[2] - https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md
[3] - https://www.mediawiki.org/wiki/OOUI
[4] - https://doc.wikimedia.org/oojs-ui/master/
Best,
Volker
--
Senior UX Engineer
Wikimedia Foundation
volker.e(a)wikimedia.org | @Volker_E
I just got banned from Freenode server for @ing a few people during a SWAT
deploy. Be careful everyone! It's a dangerous world out there!
I've taken the advice and emailed kline but in the meantime, you may want
to be careful who/how many people you @.
Will let you know when I know more and apologies if you can't get hold of
me in the mean time!
Jon
Replay of the event:
<jouncebot> Jdlrobson and nray: A patch you scheduled for Evening SWAT (Max
6 patches) is about to be deployed. Please be around during the process.
Note: If you break AND fix the wikis, you will be rewarded with a sticker.
4:03 PM <jdlrobson> Jon Robson \o
4:03 PM <nray> Nicholas Ray \o
4:05 PM <jdlrobson> Jon Robson mark: twentyafterfour RoanKattouw thcipriani
Niharika any of you able to swat right now?
4:05 PM ⇐ You were killed by Sigyn (sigyn@freenode/utility-bot/sigyn):
(Spam is off topic on freenode.)
4:05 PM ⇐ jdlrobson quit (sid92657@gateway/web/
irccloud.com/x-maentctwgksrmedg) Killed (Sigyn (Spam is off topic on
freenode.))
Check your host, port and ssl settingsDisconnected: You are banned from
this server- Please do not spam users or channels on freenode. If in error,
please contact kline(a)freenode.net. (2018/8/16 23.03)
<https://www.irccloud.com/irc/freenode/channel/wikimedia-operations>
--
Jon Robson
Senior Software Engineer
twitter: @jdlrobson
linkedin: https://www.linkedin.com/in/jorobson/
Reminder: Technical Advice IRC meeting again **Wednesday 3-4 pm UTC** on
#wikimedia-tech.
Question can be asked in English, German, Persian.
The Technical Advice IRC Meeting (TAIM) is a weekly support event for
volunteer developers. Every Wednesday, two full-time developers are
available to help you with all your questions about Mediawiki, gadgets,
tools and more! This can be anything from "how to get started" over "who
would be the best contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting:
https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
Hope to see you there!
Michi (for the Technical Advice IRC Meeting crew)
--
Michael F. Schönitzer
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
Starting September 1, huwiki and arwiki, which both take several days to
complete the revsion history content dumps, will be moved to the 'big
wikis' list, meaning that they will run jobs in parallel as do frwiki,
ptwiki and others now, for a speedup.
Please update your scripts accordingly. Thanks!
Task for this: https://phabricator.wikimedia.org/T202268
Ariel
Dear sir/madame;
I am Awah Nadege Tayebatu, female and a Cameroonian. I am an Applied
Geology student of the University of Buea, Cameroon. I will love to
contribute code to MediaWiki and so will appreciate some guide on how to
start contributing and some learning resources i can use as a beginner.
Thank you.
Hello everyone,
On 6th July 2017, we made an announcement [1] about our plans to replace
Tidy
with RemexHtml on the Wikimedia cluster.
On 5th July 2018, we made the final switchover from Tidy [2].
For those of you interested, we published a blog post [3] documenting the
process and steps in this project.
What next?
----------
RemexHtml is the default in MediaWiki since 1.31.
We are in the process of removing all traces of Tidy from Wikimedia's
configuration files and from MediaWiki itself. We'll keep the
ParserMigration
extension enabled in its current form for another couple of months.
After this
period, the ParserMigration extension will be disabled completely
alongwith all
associated Tidy configuration. It will be re-purposed at a later time as
required
to compare impacts of other parser changes.
We haven't figured out the timeline yet, but at some point in the coming
weeks,
we'll disable most of the high-priority linter categories that only make
sense
in the context of replacing Tidy. If you have any thoughts about
retaining or
dropping these categories, please leave a note on
mw:Help_talk:Extension:Linter [4].
Thanks!
-------
As noted in the blog post [3], this was a collaborative effort between the
several teams at the Wikimedia Foundation and volunteer editing
communities on
various wikis. Everyone involved played important and specific roles in
getting
us to this important milestone. An active embrace by various wikis of this
effort has let the Foundation make this much-needed and important
upgrade of a
key piece of our platform.
In that context, I want to take this opportunity to specifically name
and thank
some of these individuals.
Tim Starling, C.Scott Ananian, Kunal Mehta, Arlo Breault, and yours
truly did
all the technical work in core, Parsoid, and the extensions. Erica Litrenta
and Sherry Snyder helped develop the community engagement plan with various
wikis and specific editors. Sherry, in particular helped a lot in the final
months as we were trying to get more wikis to fix pages. James Forrester
helped review and provide feedback at various stages, especially with the
phased deployment. A number of other developers and staff were involved in
review, feedback, during RFC discussions, and in helping with fixing
templates where I couldn't do so myself.
Here is a subset of users across several wikis that I had an opportunity to
interact with and observe who helped in various ways: developing bots and
gadgets, writing help pages, asking clarification questions, providing
feedback, reporting bugs on wiki and Phabricator, and fixing lots of pages
and templates on their home and other wikis.
User:Daimano Eaytoy, User:Sakretsu, User:Anamalocaris, User:Izno, User:Lsj,
User:xaosflux, User:PerfektesChaos, User:Lómelinde, User:Ikhitron,
User:星耀晨曦, User:Deryck Chan, User:Sunpriat, User:Stryn, User:MawaruNeko,
User:NicoV, User:Bdijkstra, User:Skalman, User:Shakespearefan00,
User:Billinghurst, User:MarcoSwart, User:Mackensen, User:Andriy.v,
User:Jonesey95, User:TheDJ, and the anonymous wiki user whoever you are. :-)
Obviously, there were a number of others users involved in this effort.
Please
feel free to raise your hand or recognize other's contributions on this
thread!
My thanks to you all!
Subbu.
1.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2017-July/001625…
2. https://phabricator.wikimedia.org/T175706#4399641
3. https://blog.wikimedia.org/2018/07/09/tidy-html5-replacement/
4. https://www.mediawiki.org/wiki/Help_talk:Extension:Linter