📘 Read on Phabricator at
https://phabricator.wikimedia.org/phame/post/view/173/
-------
How’d we do in our strive for operational excellence last month? Read on to
find out!
## 📊 Month in numbers
* 5 documented incidents. [1]
* 22 new errors reported. [2]
* 31 error reports closed. [3]
* 213 currently open Wikimedia-prod-error reports in total. [4]
There were five recorded incidents last month, equal to the median for this
and last year. – Explore this data at https://codepen.io/Krinkle/full/wbYMZK
To read more about these incidents, their investigations, and pending
actionables; check
https://wikitech.wikimedia.org/wiki/Incident_documentation#2019
## *️⃣ A Tale of Three Great Upgrades
This month saw three major upgrades across the MediaWiki stack.
*Migrate from HHVM to PHP 7.2*
The client-side switch to toggle between HHVM and PHP 7.2 saw its final
push — from the 50% it was at previously, to 100% of page view sessions on
17 September. The switch further solidified on 24 September when static
MediaWiki traffic followed suit (e.g. API and ResourceLoader). Thanks Effie
and Giuseppe for the final push. – More details at
https://phabricator.wikimedia.org/T219150 and
https://phabricator.wikimedia.org/T176370.
*Drop support for IE6 and IE7*
The RFC to discontinue basic compatibility for the IE6 and IE7 browsers
entered Last Call on 18 September. It was approved on 2 Oct (T232563).
Thanks to Volker Eckl for leading the sprint to optimise our CSS payloads
by removing now-redundant style rules for IE6-7 compat. – More at
https://phabricator.wikimedia.org/T234582.
*Transition from PHPUnit 4/6 to PHPUnit 8*
With HHVM behind us, our Composer configuration no longer needs to be
compatible with a “PHP 5.6 like” run-time. Support for the real PHP 5.6 was
dropped over 2 years ago, and the HHVM engine supports PHP 7 features. But,
the HHVM engine identifies as “PHP 5.6.999-hhvm”. As such, Composer refused
to install PHPUnit 6 (which requires PHP 7.0+). Instead, Composer could
only install PHPUnit 4 under HHVM (as for PHP 5.6). Our unit tests have had
to remain compatible with both PHPUnit 4 and PHPUnit 6 simultaneously.
Now that we’re fully on PHP 7.2+, our Composer configuration effectively
drops PHP 5.6, 7.0 and 7.1 all at once. This means that we no longer run
PHPUnit tests on multiple PHPUnit versions (PHPUnit 6 only). The upgrade to
PHPUnit 8 (PHP 7.2+) is also unlocked! Thanks Max Sem, Jdforrester and
Daimona for leading this transition. –
https://phabricator.wikimedia.org/T192167
-------
## 📉 Outstanding reports
Take a look at the workboard and look for tasks that might need your help.
The workboard lists error reports, grouped by the month in which they were
first observed.
→ https://phabricator.wikimedia.org/tag/wikimedia-production-error/
Or help someone that’s already started with their patch:
→ https://phabricator.wikimedia.org/maniphest/query/pzVPXPeMfRIz/#R
Breakdown of recent months (past two weeks not included):
* February: 1 report was closed. (1 / 5 reports left).
* March: 4 / 10 reports left (unchanged).
* April: 8 / 14 reports left (unchanged). ⚠️
* May: The last 4 reports were resolved. Done!
* June: 9 of 11 reports left (unchanged). ⚠️
* July: 4 reports were fixed! (13 / 18 reports left).
* August: 6 reports were fixed! (8 / 4 reports left).
* September: 12 new reports survived the month of September.
## 🎉 Thanks!
Thank you, to everyone else who helped by reporting, investigating, or
resolving problems in Wikimedia production. Thanks!
Until next time,
– Timo Tijhof
-------
Footnotes:
[1] Incidents. –
https://wikitech.wikimedia.org/wiki/Special:PrefixIndex?prefix=Incident+doc…
[2] Tasks created. –
https://phabricator.wikimedia.org/maniphest/query/XicVcsN1XkVH/#R
[3] Tasks closed. –
https://phabricator.wikimedia.org/maniphest/query/SXjsllmYHwAO/#R
[4] Open tasks. –
https://phabricator.wikimedia.org/maniphest/query/47MGY8BUDvRD/#R
This week Fresnel 0.5.0 was released and deployed to Jenkins.
Highlighted changes:
- Add support for Mann–Whitney U test. [1] –
https://phabricator.wikimedia.org/T223977,
https://en.wikipedia.org/wiki/Mann-Whitney_U_test
- Switch regression detection from diffStdev to diffMannWhitney (for
Paint Timing metrics).
- Update Chromium from 73.0 to 77.0.
- Enable Gzip for static files in web server behind Fresnel and Quibble.
– https://gerrit.wikimedia.org/r/539427
I've written a visual guide for how to open Fresnel's performance report in
your Chrome DevTools locally:
👉 https://wikitech.wikimedia.org/wiki/Performance/Fresnel#DevTools
-- Timo Tijhof
Change log and tracking task for Fresnel 0.5.0:
https://github.com/wikimedia/fresnel/blob/0.5.0/CHANGELOG.mdhttps://phabricator.wikimedia.org/T235195
---------- Forwarded message ---------
From: Timo Tijhof <krinklemail(a)gmail.com>
Subject: Fresnel for MediaWiki (performance testing)
Date: Thu, Mar 7, 2019 at 1:10 AM
You may have noticed something called "mediawiki-fresnel" leaving messages
on Gerrit patches for MediaWiki in the past few days, and wondering what
it's all about. Allow me to introduce Fresnel!
Fresnel is an automation tool for measuring and comparing client-side
performance from web pages. Fresnel was developed over the past two
quarters and is now ready for action. [1] [2] [3]
To learn more about how to use it, what it offers, and how it works, check
out:
https://wikitech.wikimedia.org/wiki/Performance/Fresnel
Some feature highlights:
- ⏱ *Metrics* from Navigation Timing, Paint Timing, and Resource Timing
APIs.
- 📊 DevTools *Timeline* from CI recording can be viewed locally in
Chrome.
- 🖼 Recordings take a *screenshot* available in build artefacts.
- 🔥 Scenarios perform a *warmup* and multiple runs for more stable
metrics.
--
Timo Tijhof
[1] Launch task: https://phabricator.wikimedia.org/T133646
[2] Phabricator project: https://phabricator.wikimedia.org/tag/fresnel/
[3] Task list:
https://phabricator.wikimedia.org/maniphest/query/9w6EAEPPLQ72/#R
Wikimedia-l,
Wikitech-l,
It appears that there is a growing consensus in the discussion area for the Wikiwisdom sister project proposal [1] that some of the ideas which comprise the proposal are also feature requests for the Wikiquote project. These ideas include: those of a structured Wikiquote (e.g. Wikidata integration, individual quotes as structured data items, auto-generated pages from Wikidata queries (e.g. pages listing quotes by author)) as well as social media integration so that users could readily share individual quotes with one another on social media.
I would like to ask how best to get the project proposal information [1] to the Wikiquote project team as well as how best to propose the feature requests to the Wikiquote project.
Thank you,
Adam Sobieski
[1] https://meta.wikimedia.org/wiki/Wikiwisdom
The 1.35.0-wmf.3 version of MediaWiki is no longer blocked[0], and was
deployed to group 0 earlier today at 19:27 UTC, using the American
deploy window.
Thanks to andrewbogott for supplying helpful context in unblocking the
train, as well as hashar, thcipriani, mutante, and paladox for dealing
speedily with Gerrit migration issues.
The train should proceed to group 1 as regularly scheduled, during the
European deploy window.[1]
-- Your unassuming train minion
[0]. <https://phabricator.wikimedia.org/T233851>
[1].
<https://wikitech.wikimedia.org/wiki/Deployments#deploycal-item-20191023T1300>
// sorry for cross-posting
Hello,
a new feature called Reference Previews [1] will soon be released as a beta
feature to more wikis. The planned deployment date is October 24 [2] around
15:00 UTC.
As you might guess from the name, this feature gives you a preview of
references in the article text. That means, you can look up a reference
without jumping down to the bottom of the page. If specified, the preview
also shows the type of a reference: whether it’s a web, journal, book or
news reference. This can help readers evaluate the trustworthiness of a
citation more quickly. If you'd like to know more about this feature,
please visit the project page.[1]
Reference Previews originated from a wish from the German-speaking wiki
community to have the preview functionality that already exists in many
wikis integrated into MediaWiki. Coordination and implementation of this
feature were done by the Technical Wishes team at Wikimedia Deutschland,
with support from the WMF’s Reading Web team. The design for the feature
was developed by the WMF. The feature is part of the MediaWiki Popups
extension [3], which is also used for the Page Previews feature.[4]
Reference Previews have been a beta feature on German and Arabic Wikipedia
for some months already, with around 3,000 users trying it out. Since then,
a few bugs were fixed, and it’s been made easier to enable the beta
feature. Now Reference Previews will become a beta feature on all
Wikipedias and on three Wikivoyage projects.[2]
We’re inviting everyone to give the feature a try: Please activate the beta
feature [5] and let us know what you think on the central feedback page
[6]! A big thanks to everyone who gave feedback so far.
If the feature isn’t available on your wiki [2] and you want to have it as
a beta feature, please reach out to us on the central feedback page.[6]
For the Technical Wishes team,
Johanna
[1] project page:
https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/ReferencePreviews
[2] deployment roadmap:
https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/ReferencePreviews#Dep…
[3] Popups extension: https://www.mediawiki.org/wiki/Extension:Popups
[4] Page Previews: https://www.mediawiki.org/wiki/Page_Previews
[5] go to Preferences > Beta features, e.g.
https://meta.wikimedia.org/wiki/Special:Preferences#mw-prefsection-betafeat…
[6] https://www.mediawiki.org/wiki/Help_talk:Reference_Previews
Johanna Strodt
Project Manager Community Communications Technical Wishlist, Wikimedia
Deutschland
This was already announced in the last TechCom Radar email, but in case you
missed it:
In an hour, we will be talking about the use of common storage abstractions
in MediaWiki:
<https://phabricator.wikimedia.org/T215465>
You can join the discussion in the #wikimedia-office channel.
--
Alex Paskulin
Technical Writer
Wikimedia Foundation
Hi Everyone,
It's time for Wikimedia Tech Talks 2019 Episode 9! This talk will take
place 23, October, 2019 at 6PM UTC.
*Title:* ResourceLoader trips and tricks
*Speaker:* Roan Kattouw
*Summary:*
Did you know that you could require() files in JavaScript? That you could
make your own icon modules with 10 lines of code? That there's a new way to
export configuration variables to JavaScript?
Learn about new ResourceLoader features introduced this year, and how you
can use them to improve your code.
We'll start with a quick introduction to ResourceLoader, then dive into
some of the advanced features like require(), config var bundling,
generated JSON files and icon modules.
The link to the Youtube Livestream can be found here:
https://www.youtube.com/watch?v=AGiDYvcroWE
During the live talk, you are invited to join the discussion on IRC at
#wikimedia-office
You can watch past Tech Talks here:
https://www.mediawiki.org/wiki/Tech_talks
If you are interested in giving your own tech talk, you can learn more
here:
https://www.mediawiki.org/wiki/Project:Calendar/How_to_schedule_an_event#Te…
Note: Please feel free to distribute this email to any lists that you feel
are appropriate!
Hope to see you there!
Kindly,
Sarah R. Rodlund
Technical Writer, Developer Advocacy
<https://meta.wikimedia.org/wiki/Developer_Advocacy>
srodlund(a)wikimedia.org
Hi,
for HTML version see
https://www.mediawiki.org/wiki/Scrum_of_scrums/2019-10-23
Željko
--
= 2019-10-23 =
== Callouts ==
* SRE DBAs needed someone (CPT?) to pick this [[phab:T234450]] as it will
help mitigating some on-going heavy requests. Multiple people already
stepped up. Thanks
* Infrastructure foundations Blocked by CPT on service-runner PR,
requesting follow-up https://github.com/wikimedia/service-runner/pull/221
== Product ==
=== Community Tech ===
* Updates:
** 2020 Wishlist Survey started on Monday, October 21. Proposals accepted
until November 11. Voting begins November 20.
** Who Wrote That browser extension is available in beta for Firefox.
Chrome extension coming soon. A gadget version is also possible.
** Initial work on Watchlist Expiry beginning
=== Anti-Harassment Tools ===
* Updates:
** Pushing bug fixes for Blocks refactoring
** Initial work on CheckUser to begin soon
=== Editing ===
* Updates:
** Use consistent tag style when constructing nodes
** identifyLeadParagraph: Ignore invisible nodes, slugs and whitespace
(task [[phab:T235222]])
** Use plugin to ensure all message keys are documented (task
[[phab:T235502]])
** mobile.init/editor: Fix transition of collapse icon after icon changes
(task [[phab:T235692]])
** Improve message for the infamous "HTTP 404" error
** Fix media embeds which don't link to the file page (task
[[phab:T235707]])
** Don't retry when etag is undefined because the page doesn't exist (task
[[phab:T233320]])
** Detect mangled etags from RESTBase and retry via MediaWiki API (task
[[phab:T233320]])
** ApiVisualEditor: Always return 'etag' with 'content' (task
[[phab:T233320]])
** resources: Allow loading Moment.js like a module
** Don't apply styling for Special:Contributions on other pages (task
[[phab:T235137]])
=== Growth ===
* Updates:
** Add RemoteSearchTaskSuggester (task [[phab:T234426]])
** Newcomer tasks: "No results" card widget (task [[phab:T235044]])
** Suggested edits 1.0 styles (task [[phab:T232423]])
** Newcomer tasks: "No more suggestions" card widget (task [[phab:T235043]])
** Growth tasks API: Update query limit to 200 (task [[phab:T232423]])
(task [[phab:T234426]])
** Make EchoSeenTime cache entries expire after 1 year (task
[[phab:T222851]])
** Make EchoSeenTime cache type configurable (task [[phab:T222851]])
** Add faux SearchResult and SearchResultSet
=== iOS native app ===
* Updates:
** 6.5 in active development - [[phab:project/view/4245]]
***Bug fixes
****3D touch in article view
*** History & diffs feature
**** User testing next week
**** Core Platform working on Rest APIs for this
=== Android native app ===
* Updates:
** Continuing to make enhancements to EditTasks extension to support new
API requirements.
** Preparing updates for upcoming in-app fundraising.
=== Readers Web ===
* Blocked by:
** Performance: MachineVision performance review [[phab:T230813]]
=== Product Infrastructure ===
* Blocking:
** Structured Data: Security & performance review for MachineVision:
[[phab:T227349]]
* Updates:
* BUOD
** Reviewing sentry-js for client-side error logging [[phab:T235189]]
** Implementing MEP stream configuration continues [[phab:T233634]]
* MachineVision
** Avoid duplicate depicts [[phab:T234457]]
** Show labels in the appropriate language [[phab:T235556]]
** Support Wikidata Id blacklist [[phab:T231914]]
* Maps
** Deployed new version of kartotherian
=== Structured Data ===
* Blocked by:
** Product Infrastructure: Security & performance review for MachineVision:
[[phab:T227349]]
* Blocking:
** Search: RecentChange support for SDC: [[phab:T230862]] - patch in CR
** Search: Data dumps for SDC: [[phab:T221917]]
* Updates:
** MachineVision progressing
** Preparing to implement support for additional datatypes in MediaInfo
** Lua support for MediaInfo items coming soon (thanks WMDE!)
=== Inuka ===
* Updates:
** KaiOS app: Building the backlog
** KaiOS app: Working on the instrumentation plan
** KaiOS app: Working on the reading experience ([[phab:T234435]]
[[phab:T234436]])
== Technology ==
=== Analytics ===
* Blocking:
** Search Platform: [[phab:T229882]]
=== Fundraising Tech ===
* Updates:
** CentralNotice: Campaign fallback is rolling out on the train this week
** CentralNotice: Investigating new SameSite cookie attribute for Chrome
[[phab:T235204]]
** CentralNotice data pipeline: Debugging discrepancy in landing page views
from initial run [[phab:T235284]]
** CiviCRM: Investigating null values in financial table rows for S. Amer /
India payment processor [[phab:T235212]]
** CiviCRM: Fix merge for two contacts where neither has a contribution
[[phab:T235890]]
** CiviCRM: Reviewing Replay-on-write database extension to use primary /
replica DBs.
** Payments-wiki: Set banner-hide cookies earlier when asking donors to add
a monthly donation [[phab:T235209]]
=== Core Platform ===
* Blocking:
** SRE on service-runner PR, requesting follow-up
https://github.com/wikimedia/service-runner/pull/221
** SRE DBAs needed someone (CPT?) to pick this [[phab:T234450]] as it will
help mitigating some on-going heavy requests. Multiple people already
stepped up. Thanks
* Updates:
** Rollout of Kask for sessions to Group 1 wikis
** Rollout of Kask for echo notification timestamps to Group 0
** Last endpoints for history API for iOS [[phab:T231338]]
** Planning next sprint for "minimal client' REST API [[phab:T229662]]
* Thanks
** Traffic for unblocking routing of REST requests
** @halfak for fixing my notes
=== Engineering Productivity ===
==== Performance ====
* Blocked by:
** SRE/ServiceOps: PHP7 fatal errors not reported in Logstash
[[phab:T234283]]
* Blocking:
** SRE: Navtiming on Prometheus [[gerrit:534771]]
** Readers Web: MachineVision performance review [[phab:T230813]]
==== Release Engineering ====
* Blocked:
** Site Reliability Engineering: logstash-beta.wmflabs.org does not receive
any mediawiki events [[phab:T233134]]
* Updates:
** v3 of CI arch document for new CI wants review and feedback
[[User:LarsWirzenius/NewCI]]
** Gerrit is now on a newer (larger) server
*** There was some problem with rollout (patches looking unmerged that were
actually merged)
*** Resolved now
*** All gory details on [[phab:T236114]]
** Train Health
*** Last week: 1.35.0-wmf.2 - [[phab:T233850]]
*** This week: 1.35.0-wmf.3 - [[phab:T233851]]
*** Next week: 1.35.0-wmf.4 - [[phab:T233852]]
=== Scoring Platform ===
* Updates:
** Jade
*** Continuing work on Jade designs. Attending design review.
[[phab:T212370]]
*** Unblocked on Jade API review. We got what we need in the short term.
(Thanks Brad & Will) [[phab:T199834]]
**** We have a weird gerrit bug. Not sure what to think of it. Looks like
it was fixed!
** New models
*** Merged changes euwiki article quality features -- rebuilding the model
next [[phab:T223788]]
*** Released revscoring 2.6.0 with support for Basque Language features
https://pypi.org/project/revscoring/2.6.0/
** Etc.
*** Fixed ordering issue in model_info. [[phab:T229445]]
** Qs for the group:
*** Any experience with Andrew NG's AI courses on coursera? Kevin (new
engineer) working through some material now.
*** Re. UI dev, we're hearing a lot of "just use OOUIJS"
=== Search Platform ===
* Blocked by:
** Analytics: [[phab:T229882]]
** Structured Data: RecentChange support for SDC: [[phab:T230862]] - patch
in CR
** Structured Data: Data dumps for SDC: [[phab:T221917]]
* Updates:
** Started work on deploying airflow for analytics workflows
[[phab:T236180]]
=== Site Reliability Engineering ===
* Blocked by:
** No team in particular anymore, multiple people responded already: SRE
DBAs needed someone. to pick this [[phab:T234450]] as it will help
mitigating some on-going heavy requests
** CPT on service-runner PR, requesting follow-up
https://github.com/wikimedia/service-runner/pull/221
** Performance: Navtiming on Prometheus [[gerrit:534771]]
* Blocking:
** Performance on PHP7 fatal errors not reported in Logstash
[[phab:T234283]]
** Release Engineering: logstash-beta.wmflabs.org does not receive any
mediawiki events [[phab:T233134]]
Hello,
As part of the Gerrit migration to a new server, some repositories can
have stalled / outdated changes. In the Gerrit UI you might find a
change suddenly open or some recent patchsets to no more be present.
Do not panic.
Some (not all) changes are stall in the past to roughly October 1st. So
for example if the affected change got merged after that date and before
the migration, it can now show as open again, or a change might no more
show patchsets made before the migration.
The reason is some metadata have not been properly transferred to the
new server. The good news is that the changes metadata are still on the
the older server and we have to sync them to the new server.
We already fixed a few trivial use cases. We are now looking at
checking every repositories and transferring the missing metadata.
Known affected repositories:
- mediawiki/extensions/MachineVision
- mediawiki/extensions/MobileFrontend
Some changes in mediawiki/core
And there are definitely more repositories affected, but we have a
script to find them all.
The task is: https://phabricator.wikimedia.org/T236114
--
Antoine "hashar" Musso