Hi all,
We just launched [[WM:Techblog]] <https://techblog.wikimedia.org/>, a new
place to share stories from the Wikimedia technical community with the
world. Yay!
We created this venue to provide a central place for people to share
stories about the technical work that they do— like:
-
How to run a top ten website that is all Open Source?
-
How do we make our codebase more modular and future-proof?
-
How do we come together to work on technical projects?
-
How to develop a product for many different languages?
-
How do new tools and bots help run Wikimedia projects?
-
What can we learn from data science and research?
-
And, much more…
Check out these posts for examples:
Computational knowledge: Wikidata, Wikidata query Service, and women who
are mayors!
<https://techblog.wikimedia.org/2020/03/24/computational-knowledge-wikidata-…>
By Trey Jones
Parsoid in PHP, or There and Back Again
<https://techblog.wikimedia.org/2020/03/24/parsoid-in-php-or-there-and-back-…>,
By S.Subramanya Sastry and C.Scott Ananian
Wikimedia projects have many intersections with the larger Open Source
community. It’s our hope that the technical blog will create more
visibility and conversations and provide information to a wider audience of
people who are interested in our work.
If you are interested in writing a blog post, read the editorial guidelines
<https://www.mediawiki.org/wiki/Wikimedia_technical_blog_editorial_guidelines>
to learn about what would make a good story and how to get published! We
manage the process of reviewing and publishing posts here:
https://phabricator.wikimedia.org/tag/Technical-blog-posts
Thanks to everyone who contributed to making the blog possible. There were
many!
For the WM Techblog team,
Sarah R. Rodlund
Technical Writer, Developer Advocacy
<https://meta.wikimedia.org/wiki/Developer_Advocacy>
srodlund(a)wikimedia.org
Hello Everyone,
It's time for Wikimedia Tech Talks 2020 Episode 2! This talk will take
place on *25 March 2020 at 6 PM UTC*.
*Title: *Understanding Wikimedia Maps and its challenges
*Speaker:* Mateus Santos, Software Engineer
*Summary: *The WMF Product Infrastructure Team has been maintaining the
Wikimedia Maps service for the last year and a half with help from SRE.
This talk will share the challenges and work of creating a better
development environment to enhance productivity, solve technical debt and
keep up with platform modernization.
The link to the Youtube Livestream can be found here:
https://www.youtube.com/watch?v=2LH-pYdi9Ks
During the live talk, you are invited to join the discussion on IRC at
#wikimedia-office
You can watch past Tech Talks here:
https://www.mediawiki.org/wiki/Tech_talks
If you are interested in giving your own tech talk, you can learn more
here:
https://www.mediawiki.org/wiki/Project:Calendar/How_to_schedule_an_event#Te…
Note: This is a public talk. Feel free to distribute through appropriate
email and social channels!
Kindly,
Sarah R. Rodlund
Technical Writer, Developer Advocacy
<https://meta.wikimedia.org/wiki/Developer_Advocacy>
srodlund(a)wikimedia.org
Due to the uncertainty resulting from the COVID-19 pandemic, we have
decided to temporarily push back the release of MediaWiki 1.35. Especially
since MediaWiki 1.35 is a long term support (LTS) release, we want to take
the time to give the release appropriate resources and attention. We will
not be cutting the REL1_35 branch on April 7 as we had previously
announced. We will be making another announcement no later than Friday, May
29 giving an update on when we will be cutting the REL1_35 branch and
moving towards the release of MediaWiki 1.35. We appreciate your patience
in these difficult times.
Wishing you health,
Cindy
______________________________
Cindy Cicalese
Product Manager, MediaWiki Platform
Wikimedia Foundation
Hi,
In the german Technical Wishes survey 2019, the subject area "Working more
easily with templates" was chosen as the main focus. This means that the
Technical Wishes team of Wikimedia Deutschland will be working on this
topic for two years and tackle various problems in it.
Since then, research into which improvements could be implemented has been
conducted. Over the coming weeks, a series of ideas for possible projects
will be published regularly on the project page. Everyone is invited to
evaluate and comment on them. For a balanced picture, feedback is sought
from Wikimedians with different levels of experience, working methods and
fields of work. Based on this feedback, the Technical Wishes team will
consider whether to implement the ideas, revise them, or discard them in
favor of others. Depending on the outcome of this prototyping phase, there
might be another round of new ideas to provide feedback on later in the
year.
Important: Since we want to present new prototypes regularly in the coming
weeks and we don't want to flood users' inboxes, we ask all interested
people to watch the project page.
Cheers,
Michael for the team technical Wishes @ Wikimedia Deutschland
--
*M. F. Schönitzer*
Community Communication
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
https://wikimedia.de
Unsere Vision ist eine Welt, in der alle Menschen am Wissen der Menschheit
teilhaben, es nutzen und mehren können. Helfen Sie uns dabei!
https://spenden.wikimedia.de
Imagine a world in which every single human being can freely share in the
sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de
Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
Hi,
for HTML version see
https://www.mediawiki.org/wiki/Scrum_of_scrums/2020-03-25
Željko
--
= 2020-03-25 =
== Callouts ==
* Release Engineering
** [All] MediaWiki 1.35.0 will get cut on 7 April 2020. If your team has
any proposed blockers/deadlines for that, please get them done:
[[phab:tag/MW-1.35-release]]
** [All] Review guidance at [[wikitech:Deployments/Covid-19]] and Code
Deployment Office Hour at 17:00UTC in #wikimedia-office
== Product ==
=== iOS native app ===
* Updates:
** Bug fixes & testing on 6.6 release (mobile-html integration)
[[phab:project/view/4273]]
***Releasing to public beta in the next day or two
=== Android native app ===
* Updates:
** Released Suggested Edits v4 to Beta.
** Released integration with mobile-html to Beta.
** Gathering feedback and rounding out any remaining issues before
releasing to production (likely next week).
=== Web ===
* Updates:
** Summary: evolving plans for Coronavirus, continuing desktop improvements
project (DIP).
** [[Reading/Web/Desktop_Improvements|Desktop Improvements Project (Vector
/ DIP)]]:
*** [[phab:T248010|<nowiki>Vector sidebar missing on test.wikipedia.org and
weird footer</nowiki>]]
*** [[phab:T247566|<nowiki>Broken section edit links styles on
Vector</nowiki>]]
*** [[phab:T247890|<nowiki>Default to feature branch in Vector</nowiki>]]
*** [[phab:T245456|<nowiki>[Dev] Adopt template partials in Vector and
revise sidebar component</nowiki>]]
*** [[phab:T243281|<nowiki>Build opt-out link for logged-in users with new
vector on</nowiki>]]
*** [[phab:T113095|<nowiki>A cached server-side HTML template should update
when you change a partial template which it includes</nowiki>]]
*** [[phab:T247537|<nowiki>Regression: Border color of Vector action menu
is black</nowiki>]]
*** Hureo report
** Mobile website (MinervaNeue / MobileFrontend):
*** [[phab:T248360|<nowiki>Phan warning in
"includes/specials/SpecialMobileDiff.php" - working around by
disabling</nowiki>]]
*** [[phab:T248347|<nowiki>Allow customisations of
Article::getSubstituteContent for different namespaces</nowiki>]]
*** [[phab:T248141|<nowiki>Regression: MobileFrontend overrides
non-existent user page handling in desktop view</nowiki>]]
*** [[phab:T247274|<nowiki>Uncaught ReferenceError: mfTempOpenSection is
not defined on main pages</nowiki>]]
*** [[phab:T240622|<nowiki>[Technical debt payoff] Remove
InlineDiffFormatter and InlineDifferenceEngine from
MobileFrontend</nowiki>]]
*** [[phab:T126825|<nowiki>Sections on mobile views jumps around after load
(FOUC)</nowiki>]]
*** [[phab:T237230|<nowiki>Long edit summary/titles/user breaking AMC
special pages</nowiki>]]
** Miscellaneous
*** [[phab:T248133|<nowiki>WebInstallerOutput should more elegantly load
Vector stylesheet rather than hardcoding a reference</nowiki>]]
** Q4 planning
== Technology ==
=== Fundraising Tech ===
* Updates:
** About to test recurring iDEAL (NL bank transfer) via our backup payment
processor: [[phab:T230837]]
** Testing migration of recurring card donations from main payment
processor's old API to their new API: [[phab:T232017]]
** Fixing localization of error messages on payments-wiki in certain
circumstances: [[phab:T247885]]
** About ready to merge sub-national targeting to CentralNotice master
** Reviewing CentralNotice clone campaign functionality
** Fixing matching gift CiviCRM import: [[phab:T246889]]
=== Core Platform ===
* Blocking:
** Wikimedia DE/Wikidata: We'd appreciate some input on [[phab:T245535]],
and feedback on [[gerrit:580870]]. We got some comments from Krinkle
(thanks), but would appreciate some more advice there.
** Search Platform: MW Job consumers sometimes pause for several minutes
[[phab:T224425]]
* Updates:
** Finishing minimal client epic for Core REST API [[phab:T229662]]
** Adding config flag for development endpoints in Core REST API
[[phab:T247997]]
=== Engineering Productivity ===
==== Release Engineering ====
* Updates:
** [All] Deployments/Covid-19 [[wikitech:Deployments/Covid-19]]
** [All] MediaWiki 1.35.0 will get cut on 7 April 2020. If your team has
any proposed blockers/deadlines for that, please get them done:
[[phab:tag/MW-1.35-release]]
** Train Health
*** This week: 1.35.0-wmf.25 - [[phab:T233873]]
*** Next week: 1.35.0-wmf.26 - [[phab:T233874]]
=== Search Platform ===
* Blocked by:
** Core: MW Job consumers sometimes pause for several minutes
[[phab:T224425]]
== Wikimedia DE ==
=== Wikidata ===
* Blocked by:
** Not sure which team - Core Platform?: We'd appreciate some input on
[[phab:T245535]], and feedback on [[gerrit:580870]]. We got some comments
from Krinkle (thanks), but would appreciate some more advice there.
Hi all,
Tomorrow we will be issuing a security and maintenance release to all
supported branches of MediaWiki.
The new releases will be:
- 1.34.1
- 1.33.3
- 1.31.7
This will resolve two minor issues in MediaWiki core, and also includes
some fixes previously committed to git, including minor security and
hardening patches along with bug fixes included for maintenance reasons.
We've noted that these issues are minor, and as such you don't need to
apply them as quickly as with other security releases, if you're unable to
do so. We therefore decided to continue with getting the security (and
maintenance) release out for this quarter as planned, even with the global
situation as is.
We will make the fixes available in these respective release branches, and
also master. Tarballs will be available for the above mentioned point
releases as well.
A summary of some of the security fixes that have gone into non bundled
MediaWiki extensions will also follow.
Thanks!
Sam
📘 Read on Phabricator at
https://phabricator.wikimedia.org/phame/post/view/192/
-------
How’d we do in our strive for operational excellence last month? Read on to
find out!
## 📊 Month in numbers
* 8 documented incidents. [1]
* 27 new Wikimedia-prod-error reports. [2]
* 26 Wikimedia-prod-error reports closed. [3]
* 199 currently open Wikimedia-prod-error reports in total. [4]
With a median of 4–5 documented incidents per month (over the last three
years), there were a fairly large number of them this past month.
To read more about these incidents and pending actionables; check <
https://wikitech.wikimedia.org/wiki/Incident_documentation#2020>, or
Explore Wikimedia incident stats at <https://codepen.io/Krinkle/full/wbYMZK>
(interactive).
-------
## 📖 Unset vs array splice
Our error monitor (Logstash) received numerous reports about an “Undefined
offset” error from the OATHAuth extension. This extension powers the
Two-factor auth (2FA) login interface on Wikipedia (<
https://meta.wikimedia.org/wiki/Help:Two-factor_authentication>)
ItSpiderman and Reedy investigated the problem. The error message:
PHP Notice: Undefined offset: 8
at /srv/mediawiki/extensions/OATHAuth/src/Key/TOTPKey.php:188
This error means that the code was accessing item number 8 from a list (an
array), but the item does not exist. Normally, when a “2FA scratch token”
is used, we remove it from a list, and save the remaining list for next
time.
The code used the `count()` function to compute the length of the list, and
used a for-loop to iterate through the list. When the code found the user’s
token, it used the `unset( $list[$num] )` operation to remove token $num
from the list, and then save $list for next time.
The problem with removing a list item in this way is that it leaves a
“gap”. Imagine a list with 4 items, like [ 1: …, 2: …, 3: … , 4: … ]. If we
unset item 2, then the remaining list will be [ 1: …, 3: …, 4: … ]. The
next time we check this list, the length of the list is now 3 (so far so
good!), but the for-loop will access the items as 1-2-3. The code would not
know that 3 comes after 1, causing an error because item 2 does not exist.
And, the code would not even look at item 4!
When a user used their first ever scratch token, everything worked fine.
But from their second token onwards, the tokens could be rejected as
“wrong” because the code was not able to find them.
To avoid this bug, we changed the code to use `array_splice( $list, $num, 1
)` instead of `unset( $list[$num] )`. The important thing about
array_splice is that it renumbers the items in the list, leaving no gaps.
– https://phabricator.wikimedia.org/T244308 /
https://gerrit.wikimedia.org/r/570253
-------
## 📉 Outstanding reports
Take a look at the workboard and look for tasks that might need your help.
The workboard lists error reports, grouped by the month in which they were
first observed.
→ https://phabricator.wikimedia.org/tag/wikimedia-production-error/
Breakdown of recent months:
* March: 3 of 10 reports left (unchanged). ⚠️
* April: 4 of 14 left (unchanged).
* May: (All clear!)
* June: 4 of 11 left (unchanged).
* July: 8 of 18 left (unchanged).
* August: Two reports closed! 2 of 14 reports left.
* September: One report closed, 7 of 12 left.
* October: Two reports closed, 6 of 12 left.
* November: 5 of 5 left (unchanged).
* December: 6 of 9 left (unchanged).
* January: One report closed, 6 of 7 reports left.
* February: 7 new reports survived the month of February.
Last month’s total over recent months was 57 open reports. Of those, 6 got
closed, but with 7 new reports from February still open, the total is now
up at 58 open reports.
-------
## 🎉 Thanks!
Thank you to everyone who helped by reporting, investigating, or resolving
problems in Wikimedia production.
Together we’re getting there!
Until next time,
– Timo Tijhof
Footnotes:
[1] Incidents. –
https://wikitech.wikimedia.org/wiki/Incident_documentation#2020
[2] Tasks created. –
https://phabricator.wikimedia.org/maniphest/query/aT3iqdM0EJKW/#R
[3] Tasks closed. –
https://phabricator.wikimedia.org/maniphest/query/jVexIrtOPkcX/#R
[4] Open tasks. –
https://phabricator.wikimedia.org/maniphest/query/47MGY8BUDvRD/#R
Hi,
for HTML version see
https://www.mediawiki.org/wiki/Scrum_of_scrums/2020-03-18
This was the last in-person Scrum of scrums meeting. Going forward we will
just update the etherpad.
Željko
--
= 2020-03-18 =
== Callouts ==
* Release Engineering
** Deployments/Covid-19 [[wikitech:Deployments/Covid-19]]
** [All] MediaWiki 1.35.0 will get cut on 7 April 2020. If your team has
any proposed blockers/deadlines for that, please get them done:
[[phab:tag/MW-1.35-release]]
* Structured Data needs review from Security on proposed new data dump for
MachineVision: [[phab:T236431#5900726]]
== SoS Meeting Bookkeeping ==
* Updates:
** This was the last in-person SoS meeting. Going forward we will just
update the etherpad.
== Product ==
=== Editing ===
* Updates:
** No Updates, we are shifting or focus but hoping to turn on Replying 1.0
on Beta
=== iOS native app ===
* Updates:
** Wrapping up development on 6.6 release (mobile-html integration)...Beta
soon[[phab:project/view/4273]]
=== Android native app ===
* Updates:
** Release of mobile_html ready for Beta, pending final testing.
** Release of Suggested Edits v4 (image tagging) pending user testing /
polishing updates.
=== Web ===
* Updates:
** Working on Logo and Header for Desktop Refresh
=== Structured Data ===
* Blocked by:
** Security on review of proposed new data dump: [[phab:T236431#5900726]]
* Updates:
** will miss the meeting because of a clash - back next week
** units for quantities, improved geo-coordinates, date, and monolingual
text input types all live
** constraints violations code merged should be live when train rolls
** some CAT improvements on the way
== Technology ==
=== Fundraising Tech ===
* Updates:
** Deployed NL bank transfer capability for backup processor, working on
making it possible to accept recurring bank donations there
** Investigating new ways to manage PayPal recurring donations
** Testing migrating recurring donations from main card processor's old API
integration to their new integration
** Getting started making an autocomplete dropdown of employers offering
matching gifts
=== Core Platform ===
* Blocking:
** Wikimedia DE/Wikidata: would appreciate someone advising on
[[phab:T225814]].
** Search Platform: MW Job consumers sometimes pause for several minutes
[[phab:T224425]]
* Updates:
** Continued work on jobrunner, changeprops
** Core REST API next version coming soon
** API Gateway
** Developer Portal prototype
[[Core_Platform_Team/Initiatives/API_Gateway/Documentation_Plan]]
** Process of adopting or co-parenting CentralNotice
=== Engineering Productivity ===
==== Release Engineering ====
* Blocking:
** Wikimedia DE/Wikidata: would appreciate someone chiming in on
[[phab:T245826]]
* Updates:
** [All] MediaWiki 1.35.0 will get cut on 7 April 2020. If your team has
any proposed blockers/deadlines for that, please get them done:
[[phab:tag/MW-1.35-release]]
** Train Health
*** This week: 1.35.0-wmf.24 - [[phab:T233872]]
*** Next week: 1.35.0-wmf.25 - [[phab:T233873]]
=== Scoring Platform ===
* Updates:
** Pile of volunteers: chtnnh, haksoat, clemons, nikhil, (+2 more on the
way to IRC/phab)
** Jade
*** Did first round of user-testing on Beta.
*** We got the Jade Diff view to work.
https://deployment.wikimedia.beta.wmflabs.org/wiki/Jade:Diff/4
**** Considered native OOUI element for Diff. Needs PHP. Crusty.
**** Struggling to get date format strings to the client side
** Released revscoring 2.6.9 which includes section-based features (e.g.
text complexity measures)
** Implemented improved feature extraction for idioms and images (re.
articlequality)
=== Search Platform ===
* Blocked by:
** Core: MW Job consumers sometimes pause for several minutes
[[phab:T224425]]
* Updates:
** Copy English Wikipedia drafttopic scores to other wikis somewhere in the
CirrusSearch pipeline [[phab:T241015]]
** Once the ORES articletopic - ElasticSearch pipeline is set up, update
data about all articles [[phab:T243357]]
=== Security ===
* Blocking:
** Structured Data: review of proposed new data dump:
[[phab:T236431#5900726]]
=== Site Reliability Engineering ===
* Blocking:
** Product Infrastructure on creations of k8s namespaces/tokens for proton,
mobileapps. Working on it
** Research on creation of k8s namespaces/tokens for recommendation-api.
Working on it.
== Wikimedia DE ==
=== Wikidata ===
* Blocked by:
** Release Engineering: We would appreciate someone chiming in on
[[phab:T245826]]
** Not sure who - Core Platform: We would appreciate someone advising on
[[phab:T225814]].
* Updates:
** wb_terms table (the old term store) is not being read or updated
anymore. [[phab:T208425]]