This is a summary of this week's deployment of the 1.36.0-wmf.35
branch of MediaWiki and its extensions. The primary person doing this
was Ahmon Dancy, with Brennen Bearnes as backup, both from the Release
Engineering team.
Overall this week was not entirely smooth, but not too bad. The branch
was created automatically early on Tuesday, UTC time, as usual. Ahmon
deployed it to group0, but had to roll back due to T277362. All wikis
are now on 1.36.0-wmf.35. Next week is a new train.
There were several problems found that blocked the train from
progressing to the next group:
- T277056: TypeError: Argument 1 passed to
Wikimedia\Rdbms\TransactionProfiler::recordConnection() must be of
the type string, null given
https://phabricator.wikimedia.org/T277056
- T277061: Many WikibaseLexeme tests suddenly failing in
quibble-vendor-mysql-php72-{selenium,noselenium}-docker
https://phabricator.wikimedia.org/T277061
- T277094: Bug in client error logging stops any errors from being
logged in group 0 wikis
https://phabricator.wikimedia.org/T277094
- T277161: Minify v2.2.0 incorrectly minifies vue.js (faulty line
break resulting in early return)
https://phabricator.wikimedia.org/T277161
- T277795: User not found by actor ID: [id]
https://phabricator.wikimedia.org/T277795
- an attempt at fixing was made, but the fix is stiill being
assessed
- T277302: Hatnote and ambox recognition is poor and essentially only
works for enwiki
https://phabricator.wikimedia.org/T277302
- T277362: Deprecation warning client-repo wikitext link
https://phabricator.wikimedia.org/T277362
- T277593: ErrorException: Invalid cross-wiki page Q[id]. Expected:
wikidatawiki, got local (duplicate of T277593)
https://phabricator.wikimedia.org/T277593
Several people helped to find, triage, analyze, or fix of problems
during this week. Release Engineering thanks everyone, without
help we wouldn't be able to deploy MediaWiki.
- Aaron Schulz
- Addshore
- Amir Sarabadani
- Antoine Musso
- Bartosz Dziewoński
- Daimona
- Daniel Kinzler
- Ed Sanders
- FriedhelmW
- Gergő Tisza
- James D. Forrester
- Jon Robson
- Legoktm
- Leszek Manicki
- Lucas Werkmeister
- PerfektesChaos
- RhinosF1
- Roan Kattouw
- Sam Smith
- Tarrow
- Timo Tijhof
- Tobias Andersson
- XanonymusX
There may have been other people, and if so, we apologize for not
including them on the list above.
For more information, please see:
- https://phabricator.wikimedia.org/T274939
- https://wikitech.wikimedia.org/wiki/Heterogeneous_deployment/Train_deploys
--
WMF release engineering team | he/him or they/them
"Imagine a world in which every single human being can freely share in
the sum of all knowledge."
Hello,
This extension is one of the oldest extensions we still have in production
and currently does not have a maintainer. FlaggedRevs has been the cause of
several incidents and visible regressions specially since software decays
and our environment constantly changes.
Another problem with this extension is its scope, while most of its
functionalities are not enabled in WMF production or enabled in a very
small set of wikis (e.g. multiple dimensions was enabled only in Hebrew
Wikisource but they agreed to disable it). This has made maintaining the
extension a tall order (more of a nightmare). In other words, this
extension does too many things and does none well.
To move forward, we are removing barely used functionalities of this
extension. Including support for multiple dimensions, multiple tiers,
several one of its special pages and more to make it less of a burden to
maintain and improve. It would simplify its logic drastically and will
enable us to rework its old interface, fix several deprecated code that
this extension is the last to block their removal (like action=ajax),
reduce the number of issues/incidents/regressions caused by this extension
and much more.
Users in most wikis that have this extension enabled (including English
Wikipedia and German Wikipedia) won't see a difference but for example,
users won't be able to mark a revision as "good quality" or "average
quality" anymore in English Wikibooks or Finnish Wikipedia. The
"checked/stable" status will stay though.
This is also to notify third party installations of this extension that
this will happen from release 1.36 so if you need to have some of its
features we are removing, please fork.
To see the discussion around this, take a look at
https://phabricator.wikimedia.org/T185664
Thank you for understanding and sorry for any inconvenience.
--
Amir (he/him)
The Vue migration team is pleased to announce that MediaWiki finally has
built-in support for writing modern JavaScript with ES6.
Up until now, all JavaScript code in MediaWiki has been written in ES5 (a
version of JavaScript standardized in 2009), because MediaWiki maintains
support for some browsers that don't support ES6. Over the years this has
become increasingly frustrating for developers, because ES6 (standardized
in 2015) adds new language features that make writing code easier and more
pleasant, but these features couldn't be used in MediaWiki. A lot of modern
code and tools are now written in ES6, and browsers that don't support ES6
have become obsolete. At this point, the only significant non-ES6 browser
that MediaWiki continues to support is Internet Explorer 11, whose usage is
relatively low, but not yet quite low enough for us to drop support for it.
In keeping with the IE11 announcement[1] from earlier this month, we will
continue to support IE11, but some new features will not support it.
What this means for developers is that you can now use ES6 code in
MediaWiki core, extensions, and skins, as long as it's in a feature that
doesn't need to support IE11. ResourceLoader modules that use ES6 code have
to be flagged as such, and you will need to put ES6 code in a separate
directory so that different eslint rules can be applied. For detailed
instructions on how to start using ES6 in your code, see
https://www.mediawiki.org/wiki/ResourceLoader/ES6 .
Unfortunately, ES6 code is not yet supported in on-wiki JavaScript
(Gadgets, user scripts, and site scripts), because we need to check that
these scripts are syntactically valid, and our current validator only
understands ES5 [2].
To make this possible, we added functionality[3] to ResourceLoader to allow
modules to mark themselves as ES6-only, and prevent those modules from
being loaded in browsers that don't support ES6. We also made significant
changes[4] to our JavaScript minifier[5] to support the minification of ES6
code, which previously generated invalid output[6] when confronted with ES6
syntax. And we updated our eslint ruleset[7] to add a linter configuration
for ES6 code[8]. Thank you to Timo Tijhof (Krinkle) and DannyS712 for code
reviewing the ResourceLoader and minifier changes, to Lucas Werkmeister for
finding a critical bug[9] in the minifier change, and to James Forrester
and Ed Sanders for code reviewing the linter changes and releasing new
versions of all these packages.
Roan Kattouw
On behalf of the Vue migration team (Anne Tomasevich, Eric Gardner, Volker
Eckl, and myself)
[1] https://www.mediawiki.org/wiki/Compatibility/IE11
[2] https://phabricator.wikimedia.org/T75714
[3] https://gerrit.wikimedia.org/r/c/mediawiki/core/+/657953
[4] https://gerrit.wikimedia.org/r/c/mediawiki/libs/Minify/+/664700
[5] https://github.com/wikimedia/Minify/
[6] https://phabricator.wikimedia.org/T255556
[7] https://github.com/wikimedia/eslint-config-wikimedia
[8] https://github.com/wikimedia/eslint-config-wikimedia/pull/358
[9] https://phabricator.wikimedia.org/T277161
https://www.mediawiki.org/wiki/Scrum_of_scrums/2021-03-17#WMDE_Technical_Wi…
= 2021-03-17 =
== Callouts ==
* SRE ServiceOps upgrade of the k8s codfw kubernetes cluster completed.
Traffic is being repooled, do reach out to us on T277191 if issues are
observed. k8s eqiad cluster to probably follow next week.
* SRE ServiceOps blocked on
**Product Infrastructure on T274262
**Analytics on T274262
**Thanks to Product Infrastructure and PET for acting on the above task
last week
* RelEng: Developer satisfaction survey is live until 2021-03-24:
https://forms.gle/xj2jNUcP7kmSgtwV8
* RelEng: When to use train vs backports quick guide:
https://wikitech.wikimedia.org/wiki/Deployments/Train_vs_backport
== Gerrit patches or GitHub Pull Requests for reviews or feedback ==
*
=== No updates ===
CommTech, AHT, Library
=== '''No notes provided''' ===
Editing, Growth, Parsing, Language, Inuka, Cloud Services, Performance,
Quality & Test, Search Platform, Security,
== SoS Meeting Bookkeeping ==
* Updates:
== Product ==
=== iOS native app ===
* Blocked by: N/A
* Blocking: N/A
* Thank yous:
* Updates: Setting scope for next version.
=== Android native app ===
* Blocked by: N/A
* Blocking: N/A
* Thank yous:
* Updates: New version out end of last week. Small updates, we're now
releasing every 2 weeks.
=== Web ===
* Blocked by:
* Blocking:
* Thank yous:
** Michael Holloway of Product Infrastructure for diagnosing and fixing
Readers Web's second (!!) duplicate EventLogging event bug, this time
affecting the PrefUpdate schema: https://phabricator.wikimedia.org/T218835
* Updates:
** Still working on bolstering the UniversalLanguageSelector
instrumentation: https://phabricator.wikimedia.org/T268504 and its subtasks
** … and the new Desktop Improvements language switcher treatment:
https://phabricator.wikimedia.org/T256023
** Investigating how to deliver the language switcher A/B test to everyone
efficiently: https://phabricator.wikimedia.org/T275807
** Deployed and then reverted an update to MobileFrontend's lead paragraph
hoisting feature: https://phabricator.wikimedia.org/T262093 and
https://phabricator.wikimedia.org/T277367
=== Product Infrastructure ===
* Blocked by:
* Blocking:
** SRE Service Ops on T274262
* Thank yous:
** From SRE Service Ops for working on chromium-render on T274262
* Updates:
=== Structured Data ===
* Blocked by:
* Blocking:
* Thank yous:
* Updates:
** Wrapping up tasks needed before we make MediaSearch the default search
UI on commons, including https://phabricator.wikimedia.org/T273879, which
will switch the main search input to point to Special:MediaSearch (there
will be a preference to switch this back to Special:Search available)
** Continuing to work on improved search profile (
https://phabricator.wikimedia.org/T271799)
** Continued discussions across teams regarding Structured Data Across
Wikimedia (image recommendations API and structured content)
** Vue stuff: as part of our MediaSearch work (currently in
WikibaseMediaInfo), we've added reusable Checkbox, Radio, and [in-progress]
Dialog components
=== Abstract Wikipedia ===
* Blocked by:
** None?
* Blocking:
** None?
* Thank yous:
* Updates:
** Continuing on phase gamma:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Phases
** The Wikifunctions logo concept vote has now closed; working with Legal
and Comms on next steps:
https://meta.wikimedia.org/wiki/Abstract_Wikipedia/Wikifunctions_logo_conce…
=== Vue.js ===
* Blocked by:
* Blocking:
* Thank yous:
** Everyone who helped push through the changes to support ES6 in
MediaWiki: Timo Tijhof (Krinkle) and DannyS712 for code reviewing the
ResourceLoader and minifier changes, Lucas Werkmeister for finding a
critical bug in the minifier change, and James Forrester and Ed Sanders for
code reviewing the linter changes and releasing new versions of all these
packages.
* Updates:
** MediaWiki now has built-in support for writing modern JavaScript (ES6)!
*** You can now use ES6 code in MediaWiki core, extensions, and skins, as
long as it's in a feature that doesn't need to support IE11. ResourceLoader
modules that use ES6 code have to be flagged as such, and you will need to
put ES6 code in a separate directory so that different eslint rules can be
applied. For detailed instructions on how to start using ES6 in your code,
see https://www.mediawiki.org/wiki/ResourceLoader/ES6 .
*** See wikitech-l for the full announacement
== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
** SRE Service Ops on T274262
* Thank yous:
* Updates:
=== Fundraising Tech ===
* Blocked by:
* Blocking:
* Thank yous:
* Updates:
** More work on email prefs page https://phabricator.wikimedia.org/T268510
** Enhancements to optional employer autocomplete field (used for matching
gifts)
** Lots of minor Civi fixes https://phabricator.wikimedia.org/T275442,
https://phabricator.wikimedia.org/T275445,
https://phabricator.wikimedia.org/T277338,
https://phabricator.wikimedia.org/T277247
** Investigating software libraries that handle multiple payment providers
https://phabricator.wikimedia.org/T277246,
https://phabricator.wikimedia.org/T277245
** Better error handling for CentralNotice invalid banner name:
https://phabricator.wikimedia.org/T173782
=== Platform ===
* Blocked by: None
* Blocking:
* Thank yous:
** From SRE Service Ops for working on T274262
* Updates:
=== Engineering Productivity ===
==== Release Engineering ====
* Blocked by:
**
* Blocking:
**
* Thank yous:
** Moritz & John for assistance with GitLab / CAS
** Alex ServiceOps: Kudos on codfw k8s upgrade!
* Updates:
** Deployments/Covid-19
https://wikitech.wikimedia.org/wiki/Deployments/Covid-19
** Train Health
*** Last week: 1.36.0-wmf.34 [[phab:T274938]] <!--
https://phabricator.wikimedia.org/T274938 -->
*** This week: 1.36.0-wmf.35 [[phab:T274939]] <!--
https://phabricator.wikimedia.org/T274939 -->
*** Last week: 1.36.0-wmf.36 [[phab:T274940]] <!--
https://phabricator.wikimedia.org/T274940-->
=== Site Reliability Engineering ===
* Blocked by:
** Analytics on T274262
** Product infrastructure on T274262
* Blocking:
* Thank yous:
* Updates:
* SRE ServiceOps upgrade of the k8s codfw kubernetes cluster completed.
Traffic is being repooled, do reach out to us on T277191 if issues are
observed. k8s eqiad cluster to probably follow next week.
=== WMDE Technical Wishes ===
* Blocked by:
* Blocking:
* Thank yous:
* Updates:
* Due to user feedback, we will change the settings of the Popups
extension so that reference previews and page previews can be enabled
separately. Today’s reference previews release will not happen and is being
rescheduled.
* Making simplifications to the reportupdater script, to support Hive
queries natively, plus minor streamlining of the format and config.
https://gerrit.wikimedia.org/r/c/analytics/reportupdater/+/667192
== Cross-cutting ==
* Blocked by:
** [long term] Search Platform: PHP 8.0 work is long-term blocked on the
migration to ElasticSearch 7.0 https://phabricator.wikimedia.org/T263142
* Blocking:
** None?
* Thank yous:
** Roan and Kunal for driving the support for ES6 and ES5 rules in our
eslint linting (now rolled out).
** Adam Wight for working to modernise Wikidata's data-values/value-view
linting tools.
* Updates:
** PHP 8.0 work is focussed on helping upstream provide forwards and
backwards compatibility in Elastica-related PHP code.
** CI tools' upgrade status is a little behind:
https://libraryupgrader2.wmcloud.org/status?branch=master
Hello everyone,
TL;DR if you are not deploying services to the codfw kubernetes
cluster, you can safely skip this.
Long version:
After having tested twice our cluster reinitialization procedure, this
week we will be reinitializing our codfw kubernetes cluster. All
traffic will be drained from it beforehand and we expect no user
visible impact. However, for the duration of the process, the
kubernetes codfw cluster will be unavailable to deployers and thus
efforts to deploy to it will fail or worse, not have the expected
outcomes. This is normal until SRE serviceops announces that the
cluster is fully operational again.
SRE service-ops will be deploying all services before marking the
cluster as usable and pooling traffic back to it, so there will be no
need for deployers to re-deploy their services.
For your convenience the list of services that are currently deployed
on that cluster is: apertium api-gateway blubberoid changeprop
changeprop-jobqueue citoid cxserver echostore eventgate-analytics
eventgate-analytics-external eventgate-logging-external eventgate-main
eventstreams eventstreams-internal linkrecommendation mathoid
mobileapps proton push-notifications recommendation-api sessionstore
similar-users termbox wikifeeds zotero
Regards,
--
Alexandros Kosiaris
Principal Site Reliability Engineer
Wikimedia Foundation
The 1.36.0-wmf.35 train was briefly rolled out to group0 but was rolled
back due to an increase in logged errors attributed to T277362[0]. All
wikis are back to 1.36.0-wmf.34.
See the train blockers task[1] for all details about this week's train.
Thanks in advance for resolution of this issue! Sincerely, your train
operator.
[0] https://phabricator.wikimedia.org/T277362
[1] https://phabricator.wikimedia.org/T274939
Dear Wikitech/Wikitech-Ambassadors,
Over the last few months, a small team at the Wikimedia Foundation has been
working on a project that has been discussed by many people in our movement
for many years: building ‘enterprise grade’ services for the high-volume
commercial reusers of Wikimedia content. I am pleased to say that in a
remarkably short amount of time (considering the complexity of the issues:
technical, strategic, legal, and financial) we now have something worthy of
showing to the community, and we are asking for your feedback. Allow me to
introduce you to the Wikimedia Enterprise API project – formerly codenamed
“okapi”.
While the general idea for Wikimedia Enterprise predates the current
movement strategy process, its recommendations [0] identify an enterprise
API as one possible solution to both “Increase the sustainability of our
movement” and “Improve User Experience.” That is, to simultaneously create
a new revenue stream to protect Wikimedia’s sustainability, and improve the
quality and quantity of Wikimedia content available to our many readers who
do not visit our websites directly (including more consistent attribution).
Moreover, it does so in a way that is true to our movement’s culture: with
open source software, financial transparency, non-exclusive contracts or
content, no restrictions on existing services, and free access for
Wikimedia volunteers who need it.
The team believes we are on target to achieve those goals and so we have
written a lot of documentation you get your feedback about our progress and
where it could be further improved before the actual product is ‘launched’
in the next few months. We have been helped in this process over the last
several months by the approximately 100 individual volunteers (from many
corners of the wikiverse) and representatives of affiliate organisations
who have reviewed our plans and provided invaluable direction. Pointing out
weaknesses and opportunities, or areas lacking clarity and documentation in
our drafts.
A essay describing the “why?” and the “how?” of this project is now on
Meta:
https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/Essay
Also now published on Meta are an extensive FAQ [1] and operating
principles [2]. Much of this documentation is already available in French,
German, Italian, and Spanish.
For technical specifics, the new documentation on MediaWiki.org describes
the product strategy and roadmap, work currently underway, design
documents, as well as infrastructure decisions and future plans:
https://www.mediawiki.org/wiki/Wikimedia_Enterprise
The team is particularly interested in your feedback on how we have
designed the checks and balances to this project - to ensure it is as
successful as possible at achieving those two goals described above while
staying true to the movement’s values and culture. For example: Is
everything covered appropriately in the “Principles” list? Is the technical
documentation on MediaWiki.org clear? Are the explanations in the “FAQ”
about free-access for community, or project’s legal structure, or the
financial transparency (etc.) sufficiently detailed?
Meet the team and Ask Us Anything:
The central place to provide written feedback about the project in general
is on the talkpage of the documentation on Meta at:
https://meta.wikimedia.org/wiki/Talk:Wikimedia_Enterprise
Equally, for technical comments or questions the talkpage on MediaWiki.org:
https://www.mediawiki.org/wiki/Talk:Wikimedia_Enterprise
<https://meta.wikimedia.org/wiki/Talk:Wikimedia_Enterprise>
On this Friday (March 19) we will be hosting two “Office hours”
conversations where anyone can come and give feedback or ask questions:
- 13:00 UTC via Zoom at https://wikimedia.zoom.us/j/95580273732
- 22:00 UTC via Zoom at https://wikimedia.zoom.us/j/92565175760 (note: this
is Saturday in Asia/Oceania)
Other “office hours” meetings can be arranged on-request on a technical
platform of your choosing; and we will organise more calls in the future.
We will also be attending the next SWAN meetings (on March 21)
https://meta.wikimedia.org/wiki/Strategic_Wikimedia_Affiliates_Network, and
also the next of the Wikimedia Clinics
https://meta.wikimedia.org/wiki/Wikimedia_Clinics
Moreover, we would be very happy to accept any invitation to attend an
existing group call that would like to discuss this topic (e.g. an
affiliate’s members’ meeting).
On behalf of the Wikimedia Enterprise team,
*Liam Wyatt [Wittylama]*
WikiCite <https://meta.wikimedia.org/wiki/WikiCite> Program Manager & Wikimedia
Enterprise <https://meta.wikimedia.org/wiki/Okapi> Community Liaison
Wikimedia Foundation
[0]
https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2018-20/Recomme…
[1] https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/FAQ
[2] https://meta.wikimedia.org/wiki/Wikimedia_Enterprise/Principles
Dear members of Wikimedia,
We are an international team of researchers, from the University of Bari -
Italy, Federal University of Pernambuco - Brazil, University of Adelaide -
Australia, and Eindhoven University of Technology - the Netherlands. The
goal of our research is to improve software documentation and
maintainability of open-source projects by providing automatic support of
source code comments based on code review discussions. We would like to
learn identifying relevant information from code review discussions such
that tooling can be developed to improve software documentation and
maintainability of open-source projects. Therefore, we would like to talk
to open-source developers and interview them.
If you are interested in talking about your experiences as an open-source
reviewer then please schedule a meeting with us using YouCanBook.me here: [
https://tue-interview.youcanbook.me]. We are interested in different
experiences, therefore, we are looking to talk to people of all age-groups,
genders, and people with different roles in open-source projects. So even
whether you are a first-time contributor or a senior maintainer don’t
hesitate to contact us.
The interview itself will take no longer than 30 min and will be scheduled
using Microsoft Teams. We will record the interview and the recording and
transcript of the interview will be stored securely, and only made
available to the researchers that are involved in this project.
Additionally, we plan to use anonymized results of the interview in a
scientific publication.
Kind regards and thanks in advance,
--
Nicole Novielli, University of Bari,
Fernando Castor, Federal University of Pernambuco,
Christoph Treude, University of Adelaide,
Alexander Serebrenik, Eindhoven University of Technology,
Felipe Ebert, Eindhoven University of Technology
I'm not sure because I didn't understand English exactly, but if I need to log in to register, I can't do it now because I'm globally locked. I also contacted Steward, but I haven't heard back yet. In any case, thank you for your guidance.
Rightandlight