Hello,
The 1.36.0-wmf.29 update had been blocked on a caching issue apparently
caused by the FeaturedFeeds extension:
* MemcachedPeclBagOStuff: Serialization of 'Closure' is not allowed
- https://phabricator.wikimedia.org/T273242
The fix got reviewed and backported today and I will thus promote
1.36.0-wmf.29 to group 0 and immediately after to group 1 starting at
14:00 UTC today (15:00 CET).
If all goes well, I guess I can try to promote all the remaining wiki as
well or we might do it later today (at the usual 20:00 UTC).
--
Antoine "hashar" Musso
Wikimedia Release Engineering
Hi all!
tl;dr: There's a large backlog of production errors. Release Engineering is
blocking the train for any new logspam. Your help is needed!
A quick update on the deployment train:
In the process of rolling out wmf/1.36.0-wmf.28 there were a number of
issues that prevented us from rolling forward the train in a timely manner.
After the issues were resolved and backports deployed to the current
version in production (wmf/1.36.0-wmf.27), we realized there were a few
remaining spammy log messages and blocked the following week's train on
those issues.
Release Engineering has long blocked the train on logspam issues[0]. Even
when it does not indicate user-facing errors, logspam of any kind makes it
harder for us to see real problems. We have, however, defaulted to pushing
forward the train despite minor issues.
Under this custom, many log messages have been accepted as "just
occassional, not a big deal" or "yeah, we'll fix that eventually... it's
not a big deal". Frequently, "eventually" never arrives. This results in an
unmanageable accumulation of exceptions (see the ever-growing list of
exceptions in the Wikimedia-production-error workboard[1] and logstash[2]).
To deal with these issues we are now, as a matter of policy, blocking
trains that cause any new error messages. In most cases new errors are the
result of code changes that lack defensive coding practices and/or have
unexpected interactions with other code. The best resolution in these cases
is for the code to be fixed or reverted.
Release Engineering organises a weekly "train log triage" meeting, on
Wednesdays at 19:00 UTC, where we invite people who develop MediaWiki to
help triage log messages. As of this week, there is also a second one, on
Thursdays at 10:00 UTC, to be more suitable for people in EU time zones. We
invite everyone who develops MediaWiki or its extensions to join one of the
meetings each week.
Thank you,
Greg
[0]: <
https://wikitech.wikimedia.org/wiki/Deployments/Holding_the_train#Logspam>
[1]: https://phabricator.wikimedia.org/project/view/1055/
[2]:
https://logstash.wikimedia.org/app/dashboards#/view/0a9ecdc0-b6dc-11e8-9d8f…
--
| Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E |
| Dir. Engineering Productivity A18D 1138 8E47 FAC8 1C7D |
Hi all,
After considerable effort this week, we're down to one blocker for the
1.36.0-wmf.29[0] train:
* MemcachedPeclBagOStuff: Serialization of 'Closure' is not allowed
- https://phabricator.wikimedia.org/T273242
Thanks in advance for any help resolving this issue! Once resolved, the
train can resume Monday (European) morning.
-- Brennen, on behalf of your faithful train crew
[0]. <https://phabricator.wikimedia.org/T271343>
Dear all,
The Wikimedia Foundation Board of Trustees is organizing a call for
feedback about community selection processes between February 1 and March
14. Below you will find the problem statement and various ideas from the
Board to address it. We are offering multiple channels for questions and
feedback. With the help of a team of community facilitators, we are
organizing multiple conversations with multiple groups in multiple
languages.
During this call for feedback we publish weekly reports and we draft the
final report that will be delivered to the Board. With the help of this
report, the Board will approve the next steps to organize the selection of
six community seats in the upcoming months. Three of these seats are due
for renewal and three are new, recently approved.
*Participate in this call for feedback and help us form a more diverse and
better performing Board of Trustees!*
*Problems:* While the Wikimedia Foundation and the movement have grown
about five times in the past ten years, the Board’s structure and processes
have remained basically the same. As the Board is designed today, we have a
problem of capacity, performance, and lack of representation of the
movement’s diversity. This problem was identified in the Board’s 2019
governance review, along with recommendations for how to address it.
To solve the problem of capacity, we have agreed to increase the Board size
to a maximum of 16 trustees (it was 10). Regarding performance and
diversity, we have approved criteria to evaluate new Board candidates. What
is missing is a process to promote community candidates that represent the
diversity of our movement and have the skills and experience to perform
well on the Board of a complex global organization.
Our current processes to select individual volunteer and affiliate seats
have some limitations. Direct elections tend to favor candidates from the
leading language communities, regardless of how relevant their skills and
experience might be in serving as a Board member, or contributing to the
ability of the Board to perform its specific responsibilities. It is also a
fact that the current processes have favored volunteers from North America
and Western Europe. Meanwhile, our movement has grown larger and more
complex, our technical and strategic needs have increased, and we have new
and more difficult policy challenges around the globe. As well, our
Movement Strategy recommendations urge us to increase our diversity and
promote perspectives from other regions and other social backgrounds.
In the upcoming months, we need to renew three community seats and appoint
three more community members in the new seats. What process can we all
design to promote and choose candidates that represent our movement and are
prepared with the experience, skills, and insight to perform as trustees?
*Ideas:* The Board has discussed several ideas to overcome the problems
mentioned above. Some of these ideas could be taken and combined, and some
discarded. Other ideas coming from the call for feedback could be
considered as well. The ideas are:
1. *Ranked voting system*. Complete the move to a single transferable
vote system, already used to appoint affiliate-selected seats, which is
designed to best capture voters’ preferences.
2. *Quotas*. Explore the possibility of introducing quotas to ensure
certain types of diversity in the Board (details about these quotas to be
discussed in this call for feedback).
3. *Call for types of skills and experiences*. When the Board makes a
new call for candidates, they would specify types of skills and experiences
especially sought.
4. *Vetting of candidates*. Potential candidates would be assessed using
the Trustee Evaluation Form and would be confirmed or not as eligible
candidates.
5. *Board-delegated selection committee*. The community would nominate
candidates that this committee would assess and rank using the Trustee
Evaluation Form. This committee would have community elected members and
Board appointed members.
6. *Community-elected selection committee*. The community would directly
elect the committee members. The committee would assess and rank candidates
using the Trustee Evaluation Form.
7. *Election of confirmed candidates.* The community would vote for
community nominated candidates that have been assessed and ranked using the
Trustee Evaluation Form. The Board would appoint the most voted candidates.
8. *Direct appointment of confirmed candidates*. After the selection
committee produces a ranked list of community nominated candidates, the
Board would appoint the top-ranked candidates directly.
*Call for feedback:* The call for feedback[1] runs from February 1 until
the end of March 14. We are looking for a broad representation of opinions.
We are interested in the reasoning and the feelings behind your opinions.
In a conversation like this one, details are important. We want to support
good conversations where everyone can share and learn from others. We want
to hear from those who understand Wikimedia governance well and are already
active in movement conversations. We also want to hear from people who do
not usually contribute to discussions. Especially those who are active in
their own roles, topics, languages or regions, but usually not in, say, a
call for feedback on Meta.
You can participate by joining the Telegram chat group[2], and giving
feedback on any of the talk pages on Meta-Wiki. We are welcoming the
organisation of conversations in any language and in any channel. If you
want us to organize a conversation or a meeting for your wiki project or
your affiliate, please write to me.
Regards,
Krishna Chaitanya
[1]
https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Board_of_Trustees/Call…
[2] https://t.me/wmboardgovernancechat
The following changes will be made without deprecation in gerrit
626548 <https://gerrit.wikimedia.org/r/c/mediawiki/core/+/626548>:
* FirejailCommand was removed. CodeSearch reveals no usages.
* Command::execute() now returns a Shellbox\Command\UnboxedResult
instead of a MediaWiki\Shell\Result. I added a class alias, but
type hints don't necessarily cause PHP to load the alias when it
parses the file, so any such type hints should be manually
updated. CodeSearch finds no affected code.
Context:
This is part of T260330 <https://phabricator.wikimedia.org/T260330>
"PHP microservice for containerized shell execution" a.k.a. Shellbox.
The patch moves the traditional shell execution code from MediaWiki to
the Shellbox library, and mildly refactors it. Rigorous backwards
compatibility was a goal of this change. We took the opportunity to
make a few minor interface changes, but we don't think anyone will be
affected.
-- Tim Starling
This script was created in 2011 and takes an offline XML dump file,
containing page content wikitext, and feeds its entries through the
Preprocessor without actually importing any content into the wiki.
The documented purpose of the script is to "get statistics" or "fill the
cache". I was unable to find any stats being emitted. I did find that the
method called indeed fills "preprocess-hash" cache keys, which have a TTL
of 24 hours (e.g. Memcached).
I could not think of a use case for this and am wondering if anyone
remembers its original purpose and/or knows of a current need for it.
-- Timo
[1] First commit: http://mediawiki.org/wiki/Special:Code/MediaWiki/80466
[2] Commit history:
https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+log/1.35.0/m…
PS: This came up because there's a minor refactor proposed to the script,
and I was wondering how to test it and whether we it makes sense to
continue maintenance and support for it.
https://gerrit.wikimedia.org/r/641323
Hello all,
I’m pleased to announce that the Technical Decision Making Process[0]
proposed as an evolution of the Wikimedia Technical Committee[1]
(TechCom) process has been approved by the Wikimedia Foundation, and
will begin operation on 22nd of January. Back in October[2] and
November[3] I sought input into a proposed process and in December I
incorporated that feedback into the approved process.
This process is designed to be more inclusive by shifting the
representations. It has clear timelines for when decisions will be
made and to develop a clear lifecycle of a decision. The process is
designed to be clear about how and which stakeholders will be engaged.
It also introduces a Technical Decision Forum and Templates for the
process.
Currently a group from the Wikimedia Technology and Product
Departments are in the process of forming the initial Decision Forum.
The initial forum will include representatives from Wikimedia
Foundation teams, Wikimedia Deutschland, and independent +2
contributors. Please see the proposal for community representation on
the Decision Forum[4] and provide input by 2021-02-15. We know we will
need to adjust the representation in the forum over time. If you
believe you are from a group that is not represented and should be,
please contact us (tech-decision-forum-support(a)wikimedia.org).
If you currently have an RFC in process with TechCom that is not on
Last Call, it may need to be moved into this process. If you have
filed an RFC that is no longer relevant please close it. The group
setting up the process will be inquiring about the RFC status on the
individual Phabricator tickets.
To get started with this new process you just need to open a
Phabricator Ticket on the Technical Decision Making Process board[5].
If you need help getting started or have further questions please
reach out at tech-decision-forum-support(a)wikimedia.org or reply on
this thread.
[0] https://www.mediawiki.org/wiki/Technical_Decision_Making_Process
[1] https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee
[2] https://lists.wikimedia.org/pipermail/wikitech-l/2020-October/093968.html
[3] https://lists.wikimedia.org/pipermail/wikitech-l/2020-November/094037.html
[4] https://www.mediawiki.org/wiki/Technical_Decision_Making_Process/Draft_Prop…
[5] https://phabricator.wikimedia.org/project/board/5179/
Thanks,
Kate
--
Kate Chapman (she/her/hers)
Director of Architecture, Architecture Team
Wikimedia Foundation
kchapman(a)wikimedia.org
Hello,
The last blocker for 1.36.0-wmf.29 is:
* MemcachedPeclBagOStuff: Serialization of 'Closure' is not allowed
- https://phabricator.wikimedia.org/T273242
It comes from the extension FeaturedFeeds which attempt to serialize
some User objects in the cache. An issue we already identified in a
previous train when we marked User as not being serializable:
https://phabricator.wikimedia.org/T264391
This time it is unknown what caused the regression (most probably it is
due to a change in mediawiki/core). Anyway there is a patch up for
review to stop serializing those objects but instead drop usage of User
objets and cache an array of properties instead of the object itself:
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/FeaturedFeeds/+/66135…
Without this fixed, the train can not progress further.
cheers,
--
Antoine "hashar" Musso
https://www.mediawiki.org/wiki/Scrum_of_scrums/2021-02-03#2021-02-03
= 2021-02-03 =
== Callouts ==
* [RelEng] After several failed attempts to rollout wmf.28, we abandoned
the release and moved on to wmf.29. This was done because (1) wmf.29 is a
superset of code in wmf.28 (2) we need a stable base to rollback to and
there was not enough time to determine wmf.28's stability.
* [Performance] New synthetic testing user journey dashboard:
https://grafana.wikimedia.org/d/d-pdqGBGdse/wikipedia-login-user-journey?or…
* [Performance] FOSDEM happening this weekend online. We're running a
devroom: https://fosdem.org/2021/schedule/track/web_performance/
=== No updates ===
CommTech, Editing,
=== '''No notes provided''' ===
Web, Product Infrastructure, Parsing, Language, Inuka, Analytics, Cloud
Services, Fundraising Tech, Platform, Security
== SoS Meeting Bookkeeping ==
* Updates:
** For 2021-02-10 meeting, please list the recommendations from the
retrospective that you would like to adopt:
https://docs.google.com/spreadsheets/d/1EQzjJrAW5RFN7YnrAXPpeaxOB7pQ_2jrpeL…
== Product ==
=== Community Tech ===
* Blocked by:
* Blocking:
* Updates:
** No Updates
=== Anti-Harassment Tools ===
* Blocked by:
* Blocking:
* Updates:
** 3rd new engineer started - thanks to Dayllan Maza for helping with code
review while new engineers onboard
** Thanks to DBAs and Amir Sarabadani for helping with SecurePoll tables
** Thanks to Martin Urbanec and James Forrester for setting up beta
environment for votewiki
=== Editing ===
* Blocked by:
* Blocking:
* Updates:
** No updates
=== Growth ===
* Blocked by:
* Blocking:
* Updates:
** Working on link recommendations and on making the mwaddlink service
production-ready. [[phab:T252822]]
=== iOS native app ===
* Blocked by:
* Blocking:
* Updates:
** New version out yesterday! Investigated OAuth - endpoints aren't
currently adaptable for a variety of reasons.
=== Android native app ===
* Blocked by:
* Blocking:
* Updates:
** New version out yesterday with watchlists and native talk pages!
=== Structured Data ===
* Blocked by:
* Blocking:
* Updates:
** MediaSearch: more feature work and bug fixes, discussing approach for
making MediaSearch the default search interface on Commons
** Image recommendations: investigating how to compare and evaluate search
algorithm results
** Working with the Architecture team on plans for implementing
infrastructure changes related to the Structured Data Across Wikimedia
program
=== Abstract Wikipedia ===
* Blocked by:
* Blocking:
* Updates:
** Moving on to phase gamma, finally!
** Working on getting the back-end executor service runnable.
** Huge thanks to Daniel and others from the Platform team for sharing
their plans for how the core of MediaWiki's Content hierarchy is going, and
advice on our code on how to be more testable.
** Community suggestions for the logo concept for Wikifunctions continues.
=== Library ===
* Blocked by:
* Blocking:
* Updates:
** Set up hosted versions of GlitchTip for Wikilink and The Wikipedia
Library
** Bug fixes for The Wikipedia Library
** Onboarding new designer
=== Vue.js ===
* Blocked by:
* Blocking:
* Updates:
** Developing processes and timelines
** Discussing Vue, the new component library, and related infrastructure
with various stakeholders (other Product teams, WMDE, RelEng, Community
Relations...)
** Adapting MediaWiki's ES5 minifier to work with ES6
** Experimenting with alternatives to webpack (namely Vite, which uses
Rollup)
** Reviewing existing Vue components within MediaWiki projects and starting
to form a plan for building out components in WVUI
== Technology ==
=== Engineering Productivity ===
==== Performance ====
* Blocked by:
* Blocking:
* Updates:
** RUM (field) performance data for the past couple of weeks is tainted due
to partial outages during the Event Platform migration
==== Quality and Test Engineering ====
* Blocked by:
* Blocking:
* Updates:
** Blog post by Elena Tonkovidova: Testing search in MediaSearch
https://phabricator.wikimedia.org/phame/live/21/post/226
==== Release Engineering ====
* Blocked by:
** [https://phabricator.wikimedia.org/T273242 MemcachedPeclBagOStuff:
Serialization of 'Closure' is not allowed]
*** Listed as PET, has a patch
* Blocking:
** ???
* Updates:
** [All] Deployments/Covid-19
https://wikitech.wikimedia.org/wiki/Deployments/Covid-19
** Train Health
*** Last week: 1.36.0-wmf.28 [[phab:T271342]] <!--
https://phabricator.wikimedia.org/T271342 -->
*** This week: 1.36.0-wmf.29 [[phab:T271343]] <!--
https://phabricator.wikimedia.org/T271343 -->
*** Next week: 1.36.0-wmf.30 [[phab:T271344]] <!--
https://phabricator.wikimedia.org/T271344 -->
** Just starting to think about npm 7, now that's released.
=== Search Platform ===
* Blocked by:
** (DCOps) Memory issue on elastic1063 caused elasticsearch to be killed -
https://phabricator.wikimedia.org/T265113
* Blocking:
* Updates:
** Include gsrprop & gsrsinfo data in search api generator response -
https://phabricator.wikimedia.org/T270381
** Implement 50kb limit on file text indexing for to reduce increasing
commonswiki_file on-disk size -https://phabricator.wikimedia.org/T271493
** Add swift plugin to Flink k8s - https://phabricator.wikimedia.org/T269876
** Create Helm Chart (for Flink) - https://phabricator.wikimedia.org/T265526
=== Site Reliability Engineering ===
* Blocked by:
** None
* Blocking:
** Search Platform on Memory issue on elastic1063 caused elasticsearch to
be killed -https://phabricator.wikimedia.org/T265113
* Updates:
** Working on kubernetes logging for mediawiki with observability.
== Cross-cutting ==
* Blocked by:
* Blocking:
* Updates:
** No major progress on PHP 8.0 work.
** CI tools upgrades' status:
https://libraryupgrader2.wmcloud.org/status?branch=master
*** Upgrade of mediawiki-codesniffer to 35.0.0 is 93% complete.
*** Upgrade of eslint-config-wikimedia to 0.18.1 is 87% complete.