Hi all,
we have to do maintenance on Gerrit tomorrow 23rd Aug 06:30 UTC. The
maintenance requires a short reboot. The estimated downtime should be
around 10 minutes.
Greetings
Jelto
Hi all!
If you're part of the "trusted-contributors" Gerrit group, you likely got
an email from our GitLab test instance informing you you were added to the
"People/Trusted Contributors" group.
This is only on our test instance.
Context: we're testing permissions and roles for adding everyone in
GitLab.[0]
___
*Details:*
We're making folks "Owners" in the "People/Trusted Contributors" group and
that group will be "Developers" on our repositories[1].
This is similar to the access "trusted-contributors" have on Gerrit (we
think :))
Our test instance is a low-stakes way to try changes.
Feel free to login, poke around, and leave comments on Phabricator[0].
Thanks!
Tyler Cipriani (he/him)
Engineering Manager, Release Engineering
Wikimedia Foundation
[0]: <https://phabricator.wikimedia.org/T344379>
[1]: <https://docs.gitlab.com/ee/user/permissions.html>
TLDR: fresh-node now uses Node 18. To learn more or install/upgrade, refer to https://gerrit.wikimedia.org/g/fresh.
Hi all,
Fresh 23.08.1 is upon is. It's a fairly big release!
*What's new?*
After a delay of more than two years, Node.js 18 and npm 9 (replacing Node.js 16 and npm 7) are available to WMF CI as of yesterday, and can now be used locally through Fresh! We've also added early support for Node.js 20, you can opt-in via the fresh-node20 command.
The default fresh-node command was updated to Node.js 18. You can continue to use older versions for another 6-12 months via the fresh-node16 and fresh-node14 commands. The fresh-node12 command has been removed (unsupported since last year <https://github.com/nodejs/Release#end-of-life-releases>).
We've also made a few minor tweaks to improve support for Podman <https://podman.io/> to encourage competition and use of freely-licensed software (Docker for Mac/Windows require the proprietary Docker Desktop). We've also improved experimental support for Apple ARM-based devices. Node, npm, and Firefox work out of the box using Docker's default emulation (including the faster Rosetta-based emulation). For example, using `npx grunt karma:firefox`. Chrome has yet to support emulation within Docker.
Full changelog: https://gerrit.wikimedia.org/g/fresh/+/23.08.1/CHANGELOG.md
Commits: https://gerrit.wikimedia.org/g/fresh/+log/23.08.1/
*What was the hold up?*
The updating and creation of new Docker images for WMF CI is rather simple — involving little more than a directory copy or a one-line change (example 1 <https://gerrit.wikimedia.org/r/c/integration/config/+/891648/1/dockerfiles/…>, example 2, <https://gerrit.wikimedia.org/r/c/integration/config/+/894125/3> example 3 <https://gerrit.wikimedia.org/r/c/integration/config/+/946960>). We run a very efficient operation here, with people like James Forrester, Antoine Musso, and myself routinely migrating the entire Wikimedia movement's CI workloads in mere minutes! We use Zuul to automatically churn through hundreds of Jenkins jobs, used on a daily basis by thousands of Gerrit's Git repositories. It's quite the feat of automation <https://www.mediawiki.org/wiki/Continuous_integration/Docker> really.
The holdup was in upgrading the browser tests of several product features to Webdriver.io v7. (Browser tests are sometimes associated with "selenium" for hysterical raisins, they involve no Selenium tech today.)
Webdriver 6 and earlier use the deprecated Node.js Fibers <https://www.npmjs.com/package/fibers> functionality to emulate async code as synchronous code. This allowed our developers to write test cases using a simpler syntax in which async-await statements could be omitted. The underlying functionality for this was discontinued in Node.js 16. Thus until a team migrated their tests to Webdriver 7, the associated CI pipeline would have to remain on Node.js 14.
One of the strengths of our WMF CI setup is that projects can bi-directionally integrate. This is one of many capabilities that Zuul and a number of other large CI systems have provided for over a decade, that e.g. GitHub/GitLab (still in their infancy around CI) are only just starting to explore. Zuul allows MediaWiki core to ensure its changes can't break extensions, and likewise allows each extension to ensure it can't break other extensions that it is meant to work together with, e.g. the MediaWiki bundle, or WMF production. We currently run this on a single Docker container, which supports local testing without a new and different dev environment for each of the 1000+ extensions. This is efficient from a big picture perspective, especially as a non-profit (though big corporations seem to make the same choice). We've also designed our infastructure to generally support two or three major versions simultanously, which allows individual teams to schedule maintenance at their own convenience.
But, all this relies on the assumption that teams are aligned, that products in production have an owner, and that owners discover and (eventually) prioritise routine maintenance within 3-6 months.
Those assumptions allow for wide margins, they we don't meet those anymore, with dozens of major capabilities and products having lapsed <https://www.mediawiki.org/wiki/Developers/Maintainers> in ownership over the years (or with existing ownership insufficiently excercised). In addition, the CI infrastructure lacks a directly responsible individual. Thus team's upgrade tasks <https://phabricator.wikimedia.org/T256626> can linger for years, with nobody accountable for executing a chosen set of high-level priorities or knowing the needs of individual MediaWiki development teams. Everybody is waiting for somebody else while we continue to progress within our local maxima.
Our support window covered Node 12, 14 and 16. The volunteers maintaining the CI jobs (James and myself in this case) choose not to start supporting Node 18 until we can start dropping older stuff. Despite individual teams expressing <https://phabricator.wikimedia.org/T314051> their blockage <https://phabricator.wikimedia.org/T337647> and a desire to start working with newer versions, there exists no incentive for anyone to act on this.
What resolved it in this case is that last week, I too became someone wanted to start testing something on Node 18. And I'm not willing to maintain a custom dev env or CI pipeline just for myself. So I spent several days identifying the four remaining projects (analysis <https://phabricator.wikimedia.org/T256626#9063307>), upgrading each project, communicating with teams at WMDE/WMF, turning off the remaining unmaintained tests, dusting off James' patches for upgrading Quibble CI jobs from Node 14 to Node 16, adding Node 18, and finally releasing Fresh with Node 18 support.
*Welcome Release Engineering!*
We're transitioning ownership of Fresh from the Performance Team to the Release Engineering team. This release was done in collaboration with Antoine Musso!
*What is Fresh?*
**
Fresh is a fast way to launch isolated environments from your terminal. These can be used to work more securely and responsibly <https://timotijhof.net/posts/2019/protect-yourself-from-npm/> with Node.js-based developer tools, especially those installed from npm such as ESLint, QUnit, Grunt, Webdriver, and more. Example guide: https://www.mediawiki.org/wiki/Manual:JavaScript_unit_testing.
To file tasks or browse existing ones, check the Phabricator board at https://phabricator.wikimedia.org/tag/fresh/.
--
Timo Tijhof,
Principal Engineer,
Wikimedia Foundation.
Hi everybody,
The Machine Learning team rolled out a change to most of the wikis
(excluding fi, wikidata and en) that changes the backend service called by
the ORES Extension code. This is part of a process to modernize our ML
infrastructure, you can find more details here:
https://phabricator.wikimedia.org/T319170https://wikitech.wikimedia.org/wiki/ORES
The main impact that users could have is getting weird results in Recent
Changes filters (all the ones related to ORES, for example "User intent
predictions" and "Contribution quality prediction"). So far the only issue
was reported by the fiwiki's community, tracked in
https://phabricator.wikimedia.org/T343308 (we are still investigating what
happened). If you see anything out of the ordinary, feel free to contact
the Machine Learning team:
IRC libera: #wikimedia-ml
Phabricator: Machine-Learning-team tag
Thanks in advance!
Luca (on behalf of the Machine Learning team)
Hi Strainu,
Here Diego from the WMF Research team,
> 1. In [1], the bot owner is encouraged to move to the revertrisk score.
However, in [2], it's explicitly mentioned that the model should not be
used for "Auto-removing edits that a user makes without another editor in
the loop". So, should bot owners currently reverting based on goodfaith and
damaging scores explore the new models? If so, do you have any suggestions
on how to automatically match thresholds between the old and new models?
Sorry for the confusion, we have updated this model card. You can use this
model for "automatically reverting content" as you were using ORES. Here
<https://phabricator.wikimedia.org/F37149700> you can see the model's
performance comparison.
Our current recommendation is to use the Language Agnostic
<https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Language-a…>model
for this task (patrolling bots) The Multilingual
<https://meta.wikimedia.org/wiki/Machine_learning_models/Proposed/Multilingu…>
model is performing better for IP Edits, but we are still working on
improving its stability. Within the next 3 months we expect to improve
Language Agnostic accuracy in anonymous edits, and also Multilingual model
stability.
Best,
Diego
Hello all
GitLab will be down tomorrow Aug 3rd, from 07:45 AM to 08:00 AM UTC for a
major version upgrade. The expected downtime is around 15 minutes. Much of
this maintenance window will be downtime, during which you will be unable
to access GitLab through ssh or the web interface.
Some expected changes with the new GitLab version are:
* new features (see
https://about.gitlab.com/releases/2023/05/22/gitlab-16-0-released)
* Some depreciations and removals:
* Behavior of tokens changed (for example scope of CI_JOB_TOKEN changed,
all personal, project and group tokens have a expiring date of one year now)
* character length for certain keywords in .gitlab-ci.yml
* see changelog:
https://docs.gitlab.com/ee/update/deprecations.html?removal_milestone=16.0
If you notice any error, please reach out at
https://phabricator.wikimedia.org/T338460 or on IRC at #wikimedia-gitlab.
Greetings
Jelto
Hello all!
The Search Platform Team usually holds an open meeting on the first
Wednesday of each month. Come talk to us about anything related to
Wikimedia search, Wikidata Query Service (WDQS), Wikimedia Commons Query
Service (WCQS), etc.!
Feel free to add your items to the Etherpad Agenda for the next meeting.
Details for our next meeting:
Date: Wednesday, May 3, 2023
Time: 15:00-16:00 UTC / 08:00 PDT / 11:00 EDT / 17:00 CET
Etherpad: https://etherpad.wikimedia.org/p/Search_Platform_Office_Hours
Google Meet link: https://meet.google.com/vgj-bbeb-uyi
Join by phone: https://tel.meet/vgj-bbeb-uyi?pin=8118110806927
Have fun and see you soon!
Guillaume
--
*Guillaume Lederrey* (he/him)
Engineering Manager
Wikimedia Foundation <https://wikimediafoundation.org/>