TLDR: fresh-node now uses Node 18. To learn more or install/upgrade, refer to https://gerrit.wikimedia.org/g/fresh.

Hi all,

Fresh 23.08.1 is upon is. It's a fairly big release!

*What's new?*

After a delay of more than two years, Node.js 18 and npm 9 (replacing Node.js 16 and npm 7) are available to WMF CI as of yesterday, and can now be used locally through Fresh! We've also added early support for Node.js 20, you can opt-in via the fresh-node20 command.

The default fresh-node command was updated to Node.js 18. You can continue to use older versions for another 6-12 months via the fresh-node16 and fresh-node14 commands. The fresh-node12 command has been removed (unsupported since last year https://github.com/nodejs/Release#end-of-life-releases).

We've also made a few minor tweaks to improve support for Podman https://podman.io/ to encourage competition and use of freely-licensed software (Docker for Mac/Windows require the proprietary Docker Desktop). We've also improved experimental support for Apple ARM-based devices. Node, npm, and Firefox work out of the box using Docker's default emulation (including the faster Rosetta-based emulation). For example, using `npx grunt karma:firefox`. Chrome has yet to support emulation within Docker.

Full changelog: https://gerrit.wikimedia.org/g/fresh/+/23.08.1/CHANGELOG.md Commits: https://gerrit.wikimedia.org/g/fresh/+log/23.08.1/

*What was the hold up?*

The updating and creation of new Docker images for WMF CI is rather simple — involving little more than a directory copy or a one-line change (example 1 https://gerrit.wikimedia.org/r/c/integration/config/+/891648/1/dockerfiles/node16/Dockerfile.template, example 2, https://gerrit.wikimedia.org/r/c/integration/config/+/894125/3 example 3 https://gerrit.wikimedia.org/r/c/integration/config/+/946960). We run a very efficient operation here, with people like James Forrester, Antoine Musso, and myself routinely migrating the entire Wikimedia movement's CI workloads in mere minutes! We use Zuul to automatically churn through hundreds of Jenkins jobs, used on a daily basis by thousands of Gerrit's Git repositories. It's quite the feat of automation https://www.mediawiki.org/wiki/Continuous_integration/Docker really.

The holdup was in upgrading the browser tests of several product features to Webdriver.io v7. (Browser tests are sometimes associated with "selenium" for hysterical raisins, they involve no Selenium tech today.)

Webdriver 6 and earlier use the deprecated Node.js Fibers https://www.npmjs.com/package/fibers functionality to emulate async code as synchronous code. This allowed our developers to write test cases using a simpler syntax in which async-await statements could be omitted. The underlying functionality for this was discontinued in Node.js 16. Thus until a team migrated their tests to Webdriver 7, the associated CI pipeline would have to remain on Node.js 14.

One of the strengths of our WMF CI setup is that projects can bi-directionally integrate. This is one of many capabilities that Zuul and a number of other large CI systems have provided for over a decade, that e.g. GitHub/GitLab (still in their infancy around CI) are only just starting to explore. Zuul allows MediaWiki core to ensure its changes can't break extensions, and likewise allows each extension to ensure it can't break other extensions that it is meant to work together with, e.g. the MediaWiki bundle, or WMF production. We currently run this on a single Docker container, which supports local testing without a new and different dev environment for each of the 1000+ extensions. This is efficient from a big picture perspective, especially as a non-profit (though big corporations seem to make the same choice). We've also designed our infastructure to generally support two or three major versions simultanously, which allows individual teams to schedule maintenance at their own convenience.

But, all this relies on the assumption that teams are aligned, that products in production have an owner, and that owners discover and (eventually) prioritise routine maintenance within 3-6 months.

Those assumptions allow for wide margins, they we don't meet those anymore, with dozens of major capabilities and products having lapsed https://www.mediawiki.org/wiki/Developers/Maintainers in ownership over the years (or with existing ownership insufficiently excercised). In addition, the CI infrastructure lacks a directly responsible individual. Thus team's upgrade tasks https://phabricator.wikimedia.org/T256626 can linger for years, with nobody accountable for executing a chosen set of high-level priorities or knowing the needs of individual MediaWiki development teams. Everybody is waiting for somebody else while we continue to progress within our local maxima.

Our support window covered Node 12, 14 and 16. The volunteers maintaining the CI jobs (James and myself in this case) choose not to start supporting Node 18 until we can start dropping older stuff. Despite individual teams expressing https://phabricator.wikimedia.org/T314051 their blockage https://phabricator.wikimedia.org/T337647 and a desire to start working with newer versions, there exists no incentive for anyone to act on this.

What resolved it in this case is that last week, I too became someone wanted to start testing something on Node 18. And I'm not willing to maintain a custom dev env or CI pipeline just for myself. So I spent several days identifying the four remaining projects (analysis https://phabricator.wikimedia.org/T256626#9063307), upgrading each project, communicating with teams at WMDE/WMF, turning off the remaining unmaintained tests, dusting off James' patches for upgrading Quibble CI jobs from Node 14 to Node 16, adding Node 18, and finally releasing Fresh with Node 18 support.

*Welcome Release Engineering!*

We're transitioning ownership of Fresh from the Performance Team to the Release Engineering team. This release was done in collaboration with Antoine Musso!

*What is Fresh?* ** Fresh is a fast way to launch isolated environments from your terminal. These can be used to work more securely and responsibly https://timotijhof.net/posts/2019/protect-yourself-from-npm/ with Node.js-based developer tools, especially those installed from npm such as ESLint, QUnit, Grunt, Webdriver, and more. Example guide: https://www.mediawiki.org/wiki/Manual:JavaScript_unit_testing.

To file tasks or browse existing ones, check the Phabricator board at https://phabricator.wikimedia.org/tag/fresh/.

-- Timo Tijhof, Principal Engineer, Wikimedia Foundation.