Thank you, Alexandra! I like this format, it was informative and more interesting. On Thu, 2 Jul 2020 at 01:32, Alexandra Paskulin &lt;apaskulin(a)wikimedia.org&gt; wrote: ... Recently, I haven't seen the security question of npm being discussed on https://phabricator.wikimedia.org/T199004 and thought that question was not on topic. If that's not the case: is there another discussion I'm not aware of? The discussion on the ticket seems to be focused on a different topic now and I don't see security discussed elsewhere. To avoid distraction, I'm asking here. If you are aware of a relevant ticket, please share. The issue of package security has been answered by the Node community multiple times in different forms through the years. Were these solutions evaluated? These generally avoid unvetted code by hosting a *private node package repository* in some form, typically in a git repository, where only *vetted versions* of packages are checked in. These include the complete dependency trees, the package hashes and the full source code of packages, therefore provide more complete security than libraryupgrader2, which - in my understanding - only controls our top-level dependencies. Without completeness I'd mention 2 recent solutions: * Yarn 2 (berry) offline cache <https://yarnpkg.com/features/offline-cache>. Yarn 2 (introductory article <https://dev.to/arcanis/introducing-yarn-2-4eh1>) came out this year. It is a recent, fundamentally different version from (Classic) Yarn 1, which made only minor improvements to npm. Yarn 2 adds package deduplication and removes the need to uncompress packages, thus speeding up the `npm ci` step. * Pnpm package store <https://pnpm.js.org/en/about-package-store>. Pnpm came out 3 years ago, after Classic Yarn. Adds package deduplication with symlinking, recreating node_modules with uncompressed packages, but without dependency tree flattening (node_modules contains only the direct dependencies). Both package managers can live alongside npm, thus can be evaluated and introduced gracefully without hard transitions: * 'package.json' is shared, the package versions installed are expected to be the same if run at the same time (same state of npm repo). * 'package-lock.json' is NOT shared, there are separate 'yarn.lock' and 'pnpm-lock.yaml' files, these need to be generated separately. My question in this regard: 1. Were these options discussed, is there an initiative to evaluate these PMs and their impact? I haven't found signs of this on phab. 2. I've been using these PMs and made the necessary additions to some of our projects. Where to discuss my findings and is there someone interested in reviewing patches adding the missing dependencies to 'package.json' files, that would make the repositories compatible with these PMs without altering npm's behavior? To clarify: this inquiry is about evaluating these alternatives without affecting the use of npm or targetting the replacement of npm. Some of my findings, in advance: Neither PM is plug-and-play: these package managers are more strict than npm, some packages need to be updated and our 'package.json' files need some corrections. 1. npm's package tree flattening in node_modules avoids strict dependency checking: packages can use other packages installed by non-direct dependencies, or some other random package. This hides some missing dependency declarations, most notably packages' `peerDependencies` are often not declared in `dependencies` of the including package. Both Yarn2 and Pnpm are more strict than npm in this matter and require the declaration of those. This is a trivial update to mediawiki's 'package.json' files that's worth the added security. 2. Yarn 2 removes the node_modules folder and uses a custom package loader that mounts package archives directly, without uncompressing to disk (except for "unplugged" packages, such as the usual suspects 'node-gpy' and 'fibers'...). Packages using `require.resolve()` work seamlessly, but there are many packages that make assumptions about the node_modules folder and try to load files directly. Major packages have been updated (support list <https://yarnpkg.com/features/pnp#native-support>), but there are packages in our dependency tree that aren't yet. These can be solved by a simple patch to upstream - thus benefiting the open-source community - or by unplugging those packages. In any case, this is a cross-cutting concern in all repos and a coordinated approach would be beneficial. * DK: To what extent are we willing to run arbitrary code on our systems? * TT: NPM packages are known for depending on a lot of unreviewed/unknown With a privately hosted package store 100% of the code can be vetted. It's up to the capacity and diligence of the maintainers how thoroughly the code is reviewed. A git package store would require the evaluation of the performance team. I assume for CI that would act as a local cache, possibly reducing network traffic. For developers I wouldn't suggest putting load on a WMF hosted store. Packages downloaded from the public repositories are checksummed, rejecting any package that has different contents than the vetted version. I assume the repository would be updated by libraryupgrader, which would manage a significantly larger number of packages than now. Thank you for reading. Aron (Demian)

Hello, On Tue, 7 Jul 2020 at 00:30, Kunal Mehta &lt;legoktm(a)member.fsf.org&gt; wrote: I've suspected vetting and quantity poses big challenges. To my surprise I haven't found any information on how that's done. To discuss this topic, I've made the subtask to the build step RFC: T257072 <https://phabricator.wikimedia.org/T257072> "Determine Node package auditing workflows" Analyzing all that information is a superhuman task. A gerrit/gitlab like review interface would make it more approachable. The way Pnpm stores packages (uncompressed in a git repo) would enable that. Yarn stores .zip files, but a separate uncompressed repo could be used for code review, updates submitted as regularly reviewed patches. 100% review wouldn't be sustainable IMO (causing developer burnout very quickly), but looking for specific patterns exhibited in malicious packages could be a successful approach to increase trust in the audited code. Patterns like: * An unmaintained repo receiving an update, which is a common solution to inject malicious code. * New packages added to the dependency tree. An interesting article in this regard: https://portswigger.net/daily-swig/new-npm-scanning-tool-sniffs-out-malicio… I wonder if the npm-scan <https://github.com/spaceraccoon/npm-scan> tool mentioned therein has been evaluated. The repo of former malicious packages (npm-zoo <https://github.com/spaceraccoon/npm-zoo>) is also worth mentioning. I've collected a few notable incidents in the RFC under section A_few_examples_of_NPM_incidents <https://www.mediawiki.org/wiki/User:Aron_Manning/RfC:_Evaluate_alternative_Node_package_managers_for_improved_package_security#A_few_examples_of_NPM_incidents> . A tool looking for differences between the git repo and the npm tarball could be useful. It's possible though that many packages would require special treatment if the tarball isn't simple to map on the git repo. However, just a simple check to see if a new npm release has a corresponding git tag or release - or any commits at all - would catch injections done with a stolen NPM token. How do these alternative package managers address the quantity of npm None of the package managers can or intend to do code review apart from `npm audit`, available with all 3. It seems to me there is an expectation that PMs will protect us. It should be clarified that no tool can do that, the purpose of these tools is to give 100% control over what packages and versions are installed. What these PMs provide is detailed in the RfC for evaluating alternative PMs <https://www.mediawiki.org/wiki/User:Aron_Manning/RfC:_Evaluate_alternative_Node_package_managers_for_improved_package_security#Package_managers> What versions we add to the local package repo is up to us. Separate ticket: T257072 <https://phabricator.wikimedia.org/T257072> I think a 2 stage deployment process would subject packages to as much scrutiny as possible within the constraints: 1. An auditing package repository with all the updates to be vetted. This would be used in sandboxed environments to expose updates to developers, who could notice outstanding behavior, warning signs. 2. A stable package repository for CI and not sandboxed environments. The review process: 1. The auditing repo only includes packages used in WMF projects. 2. Package versions need to be greenlighted for auditing too. This is preceded by a basic check of the validity of that version to look for eg, stolen credential injections, but code is only reviewed if suspicious, eg. new packages, unexpected updates. 3. Package versions would stay in this stage for some time (eg. 2 weeks), depending on the package and urgency. 4. A changelog in the auditing repo tracks the newest updates, informing developers about what packages to pay attention to. 5. One of the developers dedicated to package vetting does a deeper review of the code. This should be aided by heuristic tools. 6. When confidence in a version is built, that version is greenlighted for the stable repo. Demian (Aron)