As someone who now works in js/node primarily I might be able to add a little outside insight.
IMHO bower was probably killed far before npm@3 came out.
- npm has always been more advanced - bower's main isn't as reliable as it is in npm. Even if modules aren't forgetting to set it as much as they used to I still see hard to deal with things like modules that include .sass sources in it. - bower's distribution model is actually very problematic for upstream developers. bower clones git repositories directly expecting to find built/compiled js and has no local build hooks. Few major js libraries work in classic js/css sources anymore, instead having some required build step. Bower makes a complete mess of this. Requiring the build result to be committed with each change, an automated build bot, a second git repo for build results only, or the upstream dev to just not care about bower. - Thanks to the rise of browserify and the like more and more client-only libraries have moved to npm despite people traditionally thinking of it as being for nodejs modules. Most client only libraries now exist in npm. And if you wave the name browserify around you can get just about any js library to start distributing to npm. - This has actually gone so far that once when I added a contribution to a library I noticed that they were actually forgetting to keep their bower.json up to date.
npm@3 is also probably not as important as it's being made out here.
npm@3 still doesn't guarantee a tree will be 100% flat. Most of npm@3's changes fix small quirks in front-end usage and stability issues with npm.
The 'major' change of 'flatness' is really that when installing `a` that depends on `b`, `b` will be hoisted up to the same level as `a`, if possible, instead of always being installed under `a`. npm@2 did have some annoying quirks during development/updating that could leave a duplicate module until you recreated your node_modules tree. And there was the side case of installing two modules that both depended on something you did not depend on. But that could be dealt with by either adding it as a dep yourself or running dedupe.
----
The bad news is that while more and more libraries are moving to npm the underlying reason for many being on npm is for the many users using tools like browserify and webpack. So the assumption of many libraries is that when you use npm for client-side libraries is you are using it in a node-style/CommonJS way where require is available other npm dependencies can be used through it and you're not including things in the traditional way of libraries being run one after another on the page in the global scope like in ResourceLoader.
It's probably going to get harder and harder to reconcile deps shared between extensions and/or use library code itself without having node installed on servers. For that matter half the advancements in the css space probably won't be making their way to php either.
Though I do have some hints to things that might open up some possibilities.
Firstly browserify can build standalone versions of libraries. It's possible you could make tweaks to that pattern and have a development build step that would convert npm packages into modules built to run in a new type of ResourceLoader that would probably be a hybrid of the current RL and npm. However you would need to take care to exclude things or else you'll end up with a script that will duplicate the source of a bunch of other modules. You also have to be wary of handling the cases where npm intentionally duplicates a library.
This actually may be even more difficult than it sounds due to npm packages that require modules inside a package rather than just the main module. You might need to go even deeper into browserify, substituting one of the last steps so it outputs a different format with objects that have metadata on the individual modules intended for use in a special loader instead of as a plain script tag.
Current RL on its own just won't work for this. Globals don't work well in the module pattern and so the ability to get a result from require() and allow npm modules to require() others are necessary. Circular requires also need consideration.
Whether you want a RL api that has a bridge to the npm ecosystem or want to go all-in to the modern js landscape and build a more CommonJS/node/npm-like RL api that loads things according to MW's use cases will be the important bike-shedding.
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
On 2015-11-05 8:32 AM, Trevor Parscal wrote:
The flat approach to NPM is a game changer for us, and a Bower killer.
Timo? Had a lot of insight at the time, I'd like to see him be invoked in this decision. Any thoughts Timo?
- Trevor
On Thursday, November 5, 2015, Jon Robson jrobson@wikimedia.org wrote:
It's been a year now, over 3-6 months. Volker can this be given a priority in the next frontend standards meeting. I think the lack of any standard is extremely damaging to the project.
On Wed, Jul 22, 2015 at 4:21 PM, Bryan Davis <bd808@wikimedia.org javascript:;> wrote:
On Wed, Jul 22, 2015 at 12:24 PM, Daniel Werner <daniel.a.r.werner@gmail.com javascript:;> wrote:
Hey,
I just wanted to check in about the status of enabling JavaScript
package
management usage in MediaWiki. I am basically talking about an
equivalent
for JS to what we have with Composer for PHP.
Real-world example: The "data-values/value-view" package[0] is defining "jquery.event.special.eachchange.js": ValueView/lib/jquery.event/jquery.event.special.eachchange.js
Now, recently I needed the same functionality in one of my
extensions, so
I just copied it over. [1]
I know that this is the worst way one could do this, but as far as I can see we don't have that much of a choice right now. Here are the
alternative
options I can see:
Moving "jquery.event.special.eachchange.js" out of the "data-values/value-view" package into its own "WMDE/jquery-eachchange" package...
- ... and using it in my extension via composer.
- pro: two or more extensions or other packages requiring this package
will still result in having only one MW-wide installation..
- con: requires MW specific code which is actually not related to the
MW-independent package to feed the resource loader.
- con: using Composer to manage pure JavaScript packages! Uuuh, ugly!
- ... and having a build step in other packages using the package,
pulling
the "WMDE/jquery-eachchange" somewhere into the file structure of the packages/extensions using it.
- pro: don't need to abuse composer, we can use "npm", "Bower" or any
other arbitrary JS package manager here.
- con: got to tell resource loader somehow... (didn't think so much
about
that yet)
- con: if more than one extensions or other packages require this
package
we still end up with the same code twice or more often in one MW installation.
- Combining 1 and 2: Start with 2, using a JS package manager. Then
going
to 1, creating a composer package and pulling the
"WMDE/jquery-eachchange"
package in via some build script.
- pro: The two pros from 1 + 2
- pro: ^^
- con: still got to tell resource loader somehow...
- con: Overhead; We now create two packages where the Composer one is
just a bridge to the MW-world, still polluting packagist.org. Still
kind of
ugly and more effort for publishing a package and therefore potentially scaring programmers away from doing so since they've got better things
to
do than doing work that could be automated.
I have not seen Approach 2 and 3 yet. Though I could imagine that the VisualEditor team has used something like that. Approach 1 is the way the "data-values/value-view" package itself is
being
handled. And that package should actually be a MW independent pure JS package but right now it contains MW specific code and uses composer for distribution! There is still another option but that had to be properly implemented:
- Choose one native JS package manager for now and go with it (add
support
for others later perhaps). Integrate it properly with MW (resource
loader
to begin with), document how to use it and finally distribute JS code coming from the MW world but useful for other projects in a way where it can actually be used in a non-MW context.
This has already been bugging me when working on Wikidata. Now I'd like
to
reuse some of the code I have written there without spending hours and hours with option 3 because there should be support for option 4 rather sooner or later. So I am wondering; Does anyone have any thoughts, any alternatives
perhaps
or is there any roadmap on anything like the option 4 that I have shown?
Cheers, Daniel
[1]:
https://github.com/DanweDE/mediawiki-ext-UserBitcoinAddresses/blob/master/re...
Option 4 was discussed last October as part of the Librarization project [0]. At the time the front end standards group wasn't ready to pick a winner in the javascript packaging landscape. They did want to revisit that in 3-6 months however so maybe it is time to find someone to look into the various options and their pros and cons again?
[0]: https://phabricator.wikimedia.org/T807
Bryan Davis Wikimedia Foundation <bd808@wikimedia.org
[[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA irc: bd808 v:415.839.6885 x6855
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org javascript:; https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org javascript:; https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l