I am somewhat concerned about the implications for JS debugging here. Debugging JS problems with the live sites is already pretty complicated: 1. debug=true won't reproduce some bugs (usually race condition related) 2. Sometimes old code gets cached with the new cache-busting timestamp (due to a race condition between bits and apache at deployment) 3. Sometimes the cache servers cache different code for the same URL, i.e. one cache server will have one version and another cache server will have another 4. In some cases ResourceLoader doesn't change the cache-busting timestamp (if you just remove files or add older files to a module) 5. You always have to face the 5 Minutes of Doom (how frequently ResourceLoader rebuilds the startup manifest)
And that's not even mentioning client-side caching. Shouldn't we work on fixing or mitigating some of these issues before we make JS debugging even more complicated?
Ryan Kaldari
On Sun, Nov 3, 2013 at 5:27 PM, Ori Livneh ori@wikimedia.org wrote:
The roll-out of 1.23wmf2 to your favorite Wikimedia wiki will inaugurate the era of ResourceLoader module storage -- an era which will be marked by terrifying and hilarious new bugs, intermittent client-side failures, and generally inexcusable disruptions to user experience. Sounds exciting? Read on!
What is it?
"Module storage" refers to the use of localStorage in ResourceLoader as an auxiliary to the browser cache. ResourceLoader, remember, is the MediaWiki subsystem that delivers JavaScript and CSS to your browser. One of its primary functions is to minimize the total number of network requests that your browser needs to make in order to render the page, and to make the remaining requests as slim as possible. One of the ways ResourceLoader does this is by making a list of all the different components your browser needs and then transmitting them in bulk.
The downside of this approach is that a small update to MediaWiki's code, touching perhaps one or two components, will often force the browser to throw out (and re-retrieve) a bunch of unrelated modules that did not change and did not ostensibly need to be re-retrieved. This is because the browser cache operates on the level of network requests; it is not intelligent enough to decompose a request into its constituitive parts and to cache these parts separately from one another.
Starting with the 1.23wmf2 branch, ResourceLoader will check if your browser implements localStorage, a mechanism for storing structure data. If it does, ResourceLoader will use it as an auxiliary cache, capable of versioning individual components.
Why?
The primary goals are:
- Destabalize MediaWiki's front-end code, by coupling it to an
inconsistently-implemented and poorly-understood HTML5 API.
- Make debugging fun again, by adding another layer of caching to
MediaWiki. Yes!!
However, as with all new features, unintended side-effects are possible. Specific concerns for module storage include the risk of making the network footprint of page loads smaller, resulting in improved latency.
But seriously,
- Module storage is enabled only if the underlying browser supports
localStorage (~89% of browsers in use, according to http://caniuse.com/#feat=namevalue-storage). Users with older browsers will not see a benefit, but they will not suffer a penalty either.
- Module storage may or may not improve site performance. We need to
test it against a wide range of browsers and platforms to know for certain. If it doesn't improve performance, we'll blame it on you, your poor choice of browsers, and your uncooperative attitude, but then we'll scrap it.
How can I test it?
Glad you asked! Module storage is enabled by default on the beta cluster, and on test & test2 wikis. The simplest way you can help is by being alert to to performance regressions and front-end code breakage. If you know how to use your browser's JavaScript console, you can get stats about module storage usage on the current page by checking the value of 'mw.loader.store.stats'.
When the code is rolled out across the cluster, it will be disabled by default, but you will be able to opt-in by manually setting a cookie in your browser. I will follow up with the exact details. If module storage proves stable enough for production use, we'll seek to asses its utility by running a controlled study of some kind. If we can demonstrate that module storage leads to a significant improvement in performance, we'll enable by it default.
Ori Livneh ori@wikimedia.org
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l