Trevor Parscal wrote:
If you are really doing a JS2 rewrite/reorganization, would it be possible for some of us (especially those of us who deal almost exclusively with JavaScript these days) to get a chance to ask questions/give feedback/help in general?
I've mostly been working on analysis and planning so far. I made a few false starts with the code and so ended up planning in a more detailed way than I initially intended. I've discussed various issues with the people in #mediawiki, including our resident client-side guru Splarka.
I started off working on fixing the coding style and the most glaring errors from the JS2 branch, but I soon decided that I shouldn't be putting so much effort into that when a lot of the code would have to be deleted or rewritten from scratch.
I did a survey of script loaders in other applications, to get an idea of what features would be desirable. My observations came down to the following:
* The namespacing in Google's jsapi is very nice, with everything being a member of a global "google" object. We would do well to emulate it, but migrating all JS to such a scheme is beyond the scope of the current project.
* You need to deal with CSS as well as JS. All the script loaders I looked at did that, except ours. We have a lot of CSS objects that need concatenation, and possibly minification.
* JS loading can be deferred until near the </body> or until the DOMContentLoaded event. This means that empty-cache requests will render faster. Wordpress places emphasis on this.
* Dependency tracking is useful. The idea is to request a given module, and all dependencies of that module, such as other scripts, will automatically be loaded first.
I then looked more closely at the current state of script loading in MediaWiki. I made the following observations:
* Most linked objects (styles and scripts) on a typical page view come from the Skin. If the goal is performance enhancement, then working on the skins and OutputPage has to be a priority.
* The "class" abstraction as implemented in JS2 has very little value to PHP callers. It's just as easy to use filenames. It could be made more useful with features such as dependency tracking, better concatenation and CSS support. But it seems to me that the most useful abstraction for PHP code would be for client-side modules to be multi-file, potentially with supporting PHP code for each module.
* Central registration of all client-side resources in a global variable would be onerous and should be avoided.
* Dynamic requests such as [[MediaWiki:Handheld.css]] have a large impact on site performance and need to be optimised. I'm planning a new interface, similar to action=raw, allowing these objects to be concatenated.
The following design documents are in my user space on mediawiki.org:
http://www.mediawiki.org/wiki/User:Tim_Starling/CSS_and_JS_caller_survey_(r56220) - A survey of MW functions that add CSS and JS, especially the terribly confusing situation in Skin and OutputPage
http://www.mediawiki.org/wiki/User:Tim_Starling/JS_load_order_issues_(r56220) - A breakdown of JS files by the issues that might be had in moving them to the footer or DOMContentLoaded. I favour a conservative approach, with wikibits.js and the site and user JS staying in the <head>.
http://www.mediawiki.org/wiki/User:Tim_Starling/Proposed_modularisation_of_client-side_resources - A proposed reorganisation of core scripts (Skin and OutputPage) according to the MW modules they are most associated with.
The object model I'm leaning towards on the PHP side is:
* A client-side resource manager (CSRM) class. This would be responsible for maintaining a list of client-side resources that have been requested and need to be sent to the skin. It would also handle caching, distribution of incoming dynamic requests, dependencies, minification, etc. This is quite a complex job and might need to be split up somewhat.
* A hierarchy of client-side module classes. A module object would contain a list of files, dependencies and concatenation hints. Objects would be instantiated by parent classes such as skins and special pages, and added to the CSRM. Classes could be registered globally, and then used to generate dynamic CSS and JS, such as the user preference stylesheet.
* The module base class would be non-abstract and featureful, with a constructor that accepts an array-based description. This allows simple creation of modules by classes with no interest in dynamic script generation.
* A new script loader entry point would provide an interface to registered modules.
There are some design decisions I still have to make, which are tricky due to performance tradeoffs:
* With concatenation, there is the question of which files to combine and which to leave separate. I would like to have a "combine" parameter which is a string, and files with the same combine parameter will be combined.
* Like Wordpress, we could store minified and concatenated files in a public cache and then link to that cache directly in the HTML.
* The cache invalidation scheme is tricky, there's not really an ideal system. A combination of cache-breaking parameters (like Michael's design) and short expiry times is probably the way to go. Using cache-breaking parameters alone doesn't work because there is referring HTML cached on both the server and client side, and regenerating that HTML periodically would be much more expensive than regenerating the scripts.
Here are my notes:
* Concatenation * Performance problems: * Changing inclusions. When inclusions change, whole contents has to be sent again. * BUT people don't change skins very often. * So combine=all=skin should save time for most * Expiry times have to be synchronised. Take the minimum expiry of all, and force freshness check for all. * Makes the task of squid cache purging more difficult * Defeats browser concurrency
* Performance advantages: * For dynamic requests: * Avoids MW startup time. * Avoids DoSing small servers with concurrent requests. * For all requests: * Reduces squid CPU * Removes a few RTTs for non-pipelining clients * Improves gzip compression ratio
* Combine to static file idea: * Pros: * Fast to stream out, on all systems * Doesn't break HughesNet * Cons: * Requires splitting the request into static and dynamic * Need webserver config to add Expires header and gzip
With some help from Splarka, I've determined that it would be possible to merge the requests for [[MediaWiki:Common.css]], [[MediaWiki:Skinname.css]], [[MediaWiki:Handheld.css]] and [[MediaWiki:Print.css]], using @media blocks for the last two, for a significant performance win in almost all cases.
Once the architectural issues have been fixed, the stylistic issues in both ancient JS and the merged code will have to be dealt with, for example:
* Poorly-named functions, classes, files, etc. There's a need for proper namespacing and consistency in naming style.
* Poorly-written comments
* Unnecessary use of the global namespace. The jQuery style is nice, with local functions inside an anonymous closure:
function () { function setup() { ... } addOnloadHook( setup ); }();
* Unsafe construction of HTML. This is ubiquitous in the mwEmbed directory and there will be a huge potential for XSS, as soon as user input is added. HTML construction with innerHTML can be replaced by document.createElement() or its jQuery equivalent.
* The identity crisis. The whole js2 concept encourages code which is poorly integrated with the rest of MediaWiki, and which is written without proper study of the existing code or thought to refactoring. It's like SkinTemplate except with a more pretentious name. I'd like to get rid of all instances of "js2", to move its scripts into other directories, and to remove the global variables which turn it on and off. Also the references to MetavidWiki and the mv prefixes should be fixed.
* Lack of modularisation. The proposed registration system makes it possible to have extensions which are almost entirely client-side code. A module like libClipEdit could be moved to its own extension. I see no problem with extensions depending on other extensions, the SMW extensions do this with no problems.
A few ideas for cool future features also occur to me. Once we have a system set up for generating and caching client-side resources, why not:
* Allow the user to choose a colour scheme for their wiki and automatically generate stylesheets with the appropriate colours.
* Include images in the system. Use GD to automatically generate and cache images with the appropriate anti-aliased background colour.
* Automatically create CSS sprites?
-- Tim Starling