If you have been following the svn commits you may have noticed a bit of activity on the js2 front.
I wanted to send a quick heads up that describes what is going on and invite people to try things out, and give feedback.
== Demos ==
The js2 extension and associated extension are ruining on sandbox-9. If you view the source of a main page you can see all the scripts and css and grouped into associated buckets:
http://prototype.wikimedia.org/sandbox.9/Main_Page
I did a (quick) port of usabilityInitiative to use the script-loader as well. Notice if you click "edit" on a section you get all the css and javascript, localized in your language and delivered in a single request. ( I don't include the save / publish button since it was just a quick port )
Part of the js2 work included a wiki-text parser for javascript client side message transformation: http://prototype.wikimedia.org/s-9/extensions/JS2Support/tests/testLang.html
There are a few cases out of the 356 tests were I think character encoding is not letting identical messages pass the test and a few transformations that don't match up. I will take a look at those edge cases soon.
The Multimedia initiative ( Neil and Guillaume's ) UploadWizard is a js2 / mwEmbed based extension and also enabled on in that wiki as well: http://prototype.wikimedia.org/sandbox.9/Special:UploadWizard
The js2 branch of the OggHandler includes Transcode support ( so embed web resolution oggs when embed at web resolution in pages ) This avoids 720P ogg videos displayed at 240 pixels wide inline ;) http://prototype.wikimedia.org/sandbox.9/Transcode_Test
The TimedMediaHandler of course include timed text display support which has been seen on commons for a while http://bit.ly/aLo1pZ ... Subtitles get looked up from commons when the repo is shared:: http://prototype.wikimedia.org/sandbox.9/File:Welcome_to_globallives_2.0.ogv
I have been working with the miro universal subtitles efforts so we should have an easy interface for people to contribute subtitles with soon
Edits pages of course include the add-media-wizard which as been seen as a remote http://bit.ly/9P144i for some time also now also works as an extension
== Documentation ==
Some initial JS2 extension is in extensions/JS2Support/README Feedback on that documentation would also be helpful.
--michael
On 2010-05-14 05:44, Michael Dale wrote:
If you have been following the svn commits you may have noticed a bit of activity on the js2 front.
I wanted to send a quick heads up that describes what is going on and invite people to try things out, and give feedback.
== Demos ==
The js2 extension and associated extension are ruining on sandbox-9. If you view the source of a main page you can see all the scripts and css and grouped into associated buckets: [...]
So does this extensions encrypt JS files into being non-debugable? I could understand that on sites like Facebook but on an open or even Open site like Wikipedia/Mediawiki? This just seems to be wrong. Simple concatenation of files would serve the same purpose in terms of requests to the server.
Regards, Nux.
On Mon, May 17, 2010 at 6:43 PM, Maciej Jaros egil@wp.pl wrote:
So does this extensions encrypt JS files into being non-debugable? I could understand that on sites like Facebook but on an open or even Open site like Wikipedia/Mediawiki? This just seems to be wrong. Simple concatenation of files would serve the same purpose in terms of requests to the server.
At the very least, newlines should be preserved, so you can get a line number when an error occurs. Stripping other whitespace and comments is probably actually be worth the performance gain, from what I've heard, annoying though it may occasionally be. Stripping newlines is surely not worth the added debugging pain, on the other hand. (Couldn't you even make up for it by stripping semicolons?)
The script-loader has a few modes of operation.
You can run it in "raw file mode" ( ie $wgEnableScriptLoader = false ). This will load all your javascript files directly only doing php requests to get the messages. In this mode php does not touch any js or css file your developing.
Once ready for production you can enable the script-loader it groups, localizes, removes debug statements, transforms css url paths, minifies the set of javascript / css. It includes experimental support for google closure compiler which does much more aggressive transformations.
I think your misunderstanding the point of the script-loader. Existing extensions used on wikipedia already do a static package and minify javascript code.
If you want to have remote user communicate javascript debugging info, we could add a url flag to avoid minification and or avoid script-grouping. Maybe be useful in the case of user-scripts / gadgets.
But in general its probably better / easier for end users to just identify their platform and whats not working, since its all code to them anyway. If they are a developer or are going to do something productive with what they are seeking they likely have the code checked out locally and use the debug mode.
--michael
Aryeh Gregor wrote:
On Mon, May 17, 2010 at 6:43 PM, Maciej Jaros egil@wp.pl wrote:
So does this extensions encrypt JS files into being non-debugable? I could understand that on sites like Facebook but on an open or even Open site like Wikipedia/Mediawiki? This just seems to be wrong. Simple concatenation of files would serve the same purpose in terms of requests to the server.
At the very least, newlines should be preserved, so you can get a line number when an error occurs. Stripping other whitespace and comments is probably actually be worth the performance gain, from what I've heard, annoying though it may occasionally be. Stripping newlines is surely not worth the added debugging pain, on the other hand. (Couldn't you even make up for it by stripping semicolons?)
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 05/18/2010 08:53 AM, Michael Dale wrote:
But in general its probably better / easier for end users to just identify their platform and whats not working, since its all code to them anyway. If they are a developer or are going to do something productive with what they are seeking they likely have the code checked out locally and use the debug mode.
The problem here is Wikimedia sites. If you leave the debug mode off and serve fully minified scripts on WMF sites, the hundreds if not thousands of people who develop user/site scripts for them will have a very hard time debugging anything that involves interactions with the minified code. If you turn debug mode on permanently, you lose most of the benefit of having a minifier to begin with (Wikimedia being the single biggest user of MediaWiki).
Then there's also the problem that the MediaWiki environment on Wikimedia sites can be quite different from a stock install, and duplicating enough of it on a test wiki to be able to reproduce a Wikimedia-specific bug may be quite difficult -- particularly so if you have to do this before you even know where the bug occurs, because you couldn't do any useful debugging due to the code being minified.
I'd like to second Aryeh's suggestion to at least tweak the minifier to leave newlines intact (although collapsing multiple consecutive newlines to one would be OK, I guess). That way users could at least set up useful breakpoints and receive error messages that tell more than which file the problem occurred in.
(Part of the blame here goes to the debuggers, of course. If common JavaScript debugging tools like Firebug could do things like expand minified code and set up breakpoints on individual statements rather than just lines, minification would be much less of a problem for debugging. But currently, as far as I know, they don't.)
2010/5/18 Ilmari Karonen nospam@vyznev.net:
The problem here is Wikimedia sites. If you leave the debug mode off and serve fully minified scripts on WMF sites, the hundreds if not thousands of people who develop user/site scripts for them will have a very hard time debugging anything that involves interactions with the minified code. If you turn debug mode on permanently, you lose most of the benefit of having a minifier to begin with (Wikimedia being the single biggest user of MediaWiki).
I have recently configured test.wikipedia.org to pull all of its JS/CSS locally rather than from bits.wikimedia.org (for better isolation) and serve all UsabilityInitiative JS/CSS in raw, multi-file format (rather than combined and minified, which we use on 'real' wikis); all of this is to aid testing. The script loader could also be configured to serve its JS minified on normal wikis and unminified on testwiki.
Roan Kattouw (Catrope)
On Tue, May 18, 2010 at 08:39, Ilmari Karonen nospam@vyznev.net wrote:
On 05/18/2010 08:53 AM, Michael Dale wrote:
But in general its probably better / easier for end users to just identify their platform and whats not working, since its all code to them anyway. If they are a developer or are going to do something productive with what they are seeking they likely have the code checked out locally and use the debug mode.
The problem here is Wikimedia sites. If you leave the debug mode off and serve fully minified scripts on WMF sites, the hundreds if not thousands of people who develop user/site scripts for them will have a very hard time debugging anything that involves interactions with the minified code. If you turn debug mode on permanently, you lose most of the benefit of having a minifier to begin with (Wikimedia being the single biggest user of MediaWiki).
Then there's also the problem that the MediaWiki environment on Wikimedia sites can be quite different from a stock install, and duplicating enough of it on a test wiki to be able to reproduce a Wikimedia-specific bug may be quite difficult -- particularly so if you have to do this before you even know where the bug occurs, because you couldn't do any useful debugging due to the code being minified.
I'd like to second Aryeh's suggestion to at least tweak the minifier to leave newlines intact (although collapsing multiple consecutive newlines to one would be OK, I guess). That way users could at least set up useful breakpoints and receive error messages that tell more than which file the problem occurred in.
I would support "a url flag to avoid minification and or avoid script-grouping", as suggested by Michael Dale, or even to have a user preference for enable/disable minification in a more permanent way (so we don't need to change the url on each test: we just disable minification, debug the code and then enable it again)
Helder
Helder Geovane wrote:
I would support "a url flag to avoid minification and or avoid script-grouping", as suggested by Michael Dale, or even to have a user preference for enable/disable minification in a more permanent way (so we don't need to change the url on each test: we just disable minification, debug the code and then enable it again)
I like the idea of a user preference. This way you don't constantly have to add &debug to the url, its easy to tests across pages and is more difficult for many people to "accidentally" invoke it.
Committed support for the preference in r66703
--michael
On Thu, May 20, 2010 at 3:20 PM, Michael Dale mdale@wikimedia.org wrote:
I like the idea of a user preference. This way you don't constantly have to add &debug to the url, its easy to tests across pages and is more difficult for many people to "accidentally" invoke it.
It also means that JavaScript developers will be consistently getting served different code from normal users, and therefore will see different sets of bugs. I'm unenthusiastic about that prospect.
Aryeh Gregor wrote:
On Thu, May 20, 2010 at 3:20 PM, Michael Dale mdale@wikimedia.org wrote:
I like the idea of a user preference. This way you don't constantly have to add &debug to the url, its easy to tests across pages and is more difficult for many people to "accidentally" invoke it.
It also means that JavaScript developers will be consistently getting served different code from normal users, and therefore will see different sets of bugs. I'm unenthusiastic about that prospect.
hmm ... if the minification resulted in "different" bugs than the "raw" code that would be a bug with the minification process and you will want to fix that minfication bug. You will want to know where the error occurred in the minfied code.
Preserving new-lines is a marginal error accessibility gain when your grouping many scripts, replacing all the comments with new lines, striping debug lines, and potentially shortening local scope variable names. Once you are going to "fix" an issue you will be fixing it in the "actual" code not the minified output, so you will need to recreate the bug with the non-minified output.
In terms of all-things-being-equal compression wise using \n instead of semicolons consider: a = b + c; (d + e).print();
With \n instead of ; it will be evaluated as: a = b + c(d + e).print();
We make wide use of parenthetical modularization, i.e all the jquery plugins do soemthing like: (function($){ /* plugin code in local function scope using $ for "jQuery" */})(jQuery); Initialization code above line with /\n/, ';' substitution will result in errors.
The use of a script-debug preference is for "user-scripts" development that is hosted "live" and developed on the server wiki pages. Development of core and extension javascript components should be tested locally in both minified and non-minified modes.
peace, --michael
On 05/22/2010 04:36 AM, Michael Dale wrote:
Preserving new-lines is a marginal error accessibility gain when your grouping many scripts, replacing all the comments with new lines, striping debug lines, and potentially shortening local scope variable names. Once you are going to "fix" an issue you will be fixing it in the "actual" code not the minified output, so you will need to recreate the bug with the non-minified output.
The gain would be that most browsers only report the filename (URL) and line number where a JavaScript error occurs. If you group all your scripts into one file with one line, you basically lose _all_ information about _where_ the error happens.
But I do think that the debug mode you added in r66703 is probably an adequate substitute, since, as you note, any bug that only occurs in the combined and minified code counts as a bug in the minifier. At least as long as we actually have people able and willing to promptly fix any such bugs (including any regressions triggered by old/unusual browsers and custom user scripts) ASAP when they're found.
Aryeh Gregor <Simetrical+wikilist <at> gmail.com> writes:
At the very least, newlines should be preserved, so you can get a line number when an error occurs. Stripping other whitespace and comments is probably actually be worth the performance gain, from what I've heard, annoying though it may occasionally be. Stripping newlines is surely not worth the added debugging pain, on the other hand. (Couldn't you even make up for it by stripping semicolons?)
Stripping semicolons is dangerous, see e.g. http://stackoverflow.com/questions/444080/do-you-recommend-using-semicolons-... (not to mention you would have to differentiate between statement separators and argument separators in for constructs)
I agree though that the small bandwidth gain of stripping newlines is not worth the pain. Even if minification can be turned off manually, it would make user reports near useless, and some javascript errors are not easy to reproduce.
Combining files has a negative side effect too: since errors stop thread execution, once something breaks, everything breaks (without combining, execution would have continued on the next file). This could be mitigated somewhat by wrapping code sections in try-catch blocks (syntax errors cannot be caught, but anything else can).
wikitech-l@lists.wikimedia.org