Trevor Parscal wrote: I might be a little tired but are you saying that debug mode does not preserve original line numbers? So this is debug mode as in "a bit easier to debug in Firebug then fully minified code"? And I still don't see how minification options are evil. One "if" won't really change performance. Regards, Nux.

It's not a matter of performance, it's a matter of complexity. More configurable does not always mean better. - Trevor On 9/29/10 3:46 PM, Maciej Jaros wrote:

That is all in our roadmap - which needs to be updated and put onto the wiki. Ideally this will happen next week when Alolita returns from Paris. - Trevor On 9/29/10 4:27 PM, MZMcBride wrote:

On 30/09/10 08:27, Trevor Parscal wrote: I made that change because it was requested by multiple people in discussions before the resource loader was implemented. It's not because I actually had any actual problem with debugging, I'm well aware of the existence of the debug mode. Concerns were raised that it may be necessary to interpret minified code in cases such as: * Where it is suspected that, due to caching issues, the debug code may not be the same as the minified code. * To debug the minifier itself, which is far short of a full JavaScript parser and may well introduce functional differences. * Where end users report platform-specific JavaScript errors, it may be useful to be able to match the line number of the error with something meaningful. Since the cost of adding line breaks is fairly small, the contention was that it was a fair compromise between size and developer (and tech supporter) sanity. So I took those comments on board and implemented it shortly after the branch merge. -- Tim Starling

2010/9/30 Trevor Parscal <tparscal(a)wikimedia.org>rg>: Or change the URL in some way that doesn't alter the meaning of the URL. There's various ways to get around caching. JSMin causing a bug sounds very unlikely to me, but I agree that we should be paranoid and account for the possibility anyway. Still, I think we can agree that JSMin breaking something should be a very, very rare occurrence (it's been around for a long time and has been used on a large body of code), and that debugging the minifier would be done locally anyway (to be sure caching is not to blame), so line breaks in production won't help. All you'd need to debug JSMin locally without going insane is to be able to preserve line breaks locally, which can be accommodated with a setting (off by default) or even just requiring commenting out of code (as long as it's clear how to do this and not cumbersome, this should be fine). This is false in almost all cases. Because we make our requests as cacheable as possible, the vast majority of RL responses will *not* depend on user preferences, or in fact on anything not in the URL. For these requests, you'll be able to pull up the same URL that the user is pulling and see the exact same thing (provided you both clear your caches). User preferences and other factors may be used to *build* the URL, so whether you see the issue may depend on your preferences, but once you find out which URL the user is getting their stuff from you'll have a meaningful line number. There should only be one request whose contents depend on user preferences: the one getting the user's preferences plus the user scripts. The former is really just one call to a setter with a JSON-encoded preferences map, so it consists of one simple statement; inspecting that is doable. The rest is the user's own user JS, which can contain almost anything. This is the only thing you won't reasonably be able to debug based on just a line number, but then there's lots of other easy ways to nail down user JS as being the culprit. Per the above this is not entirely true, but the one case in which they are useful (culprit is in user JS+prefs request and all you have is a line number) doesn't warrant preserving line breaks IMO. I agree that we should get our numbers first, but some preliminary playing around seems to indicate Tim's right about this one. Roan Kattouw (Catrope)

On 9/30/10 9:31 AM, Trevor Parscal wrote: OK, now I've calculated it... On a normal page view with the Vector skin and the Vector extension turned on there's a 2KB difference. On an edit page with the Vector skin and Vector and WikiEditor extensions there's a 4KB difference. While adding 2KB to a request for a person in a remote corner of the world on a 56k modem will only add about 0.3 seconds to the download, sending 2,048 extra bytes to 350 million people each month increases our bandwidth by about 668 gigabytes a month. I don't know what that kind of bandwidth costs the foundation, but it's not free.

On 9/30/10 1:55 PM, Maciej Jaros wrote: There's this assumption here that when a bug is reported that a JavaScript developer is going to be lost without a line number. My experience is that when bugs are reported, the first step is reproduction, and then investigation. If we shouldn't be relying on users with little or no expertise to be participating in investigation. Also, error reporting with Firebug and most browsers is much more verbose than "error on line ##". Compressed of course. It would be 5-10 times that much without compression. This is what debug mode is for. Developers. Developers should always use debug mode while debugging. Switching back and forth is not necessary for every tiny change, just a good thing to verify before checking in code. It's not a matter of me thinking our code is so good it won't break. It's a matter of wanting to send fully optimized code to clients, not partially optimized code. I agree that something will always break - I just disagree that there's much merit in your approach to solving this. If we are talking about allowing the ResourceLoader to be configurable with LocalSettings.php adjustments, then you are already assuming a level of expertise about the developer debugging the problem that these middle-ground approaches will be useless. If we are talking about adding yet another URL parameter to be able to turn minification off, or otherwise customize the processing of the modules beyond turning debug on and off, I think you are adding complexity without reasonable benefit. - Trevor

At 2010-09-30 23:13, Trevor Parscal wrote: It's not an assumption. I AM one of the main developers working on bugs reported by users on pl.wiki. And yes line number is very helpful if accurate. Also note that I was picturing a situation where I have to debug the minified code when I don't get the error in my browser(s) at hand. Usually the problem is some extension/plug-in, but I can't tell unless I investigate. And reading minified code at it's current state is impossible for me. I understand the need to minify code I really do. I just don't get it why do you want to minify this that much. HTML which is usually served at every page load is not minified. Further more even minified HTML can be readable for debuging purposes as every debugger (inspector) shows the structure of the code rather then a code pulp. Also why are there are explicitly added comments like "<!-- /navigation -->". People don't need comments and whitespace to read code, right? Also I have a PHP minifier if you are interested. Let's serve Mediawiki encode and if some developer want to debug the code he can always switch folders. This is impossible. Fully optimized code would only contain one byte per variable name. Yo need to make sacrifices. And this seems to be needed as shown before and above. Well we differentiate on that. I'm wondering of what is you scenario on debugging the minified code. Regards, Nux.

At 2010-10-01 01:50, Trevor Parscal wrote: I'm very productive with inspecting code with my eyes. Ask my boss ;-). Two reasons probably: 1. The gain is not big for average user (loading big JS is only painful at first visit in a day or even more). 2. Wikipedia is not Google (you go to a Google's website and use it; you go to Wikipedia and change it). If I could be honest I would say I hope no one will ever have time to implement it. I'm asking you how are you going to cope with a situation in which some brand new, minified code is buggy. How are you going to fix it? Two scenarios: you are able to reproduce this only in Opera Mobile, you are only able to reproduce it in IE6. Regards, Nux.

On 9/30/10 6:04 PM, MZMcBride wrote: The front end that we have *now* is fast for some people. The point of writing a new platform is to be able to do more, if we want. I think we have a large difference of vision here. By contemporary standards the amount of JS in the UsabilityInitiative, and related projects, is a mere trifle. Our competitors invest a lot in Javascript infrastructure because they find that every last efficiency is usually worth it. If stripping newlines gives us 1K more per page, we can squeeze 1K more of usability or other interface niceties in. Now, we might decide not to do certain kinds of minification for other very good reasons. But I say this to put the Resource Loader in context. The point here, at least as I see it, is to bring Wikimedia sites up to the standards of the rest of the web, not to "clean up" some "mess". I agree this is a very good reason why we should hesitate before doing anything that would obfuscate our pages. "View Source" doesn't work as well on the web any more, and it should on Wikipedia. (I personally would set the balance at a different point -- I'd like there to be a note right in the page source explaining how to view an unoptimized version.) That said, your assertion that Google doesn't allow customizations of its page is just not true. Google has offered "skins" for something like a year or more now, and their home page widget platform is literally called Gadgets. http://www.google.com/ig/ The great grandaddy of all site-Javascript modifiers is GreaseMonkey, and the guy that wrote that is even a Google employee (Aaron Boodman). Mihai Parparita, another Google employee, circulates a very popular set of Greasemonkey improvements to Gmail. So, the argument that Google etc. just don't get developer hacks, or that they do them badly, is quite wrong. Their platforms for doing this are far, far more successful than ours. We should be learning from them on this, not vice versa. Here's how they do it: Google guarantees that certain Javascript functions or DOM objects are reliably available for use and modification, and others are not. So they use minification for efficiency while explicitly preventing some things from being altered (that's a feature of Google's Closure Compiler). This is a story as old as computing; to enable a community, create a rich platform with a stable public API. Merely leaving the pages open to any modification is actually worse for gadget writers, since anything could change. Anyway, I think that's somewhat in the flavor of what the Resource Loader people are trying to achieve here. Efficiency *and* community. Minification *and* openness. Gadget-writers are a big part of their targeted use cases. Otherwise, they would have just used something off the shelf. There are a lot of good JS libraries out there, but none that quite fit the needs of our community. -- Neil Kandalgaonkar (| &lt;neilk(a)wikimedia.org&gt;

On 9/30/2010 10:03 PM, Neil Kandalgaonkar wrote: As a gadget developer, I would love that. However, that seems to be, in part, the opposite of what's happening. We're taking a public API that's been used for years (wikibits.js) and deprecating the whole thing, with no documentation (that I can find in a couple minutes of searching) for the replacements. When the new skin/editor was rolled out, there was basically no documentation for anything and there still isn't very much. When I rewrote the reftoolbar gadget[1] to support the new toolbar, in advance of the rollout, I had to do it by reading the Wikieditor code and testing on a personal sandbox wiki running trunk, which changed often and was frequently broken. Now I'm probably going to have to do that again to make sure it won't break with ResourceLoader. That's not a very good development process. That's all great, but if you're going to go that route, you have to make damn sure you get everything right, else we're going to end up with a bunch of frustrated gadget developers and a ton of angry users with broken gadgets. [1] http://en.wikipedia.org/wiki/Wikipedia:RefToolbar_2.0 -- Alex (wikipedia:en:User:Mr.Z-man)

On 01/10/10 04:35, Trevor Parscal wrote: We don't pay by volume (GB per month), we pay by bandwidth (megabits per second at the 95th percentile). They should be roughly proportional to each other, but to calculate a cost we have to convert that 668GB figure to a percentage of total volume. I took this graph: http://www.nedworks.org/~mark/reqstats/trafficstats-monthly.png And I used the GIMP histogram tool to integrate the outgoing part for 30 days between week 34 and week 37. The result was 31,824 pixels of blue and 20,301 pixels of green, which I figure is about 2113 TB/month. So on your figure, the cost of adding line breaks would be about 0.03% of whatever the bandwidth bill for that month is. I don't have that number to hand, but I suspect 0.03% of it is not going to be very much. For 2009-2010 there was a budget of about $1M for "internet hosting", of which bandwidth is a part, and 0.03% of that entire budget category is only $25 per month. I think your 668GB figure is too low, because current uniques is more like 390M per month, and because some unique visitors will request the JS more than once. You can double it if you think it would help you make your case. Developer time is not free either. -- Tim Starling

This is getting a little out of hand? People are going to spend more time talking about the potential for minification errors or sub optimisation cost criteria then we will ever actually be running into real minification errors or any real readability issue. Reasonable efforts are being made to make the non-minified versions easily accessible. We can add additional comment to every outputted file that says request me with ?debug=true to get the source code version, add a user preference, add a cookie, and a view non-minfied source link to the bottom every page... But try to keep in mind the whole point of minification is identical program execution while minimising the file size! If we are to optimise with random adjustments for "readability" we are optimising in two opposite directions, and every enhacment in the direction of optimized pacakge code delivery could potentially go against the 'readability' optimisation. We are already commited to supporting two modes one that optimized readability with raw source code delivery and one that is optimised for small packaged delivery. No sense in setting readability as criteria for the packaged mode since the 'readability' mode will always do 'readability' better. --michael On 10/01/2010 12:38 AM, Trevor Parscal wrote:

On Thu, Sep 30, 2010 at 8:27 AM, Trevor Parscal &lt;tparscal(a)wikimedia.org&gt; wrote: There are two features here. modules optimising traffic It seems everyone realises that the serving of raw files has to remain. Why is this old and remaining functionality going to be named 'debug mode'? Why not give the new feature a name, put it in the prefs, and turn that pref on by default after sufficient road testing. As happened with the new skin. There are going to be mediawiki implementers and wikimedia users who want to disable this feature for reasons other than debugging. (I've always considered the 'Misc' section of the prefs to be 'Performance') -- John Vandenberg

W dniu 2010-10-21 20:01, Trevor Parscal pisze: What about setting a cookie to go into a debug mode? I thought you liked the idea? Regards, Nux.

@2010-10-22 00:53, Trevor Parscal: Sure. I see new line characters are there, too. Cool! Nux.