Hi Gilles,

Thanks so much for doing this important study, which gives us a better idea on the overall image load performance of Media Viewer compared to the current file pages.

I agree with you that a 25% increase in load time between these two solutions on a warm cache is not a show-stopper for this release -- considering that we are displaying larger images and providing a more immersive experience that can make up for this slight delay. 

But I am more concerned with the 200% increase in load time on a cold cache, particularly if it happens often. I encourage us to look at practical solutions to address this issue sooner rather than later — whether it is your proposal to get rid of mmv.bootstrap entirely, caching the results of our API requests across users, or other solutions.

These issues will be particularly important in world regions where low bandwidth / slow connections are more prevalent, so we will want to monitor performance closely for those sites. If we can identify early on projects that might be impacted the most by this issue, we can make those communities aware of the trade-off between richer experience and image load time. 

In any case, many thanks for these useful test results, and I look forward to learning more together in coming days :)


Fabrice

On Apr 9, 2014, at 4:51 AM, Gilles Dubuc <gilles@wikimedia.org> wrote:

Looking at the timing waterfall, I think that the answer is simply the thumbnail info API call, for which Gergo has a WIP solution: https://gerrit.wikimedia.org/r/#/c/124796/

Basically at the moment Media Viewer (unlike the File: page) has to get the results of a PHP API call before it knows the URL of the image to load. My hunch, seeing that on my machine the API calls can take up to a full second, is that the 25% overhead will go away entirely when that improvement gets merged.




On Wed, Apr 9, 2014 at 1:28 PM, Gilles Dubuc <gilles@wikimedia.org> wrote:
Hi,

Cold cache concerns aside (see the other email I just sent), I now have preliminary performance figures comparing Media Viewer to the status quo (File: page) in production. I'm still working with QA on getting our test merged and running automatically on cloudbees. The results I have were performed on a 2MB DSL connection in France, with nothing else running on the network.

Since we don't collect browser resolutions in our own stats (that I could find) I picked "average" and "large" desktop browser window sizes based on 3rd-party "internet-wide" statistics I could find online. These statistics differentiated desktop and mobile, so mobile's lower resolutions didn't affect the average. The "small" size was picked specifically to hit a bucket size identical to the File: page's image.

On an small browser window size (900x700), Media Viewer displays the image in a little less than 125% of the time it takes to display the image on the File: page

On an average browser window size (1366x768), Media Viewer displays the image in a little less than 125% of the time it takes to display the image on the File: page

On a large browser window size (1920x1080) I found the same result, but I suspect it might not be true, because my desktop wasn't big enough to accommodate that size in selenium's Firefox. I think we'll need to wait for cloudbees to run the tests headless for the real figure.

For the sake of argument, let's look at the difference in file size between the file page and the "average" browser size:
- on the File: page, the image displayed is 800x600 (480,000 pixels) and weighs 52.7kb
- on Media Viewer, at the "average" browser resolution, the image displayed is 1024x768 (786,432 pixels) and weighs 79.6kb

Therefore, the Media Viewer image has 63.8% more pixels on the screen and weighs 51% more.

But, this is only a consolation prize, because the small browser window size (900x700) which serves the same 800x600 image as the File: page still takes 20-25% longer, which seems to indicate that Media Viewer's overhead isn't due to the image size. Basically, if we made the File: page serve a 1024x768 image, it would certainly still beat Media Viewer.

We've already done multiple rounds of optimizing the performance of Media Viewer, but I'll investigate further to see if there's anything obvious that makes Media Viewer slower than opening the File: page.

In my opinion between 20 and 25% of overhead isn't a show-stopper for launch, considering that Media Viewer serves a bigger image in most situations, but it's worth giving optimizing one last try to see if we've missed anything. However, it could very well be that we can't do better and that the overhead is due to doing adding the image dynamically with JS instead of it being present in the pageload DOM of a relatively lightweight page.

_______________________________________________
Multimedia mailing list
Multimedia@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/multimedia

_______________________________

On Apr 9, 2014, at 3:03 AM, Gilles Dubuc <gilles@wikimedia.org> wrote:

Hi,

Since mediawiki.org finally moved to '21, I was able to run the performance tests against production again. We now have a figure for how bad the cold cache experience can be. On an average-sized browser window, the cold cache image load in media viewer takes around 200% of the time it takes for the image to appear on the File: page with a warm cache*. And it takes around 230% of the time for a large browser window. We're talking seconds of difference, that's a long time to wait.

When we first built mmv.bootstrap (the on-demand loading of JS), we didn't have the click catcher (mmv.head). Now that we do, I think we can afford to sacrifice some user bandwidth and just load Media Viewer's JS with the async attribute (which means it wouldn't block page rendering). And mmv.head would take care of catching and replaying clicks that happened before Media Viewer's JS was loaded.

Doing that means we could get rid of mmv.bootstrap entirely, increase the cold cache performance (chances are, the thumbnails on a given page will take longer to load than Media Viewer's JS and CSS) and simplify our code.

Any objections to doing this?

*the reason why we're not comparing figures with the File: page on a cold cache is that it's an unfair comparison: all of mediawiki's JS and CSS would be cold in that case, whereas for Media Viewer it's only Media Viewer's JS and CSS that is cold.
_______________________________________________
Multimedia mailing list
Multimedia@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/multimedia