Hi everyone,

We would appreciate your advice on our upcoming research study of image load times on Media Viewer. 

Here are proposed goals, questions and outcomes for this study. They are presented for discussion purposes, not as a prescriptive requirement - and will be adjusted based on your feedback.

The goal of this study is to determine whether or not Media Viewer is loading images fast enough for the majority of our users in most common situations. 

As a typical user of the Media Viewer, I want images to load quickly, in just a few seconds, so I don't have to wait a long time to see them.

Here are our recommended performance targets for image load times by connection speed, to match user expectations on the Web:
* 1-2 seconds for a medium-size image on a fast connection
* 2-3 seconds for the same image on a medium connection
* 5-8 seconds for the same image a slow connection

If tracking connection speeds is too hard in our time-frame, we could base our performance targets on image size instead. For example:

* 1-2 seconds for a small-size image on a medium connection
* 2-3 seconds for medium-size image on the same connection
* 5-8 seconds for large-size image on the same connection

Definitions:

* Image load time = the number of seconds from when you click on a thumbnail to when you see the full image

* Image size:  large = over 2Mb, medium = 1 to 2Mb, small = under 1Mb

* Connection speed: fast = over 256 Kbs, medium = 64 to 256 Kbs, slow = under 64 Kbs

The above numbers are for discussion purposes, and can be adjusted based on your feedback.

Here are the main research questions we propose to answer about image load performance.

(image load = total time from thumbnail click to full image display)

a. by image size: 

    load times for large images? medium images? small images?

b. by web site: 

    load times for mediawiki.org? commons? enwiki? frwiki? huwiki? other sites?

c. by connection speed: (optional)

    load times for fast connections? medium connections? small connections? (this may not be feasible in our time frame)

This question could be answered by storing the timestamp for thumbnail clicks, as well as the timestamp for the full image display, then log the difference. 

We would then prepare different bar graphs for each condition set above, with categories on the vertical axis, and number of seconds on the horizontal axis. The graphs could be based on data from the last 7 days.

This question could be answered by preparing different histograms, with number of images on the vertical axis, and number of seconds on the horizontal axis (deciles).

Here are possible outcomes from this study:

Outcome 1: Favorable - e.g.: 80% of images load quickly 
Action: Go ahead with current release plan to deploy Media Viewer everywhere by default.

Scenario 2: Neutral - e.g.: 50% of images load quickly 
Action: Go ahead with current release plan, but deploy Media Viewer as an opt-in feature on wikis that don’t want it by default 

Scenario 3: Unfavorable - e.g.: 20% of images load quickly 
Action: Revisit release plan: consider making this opt-in everywhere — or work on faster image load solutions.

We would be grateful for your comments on this, so we can refine our plans before we start this study next week. Please let us know which metrics above seem most important, given that we only have a few developer days to collect and analyze a few key metrics in coming weeks, to determine if we are meeting our objectives. Some related links are included below, for your convenience.

To end on a positive note, we just deployed yesterday a new version of Media Viewer that is much faster, thanks to all the fine work from our development team. This morning, I looked at a variety on 'non-popular' images on enwiki today, and the Media Viewer experience was quite good overall. Most images load within the 2 second maximum which we recommend for a ‘fast’ connection — and this was a home wifi connection. I realize this is completely anecdotal, and not supported by hard data, so we can’t make any decisions about it. But it makes me hopeful that we are getting close to our objectives. Even compared to large commercial sites like Flickr, we hold up pretty well on this computer. :)

Thanks for your interest in this project.

All the best,

Fabrice

* Media Viewer Release Plan:

* First Media Viewer Metrics:

http://multimedia-metrics.wmflabs.org/dashboards/mmv Metrics

* Media Viewer Test Page:

* Metrics Tasks under consideration (Mingle):

* Next Development Cycle (Mingle):

http://ur1.ca/gtvvr