On Fri, Jul 11, 2014 at 2:24 PM, Federico Leva (Nemo) <nemowiki@gmail.com> wrote:

This does make me wonder, what do those graphs *actually* measure? Are
they really measuring a comparable/representative set of image requests
from which we can infer that one or the other method is intrinsically
faster than the other, or are they measuring different things e.g. for
some selection bias?

By "one or the other" you mean image load time for varnish hits vs. image load time for varnish misses (on-demand generated thumbnails)?

We are measuring all MediaViewer image loads (with some random sampling to keep the volume sane), and check the X-Varnish headers to tell if it was a hit or a miss.

I see no potential for selection bias there.