Hi Gilles,
On 04 Jul 2014, at 10:03, Gilles Dubuc <gilles(a)wikimedia.org> wrote:
Hi ops,
Has the swift capacity been increased yet thanks to the new hardware? If so, could we
resume the discussion of "pre"generating specific thumbnail sizes at upload
time?
Media Viewer could benefit greatly from this performance-wise. As seen on this graph, the
launch to all wikis affected the average considerably, since users started hitting a lot
of images that didn't have Media Viewer-sized thumbnails yet:
http://multimedia-metrics.wmflabs.org/dashboards/mmv#overall_network_perfor…
Yes, our Swift capacity expansion has completed. Although the capacity expansion had been
planned with reduced storage for thumbs in mind for the future, work that hasn’t started
yet, we should have space for this available right now and it shouldn’t be problematic
provided that we monitor it well. So please go ahead with this for newly uploaded images -
we’ll see how it goes.
Filippo will look into whether Swift’s TTL feature is usable for us these days; hopefully
we can use it to reduce storage of unused thumbs / thumb sizes until we move them out of
Swift completely.
Thumbnailing improvements are still in the works on
our end, and the idea of not using swift anymore for those is definitely on the team's
radar (we've started working on more modest thumbnailing improvements at this point),
but if the capacity is there, we might as well improve the average image load time for our
users, even if the ever-increasing swift use for thumbnail is still a problem itself.
Yeah. As you know there’s a fair amount of desire to change the way thumb handling &
storage works, support more variants (sizes, quality), etc by various teams. I’m starting
to think it would be good for us to organise a sprint on this very topic, get the
multimedia team, relevant Ops people and some other interested developers from other teams
and really dive into these problems.
—
Mark Bergsma <mark(a)wikimedia.org>
Lead Operations Architect
Director of Technical Operations
Wikimedia Foundation