Hi Gilles,
On 04 Jul 2014, at 10:03, Gilles Dubuc gilles@wikimedia.org wrote:
Hi ops,
Has the swift capacity been increased yet thanks to the new hardware? If so, could we resume the discussion of "pre"generating specific thumbnail sizes at upload time?
Media Viewer could benefit greatly from this performance-wise. As seen on this graph, the launch to all wikis affected the average considerably, since users started hitting a lot of images that didn't have Media Viewer-sized thumbnails yet: http://multimedia-metrics.wmflabs.org/dashboards/mmv#overall_network_perform...
Yes, our Swift capacity expansion has completed. Although the capacity expansion had been planned with reduced storage for thumbs in mind for the future, work that hasn’t started yet, we should have space for this available right now and it shouldn’t be problematic provided that we monitor it well. So please go ahead with this for newly uploaded images - we’ll see how it goes.
Filippo will look into whether Swift’s TTL feature is usable for us these days; hopefully we can use it to reduce storage of unused thumbs / thumb sizes until we move them out of Swift completely.
Thumbnailing improvements are still in the works on our end, and the idea of not using swift anymore for those is definitely on the team's radar (we've started working on more modest thumbnailing improvements at this point), but if the capacity is there, we might as well improve the average image load time for our users, even if the ever-increasing swift use for thumbnail is still a problem itself.
Yeah. As you know there’s a fair amount of desire to change the way thumb handling & storage works, support more variants (sizes, quality), etc by various teams. I’m starting to think it would be good for us to organise a sprint on this very topic, get the multimedia team, relevant Ops people and some other interested developers from other teams and really dive into these problems.
— Mark Bergsma mark@wikimedia.org Lead Operations Architect Director of Technical Operations Wikimedia Foundation