On Thu, Jan 15, 2015 at 2:30 PM, Gilles Dubuc <gilles(a)wikimedia.org> wrote:
Erik came up with the following suggestion on the
whole
thumbnails-stored-in-swift issue: would it be possible to store thumbnails
in a separate Swift cluster than originals, and have the replica count
setting for the thumbnail swift cluster set to 1?
The rationale being that this would solve the main issue of storage space
waste without having to learn/test/deploy a new tool.
it is possible to have a separate swift cluster but I'm not sure it'd yield
many benefits because e.g. each swift machine would become a "SPOF" in the
sense if it is down imagescalers would have to take all hits for all thumbs
stored on that machine.
To give some numbers, originals take up ~58T and thumbs ~32T while there
are ~27M originals vs ~428M thumbs and from swift thumbs are requested
~6.5x more than originals.
There's more proposals here too:
https://www.mediawiki.org/wiki/Talk:Requests_for_comment/Simplify_thumbnail…
HTH,
filippo