I have just posted a new draft RFC on changing the thumbnail storage and caching pipeline for Wikimedia projects [0].
This RFC was started as part of the Multimedia tech debt project. Aaron, Faidon and I would like to move discussion forward on the issue of the thumbnail caching layer in general and specifically would like to find consensus on a reasonable change to address some of the shortfalls of the current system in place for upload.wikimedia.org and related wikis.
The core problem has been discussed on list previously [1], namely that scaled media files (aka thumbnails) take up a lot of space in the storage servers. This RFC proposes following a course of action proposed in that prior discussion [2] to customize the Varnish hashing configuration for thumbs such that a single HTCP purge would invalidate all size variations for a given file. It attempts to document some of the benefits and drawbacks of that proposal as well as present a few related variations.
Any and all feedback from this group would be greatly appreciated in vetting the core idea and determining reasonable next steps towards a viable long term solution.
[0]: https://www.mediawiki.org/wiki/Requests_for_comment/Simplify_Thumbnail_Cache [1]: http://wikimedia.7.x6.nabble.com/scaled-media-thumbs-as-temporary-files-not-... [2]: http://wikimedia.7.x6.nabble.com/scaled-media-thumbs-as-temporary-files-not-...
Bryan