On 03/09/12 02:59, Tim Starling wrote:
I'll go for option 4. You can't delete the images from the backend while they are still in Squid, because then they would not be purged when the image is updated or action=purge is requested. In fact, that is one of only two reasons for the existence of the backend thumbnail store on Wikimedia. The thumbnail backend could be replaced by a text file that stores a list of thumbnail filenames which were sent to Squid within a window equivalent to the expiry time sent in the Cache-Control header.
The other reason for the existence of the backend thumbnail store is to transport images from the thumbnail scalers to the 404 handler. For that purpose, the image only needs to exist in the backend for a few seconds. It could be replaced by a better 404 handler, that sends thumbnails directly by HTTP. Maybe the Swift one does that already.
-- Tim Starling
The second one seems easy to fix. The first one should IMHO be fixed in squid/varnish by allowing wildcard purges (ie. PURGE /wikipedia/commons/thumb/5/5c/Tim_starling.jpg/* HTTP/1.0)
A wiki with such setup could then disable the on-disk storage.