[Foundation-l] Massive image loss

[Tim Starling]

This is a triple-crosspost. I suggest you reply to wikitech-l only.

A mistake I made caused the loss of 496 full-resolution images from
Wikimedia servers.

I have recovered as many images as I can, drawing on the following sources:

* Squid cache (pmtpa, knams and yaseo)
* May 8 backup of some wikis on storage1
* Duplicates with the same signature, found on the same or other wikis

That brought the number lost down from about 3000 to the current 496. For
the remaining files, I made a copy of their thumbnail directories:

http://upload.wikimedia.org/lost-image-thumb-backup/

A list of missing images can be found here:

http://noc.wikimedia.org/~tstarling/missing-images-2008-09

If anyone has any ideas about where to find more backup files, I'd be
willing to hear them. Otherwise, the community will just have to reupload
as many as possible.

The technical details were as follows: I fixed a bug in File.php, and
without checking what other changes were made to it, deployed the most
recent version of the file on the Wikimedia servers, without also updating
the rest of MediaWiki. Because FileRepo::$thumbDir was unset,
LocalFile::migrateThumbFile() had the effect of deleting the source image
for any thumbnail request which reached the backend. I reverted the change
after about 20 minutes, following a report on IRC.

My sincere apologies.

-- Tim Starling