This is a triple-crosspost. I suggest you reply to wikitech-l only.
A mistake I made caused the loss of 496 full-resolution images from Wikimedia servers.
I have recovered as many images as I can, drawing on the following sources:
* Squid cache (pmtpa, knams and yaseo) * May 8 backup of some wikis on storage1 * Duplicates with the same signature, found on the same or other wikis
That brought the number lost down from about 3000 to the current 496. For the remaining files, I made a copy of their thumbnail directories:
http://upload.wikimedia.org/lost-image-thumb-backup/
A list of missing images can be found here:
http://noc.wikimedia.org/~tstarling/missing-images-2008-09
If anyone has any ideas about where to find more backup files, I'd be willing to hear them. Otherwise, the community will just have to reupload as many as possible.
The technical details were as follows: I fixed a bug in File.php, and without checking what other changes were made to it, deployed the most recent version of the file on the Wikimedia servers, without also updating the rest of MediaWiki. Because FileRepo::$thumbDir was unset, LocalFile::migrateThumbFile() had the effect of deleting the source image for any thumbnail request which reached the backend. I reverted the change after about 20 minutes, following a report on IRC.
My sincere apologies.
-- Tim Starling
On Fri, Sep 5, 2008 at 6:11 AM, Tim Starling tstarling@wikimedia.org wrote:
This is a triple-crosspost. I suggest you reply to wikitech-l only.
A mistake I made caused the loss of 496 full-resolution images from Wikimedia servers.
[snip]
http://noc.wikimedia.org/~tstarling/missing-images-2008-09
If anyone has any ideas about where to find more backup files, I'd be willing to hear them. Otherwise, the community will just have to reupload as many as possible.
[snip]
I have 30 of the 496 images in that list based on an exact path match. It's possible that I have more based on hash matches for image which were moved between sites or 'renamed' after my last sync.
I have some chores to run, but I will later pull the hashes from the database and check for hash matches.
I would likely have had nearly all of them if the rsync push to me had not been down most of the year.
:(
There's a list at http://commons.wikimedia.org/wiki/User:Patr%C3%ADciaR/missing_images to keep up with the ones that are recovered (Commons only). Multichill suggested warning all uploaders to get the "own work" ones, at least. Good news: there seems to be some images that were *not* lost after all; please see http://commons.wikimedia.org/wiki/Commons:Village_pump#Massive_image_loss too.
Tim, s*it happens :). Patrícia
--- On Fri, 5/9/08, Gregory Maxwell gmaxwell@gmail.com wrote: From: Gregory Maxwell gmaxwell@gmail.com Subject: Re: [Commons-l] Massive image loss To: wikitech-l@lists.wikimedia.org, "Wikimedia Commons Discussion List" commons-l@lists.wikimedia.org Cc: "Wikimedia Foundation Mailing List" foundation-l@lists.wikimedia.org Date: Friday, 5 September, 2008, 12:59 PM
On Fri, Sep 5, 2008 at 6:11 AM, Tim Starling tstarling@wikimedia.org wrote:
This is a triple-crosspost. I suggest you reply to wikitech-l only.
A mistake I made caused the loss of 496 full-resolution images from Wikimedia servers.
[snip]
http://noc.wikimedia.org/~tstarling/missing-images-2008-09
If anyone has any ideas about where to find more backup files, I'd be willing to hear them. Otherwise, the community will just have to reupload as many as possible.
[snip]
I have 30 of the 496 images in that list based on an exact path match. It's possible that I have more based on hash matches for image which were moved between sites or 'renamed' after my last sync.
I have some chores to run, but I will later pull the hashes from the database and check for hash matches.
I would likely have had nearly all of them if the rsync push to me had not been down most of the year.
:(
_______________________________________________ Commons-l mailing list Commons-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/commons-l
On Friday 05 September 2008 14:26:54 Patricia Rodrigues wrote:
There's a list at http://commons.wikimedia.org/wiki/User:Patr%C3%ADciaR/missing_images to keep up with the ones that are recovered (Commons only). Multichill suggested warning all uploaders to get the "own work" ones, at least. Good news: there seems to be some images that were *not* lost after all; please see http://commons.wikimedia.org/wiki/Commons:Village_pump#Massive_image_loss too.
It seems that it only affected latest revision of each image. In some cases, older versions could be used as a last resource.
Thanks for full-disclosure.