Brion Vibber wrote:
Further, edits went back on ariel for several minutes while figuring out what had happened.
en.wikipedia is currently locked while we examine the databases to see whether recovery of the last 40 minutes' work is feasible.
Tim tried running the slave de-sync script, but it was slow enough it could take two or three hours to complete.
To get things back on their feet more quickly, we've bumped the revision ID position on the master so new edits won't interfere, and the recovery can continue in the background.
There were about 3500 affected edits; in theory most should be recoverable without significant conflicts. A few might possibly have conflicting id numbers and get scrapped.
en.wikipedia is back read-only. There may be some broken diffs and bad cached pages.
-- brion vibber (brion @ pobox.com)