We had a serious MySQL crash on suda with associated data corruption today (6/6). There's a summary of the events leading up to the crash at http://openfacts.berlios.de/index-en.phtml?title=Wikipedia_plans Whether it was the kill -9 that led to the corruption or whether the database was already corrupted and did therefore not respond we do not know; in any case, there seem to have been no alternatives to killing it (people on #mysql knew nothing either). Shaihulud made a copy of the CUR tables from all wikis earlier today and imported it on Ariel. We've switched the live wikis to readonly from Ariel; readonly because Ariel doesn't have OLD and lots of other stuff, because it's not sufficiently tested, and because we'd like to prevent any data loss if possible. Tim created a special "readonly" user on ariel for this purpose. The following have been changed as long as we are in readonly mode: 1) Counters disabled on all wikis 2) Linkscc disabled 3) readonly file set to /home/wikipedia/common/readonly 4) user_newtalk disabled 5) $wgDatabaseServer and $wgDBuser changed to ariel There will still be lots of error messages and because the OLD tables are not on Ariel revision histories are missing etc. This is *only* to make sure that people can read our articles. The next step is to fix the data corruption on suda. Regards, Erik _______________________________________________ Wikitech-l mailing list Wikitech-l(a)Wikipedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l

Whether it was the kill -9 that led to the corruption or whether the database was already corrupted and did therefore not respond we do not know; in any case, there seem to have been no alternatives to killing it (people on #mysql knew nothing either).