Recently somebody dumped a huge amount of text into Template:POTD on enwiki, and people were unable to revert it as the edits failed (timing out and showing squid error pages).
As a temporary measure I cleared its outgoing pagelinks and reverted the page_latest manually to an earlier revision. However it remained uneditable.
I found that the edit updates were hanging while trying to touch the page_touched timestamp on the pages linking to the template. By changing it from a single multiple-target UPDATE statement to separate per-record UPDATEs I found that it was hanging on a particular record:
mysql> UPDATE `page` SET page_touched='20050711222253' WHERE page_id=2152287 limit 1;
Updating this record, referenced either by title or by id, just hangs for a couple minutes until it times out trying to get a lock. But there's no competing query that I can see, and no reason for an open lock.
I've got James poking at it, hopefully we can figure it out...
-- brion vibber (brion @ pobox.com)
Brion Vibber wrote:
mysql> UPDATE `page` SET page_touched='20050711222253' WHERE page_id=2152287 limit 1;
Updating this record, referenced either by title or by id, just hangs for a couple minutes until it times out trying to get a lock. But there's no competing query that I can see, and no reason for an open lock.
I've got James poking at it, hopefully we can figure it out...
On James's advice I've restarted the master server with the innodb_status_file=1 option. Restarting had the happy side effect of removing whatever locks were open. ;)
But if it happens again, hopefully we'll have a better idea what's going on with some more debugging...
-- brion vibber (brion @ pobox.com)
wikitech-l@lists.wikimedia.org