Carlton B wrote:
This is a technical question... the Wikipedia upgrade
appears to have been
going on for around 9 hours at this point. So that I can factor this into
my wiki maintenance plan, what's taken so long? Was there a database schema
change or other database transformation?
Yes; for a general (but not quite exact anymore, some details changed)
overview of the main schema change see:
http://meta.wikimedia.org/wiki/Proposed_Database_Schema_Changes/October_2004
cur+old -> page+revision+text
links+brokenlinks -> pagelinks
Plus some field changes on several other tables and UTF-8 conversion for
remaining Latin-1 wikis.
Is it the sheer size of the database dumps?
Yes. Very small wikis convert in under a minute; Commons took a couple
hours, mostly in processing of metadata for its 144k+ uploaded files
loaded over NFS.
English Wikipedia has 58 gigabytes of old-revision data which is being
processed to split up into the new, more efficient schema and partially
convert to UTF-8. It just plain takes a while, but we had to eat that
cost at some point or another -- and by splitting off the bulk text,
this will greatly speed up future upgrades.
If your wiki has a million or so pages and 15 million old revisions, it
may take hours and hours for you to upgrade, too. :)
Were there unforeseen problems?
Always some small glitches here and there, but that's not unforeseen. :)
Is it the distributed configuration of Wikipedia?
See
http://mail.wikipedia.org/pipermail/wikitech-l/2005-June/030415.html
for some notes.
-- brion vibber (brion @
pobox.com)