Just a heads-up for those not following CVS; I'm fixing up an alternate
schema upgrader which will do the major schema updates and UTF-8
conversion for the 1.5 upgrade, and should be relatively friendly to
replication.
One of the many tricky things about MySQL server replication is that the
update log is serialized. When you're working on one server, you can be
making updates to many databases and tables simultaneously (so long as
they don't interfere with each other); the update log used for
replication reorders the queries into a logically equivalent stream, one
after the other.
That makes your life much simpler if your job is writing replication
support for MySQL. ;) However it makes your life more complicated if
you're trying to use replicated servers for load balancing of read-only
queries with time-sensitive data (that's us!).
The worst case is when something like a major database schema change
happens -- you're interleaving small, fast changes (edits, saved
preferences, new accounts...) with large, slow changes (change the
layout of a million old-page records). On the single server things may
mostly work, but when the replication gets to that million-record
honker, nothing else gets applied on the slave server until it's done
with that one. As a result the view of the database in the slave servers
get more and more behind the current state of the master. Weird things
happen on the site, with out-of-date or missing pages.
We get that to a degree when certain mass nasties happen (like page-move
attacks on massively-linked pages), but doing database maintainence
while keeping wikis online would be particularly ugly.
I've been putting together an alternate 1.5 schema upgrader which also
includes the UTF-8 conversion (which we need anyway), which applies the
updates in smaller chunks. That is, instead of "copy these fields from a
million 'old' records into the 'revision' table", it chunks in say
100
records at a time.
This is probably slower 'in total' (though since we have to do UTF-8
conversion on several large wikis anyway, it's necessary to a degree),
but the main benefit is that it should allow the replication stream to
be interleaved -- a chunk of upgrade, a few edits, a chunk of upgrade, a
few edits. The slaves should stay up to date during the process, always
applying small, recent updates.
So instead of turning everything off indefinitely while tables are
churned around, we should be able to upgrade a wiki at a time, while
keeping the rest of the wikis online and editable as they wait their turn.
For instance while
en.wikipedia.org is in read-only for conversion,
people should still be able to edit Wikinews or Wiktionary, or another
language Wikipedia.
In theory. :)
We'll see how it goes this weekend...
-- brion vibber (brion @
pobox.com)