Just a heads-up for those not following CVS; I'm fixing up an alternate schema upgrader which will do the major schema updates and UTF-8 conversion for the 1.5 upgrade, and should be relatively friendly to replication. One of the many tricky things about MySQL server replication is that the update log is serialized. When you're working on one server, you can be making updates to many databases and tables simultaneously (so long as they don't interfere with each other); the update log used for replication reorders the queries into a logically equivalent stream, one after the other. That makes your life much simpler if your job is writing replication support for MySQL. ;) However it makes your life more complicated if you're trying to use replicated servers for load balancing of read-only queries with time-sensitive data (that's us!). The worst case is when something like a major database schema change happens -- you're interleaving small, fast changes (edits, saved preferences, new accounts...) with large, slow changes (change the layout of a million old-page records). On the single server things may mostly work, but when the replication gets to that million-record honker, nothing else gets applied on the slave server until it's done with that one. As a result the view of the database in the slave servers get more and more behind the current state of the master. Weird things happen on the site, with out-of-date or missing pages. We get that to a degree when certain mass nasties happen (like page-move attacks on massively-linked pages), but doing database maintainence while keeping wikis online would be particularly ugly. I've been putting together an alternate 1.5 schema upgrader which also includes the UTF-8 conversion (which we need anyway), which applies the updates in smaller chunks. That is, instead of "copy these fields from a million 'old' records into the 'revision' table", it chunks in say 100 records at a time. This is probably slower 'in total' (though since we have to do UTF-8 conversion on several large wikis anyway, it's necessary to a degree), but the main benefit is that it should allow the replication stream to be interleaved -- a chunk of upgrade, a few edits, a chunk of upgrade, a few edits. The slaves should stay up to date during the process, always applying small, recent updates. So instead of turning everything off indefinitely while tables are churned around, we should be able to upgrade a wiki at a time, while keeping the rest of the wikis online and editable as they wait their turn. For instance while en.wikipedia.org is in read-only for conversion, people should still be able to edit Wikinews or Wiktionary, or another language Wikipedia. In theory. :) We'll see how it goes this weekend... -- brion vibber (brion @ pobox.com)