We have been looking into possibly setting up a MySQL cluster to avoid some of the complexities of a master-slave environment. Does the mediawiki code support MySQL clustering? Has anyone tried testing a cluster with Wikipedia or a smaller db? Also, how does wikipedia currently handle automatic failover of a master-slave transition and then also later the slave-master transition when the master comes back online? What do your scripts do now to deal with that and what kind of monitoring do you to make sure everything is in good health?
Thanks, Chris
chris muse wrote:
We have been looking into possibly setting up a MySQL cluster to avoid some of the complexities of a master-slave environment. Does the mediawiki code support MySQL clustering?
We don't support MySQL Cluster at this time, as it's currently limited in many ways. (Everything must fit in memory, can't have certain field types, etc.)
Also, how does wikipedia currently handle automatic failover of a master-slave transition and then also later the slave-master transition when the master comes back online?
We currently handle failover manually by switching things around from the primary master to a standby. The previous master then gets cleaned up, put back into service as a slave, and is available as a standby for the next exciting failure.
what kind of monitoring do you to make sure everything is in good health?
Not enough; servmon is currently broken and offline.
-- brion vibber (brion @ pobox.com)
We don't support MySQL Cluster at this time, as it's currently limited in many ways. (Everything must fit in memory, can't have certain field types, etc.)
What mediawiki code would need to be rewritten and has anyone tried to do a cluster with the non-supported stuff as a regular InnoDB? Is it just the blob stuff that's a problem? I was also just kind of curious of what the game plan was on the horizon, if you were holding out until MySQL added support in a later version or if other things were eclipsing database stuff at the moment. What is the general consensus on Oracle?
Thanks, Chris
On 3/15/06, Brion Vibber brion@pobox.com wrote:
chris muse wrote:
We have been looking into possibly setting up a MySQL cluster to avoid some of the complexities of a master-slave environment. Does the mediawiki code support MySQL clustering?
We don't support MySQL Cluster at this time, as it's currently limited in many ways. (Everything must fit in memory, can't have certain field types, etc.)
Also, how does wikipedia currently handle automatic failover of a master-slave transition and then also later the slave-master transition when the master comes back online?
We currently handle failover manually by switching things around from the primary master to a standby. The previous master then gets cleaned up, put back into service as a slave, and is available as a standby for the next exciting failure.
what kind of monitoring do you to make sure everything is in good health?
Not enough; servmon is currently broken and offline.
-- brion vibber (brion @ pobox.com)
Wikitech-l mailing list Wikitech-l@wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l
chris muse wrote:
What mediawiki code would need to be rewritten and has anyone tried to do a cluster with the non-supported stuff as a regular InnoDB? Is it just the blob stuff that's a problem? I was also just kind of curious of what the game plan was on the horizon, if you were holding out until MySQL added support in a later version or if other things were eclipsing database stuff at the moment.
Currently waiting for the upcoming versions which allow disk-backed data, etc. No point in rushing in when our own contributors who are MySQL employees are telling us to wait for it to mature a bit. :)
What is the general consensus on Oracle?
From what I hear on the interweb, they are devil-people who eat babies.
-- brion vibber (brion @ pobox.com)
On Wed, Mar 15, 2006 at 01:00:26PM -0800, Brion Vibber wrote:
What is the general consensus on Oracle?
From what I hear on the interweb, they are devil-people who eat babies.
I believe you mispelt "intarweb".
(And "teh", come to think of it)
How about PostGreSQL?
Cheers, -- jra
On 3/16/06, Jay R. Ashworth jra@baylink.com wrote:
How about PostGreSQL?
Working on porting Wikipedia to PostgreSQL is a good way to get hired working for MySQL AB. Seems so far the only group thats been immune to it are the folks at http://pgfoundry.org/projects/wikipedia
As far as your question goes, PG is still stuck with master/slave replication as well.. True multimaster was accomplished in Postres-R as a proof of concept, but hasn't been synced up and made part of the official code yet. Someday.
For the queryload that Wikipedia creates, MySQL runs fine... although, for adhoc queries PG wipes the floor (no bitmap joins in mysql @#$@#$). Also, if you import Wikipedia into PG it's much smaller than MySQL even with gzip disabled because PG gzips on the back end and doesn't insert airspace in the tables (if you want your tables physically clustered on some field, you have to configure that explicitly, it's nor forced like w/ inno and primary keys). Also, since PG has proper UTF-8 support you don't need to use blobs... thus it's easy to throw in full text indexes and perform (index accelerated) queries on article text.
Related to this I've mostly given up on using toolserver for analysis stuff... between the current unavailability of article text, the unresponsiveness of the folks operating it, and the poor performance of MySQL with complex queries... .. Being forced to choose between working with old data that I can query flexibly and painlessly and working with fresh data that is painful to query.. :(
On Thursday 16 March 2006 11:57, Jay R. Ashworth wrote:
On Wed, Mar 15, 2006 at 01:00:26PM -0800, Brion Vibber wrote:
What is the general consensus on Oracle?
From what I hear on the interweb, they are devil-people who eat babies.
:-)
I believe you mispelt "intarweb".
(And "teh", come to think of it)
How about PostGreSQL?
We've seen the PostgreSQL port tested with Slony replication in a simple master->slave setup and it worked well on the surface (This was with the wikipedia dump data, but I believe without binary data) but I don't want to claim it was exhaustive or anything.
wikitech-l@lists.wikimedia.org