Mashiah Davidson schrieb: It will be about a month until the new boxes are online. -- daniel

Pietrodn schrieb: It's not a matter ofrestarting. The process is: take a slave server out of rotation and make a dump (I don't have access on the main cluster, so I can't do that), then copy the dump over, import it, and *then* start replication. This takes quite some time (a week, i'd guess), and is a lot of hassle. Also, during import, s3 is going to be completly unavailable for several days, instead of just having old data. Doing this twice is no fun. So it's not an easy choice. I leave it to river to make it :) -- daniel

How long are the binlogs kept for on Wikimedia servers? Surely it would be possible to take a dump now, import it to s3, start replication, then import the same dump onto the new server, and let it catch up from a month of replag? Of course, this wouldn't be possible if the binlogs are not kept for that long. 2009/1/5 Pietrodn <powerpdn(a)gmail.com>om>: -- Regards, Simon Walker User:Stwalkerster on all public Wikimedia Foundation wikis Administrator on the English Wikipedia Developer of Helpmebot and the ACC tool Your donations keep Wikipedia running! Support the Wikimedia Foundation today: http://www.wikimediafoundation.org/wiki/Donate

2009/1/5 River Tarnell <river(a)loreley.flyingparchment.org.uk>uk>: And if WM-US knows, that WM-EU is expecting problems they can't just change setting to leave data you need? AJF/WarX

Simon Walker wrote: No need to reimport the same dump. Dump Import Start replication Normal work with the toolserver *New boxes arrive here* Configure new servers Stop s3 Move mysql data files to the new box Restart s3 Which would also be faster. Importing a dump is slow with this amount of data. The files are bigger, but you avoid the index rebuild. And the servers being on the same location, transfer speed would be really high. River Tarnell wrote: And on other servers? Or even copying the binlogs to knams/toolserver. After all, binlog deletion is a manual process. The same problem arises every few months. The procedures aren't good enough yet. :-(

River, Good news, does it mean we might have s1 and s3 separated within around a month and it is a good time to adopt scripts and tools for this change? Mashiah 2009/1/5 River Tarnell <river(a)loreley.flyingparchment.org.uk>

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mashiah Davidson: yes. i cannot think of anything you would need to change. - river.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Kalan: yes. MySQL federation is not useful for what people want to use it for (joins). for anything beyond simple single-row queries (like the toolserver database), it's far too slow. a better alternative would be MySQL Cluster. i've looked at this in the past, but it had some restrictions that made it unsuitable for us. once these are resolved, it would be easier for both users (as all databases would appear to be on the same server) and for us (as it makes it much easier to add additional capacity to the cluster). - river.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Kalan: binlog replication is not supported. rows can't be larger than 8KB. all indices must be stored in memory at all times. only 20,000 objects (databases, tables, indices) are supported. there's probably more, but those are a few i see from looking at the cluster manual. - river.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Daniel Kinzler: yes. that shouldn't be hard, since we already do it for yarrow. i don't know what that is, but if it's just another database, it would be trivial to add to trainwreck's regex. it's possible, but it increases the size of the backups and the time it takes to make them. probably possible to do it in parallel rather than one at a time... - river.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Platonides: yes, i know. but that isn't the only reason i listed for doing a separate import. - river.