2008/9/7 River Tarnell river@wikimedia.org
i'm not sure what you're asking here; there has never been any redundancy for host network connections, or for toolserver database replicas, so nothing happened to that - it wasn't there in the first place.
if you're suggesting redundancy should be added - yes, that would be nice, except that to duplicate 3 database servers would cost around EUR20,000, and even that wouldn't help if the network problem was elsewhere (for example, if our transit connections had been on the card that failed, everything would have been offline, not just one server).
perhaps we could duplicate the entire toolserver setup at another location. that would cost around EUR30,000 in hardware, as well as monthly rental and transit (for at least 16RU space, which wouldn't be cheap, and a fair amount of bandwidth), and would significantly increase the administration effort needed, when we hardly have enough time to maintain the current set of servers. and what would it save - a few hours downtime in the rather unlikely event of a linecard failure?
I was actually talking about connecting the servers to the net down different routes, ie multiple linecards, so if one went down, the other would be there to take the load. Replicating the entire toolserver would be way too much bother and expense, I agree, but surely another connection can't hurt too much?
(apologies if I've got completely the wrong end of the stick here)
Stwalkerster