Daniel Mayer wrote:
I thought each db server was on a UPS? If not, then why not?
As I understand it, our whole rack has two UPSs. The breaker that went is the one that feeds power *from* those UPSs *to* the racks.
It's specced way over what should be needed there; according to bw at the colo they've got the company running diagnostics on it as it shouldn't have been able to blow.
I also thought we had the capability to go read only from the other server farms when the Tampa farm went down. If not, then why not?
No, we've not had this capability so far.
We are a top 20 website that makes more than enough money to hire enough people to make sure services decay gracefully and to make sure we have adequate redundancy. I'm working on the current year budget. Please start thinking about your staffing and equipment needs for the rest of the year.
If you've got the budget now we'll be happy to spend it. But for a "top 20 website" we've still got a piddly little budget and a single employee. Sites with comparable traffic sell for many many millions...
-- brion vibber (brion @ pobox.com)