-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
hi,
we lost power for unknown reasons last night, causing an outage of the Toolserver. most of the platform is back up, but s3 (yarrow) remains down.
as we had downtime anyway, i took the opportunity to move stable to the new server. the old stable server (willow) is now offline.
- river.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
River Tarnell:
most of the platform is back up, but s3 (yarrow) remains down.
more information: it seems yarrow's external disk array did not come back up after the downtime. i am waiting for someone to visit the colo to investigate further. s3 will be offline until then.
additionally, commons replication on s1 (rosemary) is offline for the moment due to some kind of data corruption issue; i will be discussing this with Sun, but we may need to reimport commons on that server, which i will do next week if necessary.
- river.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
yarrow's array has been repaired, and s3 is now online. the problem was the same one we saw with this array last time, which i'm now discussing with the vendor.
- river.
toolserver-announce@lists.wikimedia.org