Because bad things come in threes (I'm hoping it's threes and not
sevens) the server that hosts toolsdb is now also misbehaving. Brooke
just now disabled a troubled drive which may have resolved things, but
if the last few hours are any indication then the vast majority of
connection or query attempts are likely to fail until we have a better
solution in place.

We're working on multiple fronts, trying to diagnose and fix the primary
issue while also working to get new hardware online as a possible
replacement server.  Neither of those things are likely to get done
until tomorrow, though, so toolforge will be in pretty bad shape in the
meantime.

It has been a rough couple of days, but rest assured we're taking notes
about how to prevent outages like these in the future. Thank you for
your patience in the meantime!

-Andrew + the cloud team



_______________________________________________
Wikimedia Cloud Services announce mailing list
Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce
_______________________________________________
Wikimedia Cloud Services mailing list
Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud