[Labs-l] Partial outage in progress -- update

[Andrew Bogott]

No data was lost.  I'm currently migrating VMs from virt1005 onto a new, 
more trustworthy server. All affected instances will start back up one 
by one over the next hour or so.

-Andrew


On 2/17/15 9:31 AM, Andrew Bogott wrote:
> One of the labs virtualization hosts, virt1005, is suffering a disk 
> failure.  I'm restarting right now -- that may allow us to gradually 
> recover.  If we're less lucky, then rebuilding from the outage will be 
> a prolonged process.
>
> A list of affected instances can be found here:
>
> https://phabricator.wikimedia.org/P305
>
> Note that this box was hosting the Tools web proxy, so the web 
> interface for most tools is currently down.  That should be easy to 
> rebuild if necessary, and will be a high priority.
>
> Updates as events warrant!
>
> -Andrew