[Labs-l] Outage of labs in progress

Andrew Bogott abogott at wikimedia.org
Thu Nov 6 18:55:46 UTC 2014


On 11/6/14 11:45 AM, Marc A. Pelletier wrote:
> Hello all,
>
> There is currently a hardware issue that took down labs.  We're hard at
> work on it and will give news the moment we have them.
>
> -- Marc
>
>
Since Marc is busy troubleshooting, here are some details:

Labs and Toollabs are currently offline.  The file server that supports 
home directories, shared storage, and login keys is suffering a hardware 
failure.

Depending on their use of the filesystem, some tools and other processes 
may still be working during this outage.  I expect most things will have 
died, though, and user logins will definitely not work.

Coren (aka Marc) and Chris Johnson (who is on site at the datacenter) 
are currently working frantically to solve this problem.  Once the issue 
is resolved, Coren or I will send a followup email which includes 
instructions about restarting services if necessary.

Sorry for the interruption, hope to be back running soon!

-Andrew




More information about the Labs-l mailing list