This is current load report for -dev:
Current Load CRITICAL 2014-02-09 16:51:43 0d 1h 21m 34s 4/4 CRITICAL - load average: 122.26, 118.04, 106.45
pretty much typical for processes hang waiting for IO (NFS outage). The ssh might work, but because all /home folders are mounted on nfs there is no way to use it unless you have access to account which doesn't have home in /home
On Sun, Feb 9, 2014 at 5:50 PM, Petr Bena benapetr@gmail.com wrote:
Pretty much nothing regular admins can fix, we indeed need someone from WMF with root keys / access to physical servers, because this seems to be issue on NFS again (IMHO) can't confirm that of course...
Anyway, I can't even ssh there, I /could try/ to reboot some of the boxes like tools-dev but I am afraid it's not going to help anything.
On Sun, Feb 9, 2014 at 4:43 PM, John phoenixoverride@gmail.com wrote:
All connections to tools.wmflabs is out, ssh, ftp, http, and https are all non-functional. I am getting zero response on IRC, can someone with Coren's mobile number give him a call?
John
Labs-l mailing list Labs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/labs-l