[Labs-l] Outage report for last night's labs weirdness (22:30-23:30 UTC)

Andrew Bogott abogott at wikimedia.org
Thu Aug 13 16:29:01 UTC 2015


     Labs services freaked out a bit last night, and caused some goofy 
ssh behavior.  One of the symptoms was a mis-reporting of hostkeys for 
the labs bastions... if you accepted the new host keys you will need to 
accept the bastion host keys /again/ because they are now reported 
correctly.  The actual, official host keys (same as they ever were) can 
be found here:

https://wikitech.wikimedia.org/wiki/Help:SSH_Fingerprints

     If you didn't try to ssh during the incident window, everything 
should be fine.  You may see hiccups in your service logs but no action 
should be required on your part.

     The full incident report is here: 
https://wikitech.wikimedia.org/wiki/Incident_documentation/20150812-LabsOutage

-Andrew



More information about the Labs-l mailing list