Hi all,
We've indeed had a total site outage for roughly 30 minutes. We're still collecting all data, but we've tracked down the cause to multiple cascading issues including loss of power to a critical SPOF network switch and HHVM MediaWiki application servers getting blocked due to multiple unoptimal timeout settings. We'll post a full incident report soon, and work to correct the underlying issues as soon as possible.
Our apologies,
On Thu, Feb 5, 2015 at 7:03 PM, Guillaume Paumier gpaumier@wikimedia.org wrote:
Hi,
Le jeudi 5 février 2015, 09:58:01 George Herbert a écrit :
I saw a WMF tweet of a site outage (network?) around 9:30am Pacific
time, by
the time I could check now things seem ok on en
Sites are mostly back up but there are still issues with login, so the Ops team hasn't had time to write a postmortem yet.
-- Guillaume Paumier
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l