I'm getting this same error on multiple Wikimedia projects:
"An error has occurred while searching: Search is currently too busy. Please try again later."
Help?
Pine
#wikimedia-operations, they're already working on it :)
Best, Florian
-----Original-Nachricht----- Betreff: [Wikitech-l] Search errors on multiple Wikimedia projects Datum: Mon, 15 Jun 2015 09:12:58 +0200 Von: Pine W wiki.pine@gmail.com An: "wikitech-l@lists.wikimedia.org" wikitech-l@lists.wikimedia.org
I'm getting this same error on multiple Wikimedia projects:
"An error has occurred while searching: Search is currently too busy. Please try again later."
Help?
Pine _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Mon, 2015-06-15 at 00:11 -0700, Pine W wrote:
I'm getting this same error on multiple Wikimedia projects:
"An error has occurred while searching: Search is currently too busy. Please try again later."
Help?
Likely https://phabricator.wikimedia.org/T102454
andre
The solution was backported to wmf9 and deployed on the cluster: 07:21 logmsgbot: ori Synchronized php-1.26wmf9/extensions/CirrusSearch/includes/Util.php: I504dac0c3: Add missing 'use \Status;' to includes/Util.php (duration: 00m 13s)
The failures now doesn't create fatal errors anymore, but unfortunately the "main" problem isn't solved with that.
Best, Florian
-----Original-Nachricht----- Betreff: Re: [Wikitech-l] Search errors on multiple Wikimedia projects Datum: Mon, 15 Jun 2015 10:15:43 +0200 Von: Andre Klapper aklapper@wikimedia.org An: wikitech-l@lists.wikimedia.org
On Mon, 2015-06-15 at 00:11 -0700, Pine W wrote:
I'm getting this same error on multiple Wikimedia projects:
"An error has occurred while searching: Search is currently too busy. Please try again later."
Help?
Likely https://phabricator.wikimedia.org/T102454
andre
On Mon, Jun 15, 2015 at 12:12 AM Pine W wiki.pine@gmail.com wrote:
I'm getting this same error on multiple Wikimedia projects:
"An error has occurred while searching: Search is currently too busy. Please try again later."
Help?
The elasticsearch cluster exploded completely. It's mostly recovered now (just waiting for full redundancy) and searches should now be back on for users. Full incident report to follow I imagine.
It's actually not the bug with "Status" not being found...that was just bad timing (but is now also fixed).
-Chad
On Mon, Jun 15, 2015 at 3:28 PM, Chad innocentkiller@gmail.com wrote:
On Mon, Jun 15, 2015 at 12:12 AM Pine W wiki.pine@gmail.com wrote:
I'm getting this same error on multiple Wikimedia projects:
"An error has occurred while searching: Search is currently too busy. Please try again later."
Help?
The elasticsearch cluster exploded completely. It's mostly recovered now (just waiting for full redundancy) and searches should now be back on for users. Full incident report to follow I imagine.
You are correct. I'm not making promises on how soon I'll get to write it, though. Also, we'd probably need to do some root cause investigation (I think we do have a candidate, but I'd like for some ElasticSearch expert to take a look more thoroughly).
I will update this thread with a link to the outage report as soon as it's ready.
Cheers
Giuseppe
wikitech-l@lists.wikimedia.org