On Thu, Oct 30, 2014 at 10:18 PM, MZMcBride <z(a)mzmcbride.com> wrote: I have OK news and bad news on that front unfortunately. We used to have brute force regex searches and they were pretty much garbage. It was just too easy to write a search that would take minutes to complete. That would cause the queue of other regex searches to get backed up. And it'd timeout on the varnish side so the user, even if they had the patience to wait for 5 minutes to get a result, wouldn't get one. The OK news is that we cut over to using trigram accelerated regex searches about a week ago. Its better. Almost usable but not quite. The worst case run time is now 30 seconds. If you search for something that is rare it'll probably come back in a few seconds. There are issues with consistency when your request runs really long as well ( https://bugzilla.wikimedia.org/show_bug.cgi?id=72128). Far from perfect but serviceable. The bad news. We've had two Cirrus outages in the past week that I believe are caused by the accelerator so its disabled and we're back to brute force for now. The first outage was on Monday and we didn't have a clue what caused it. We added logging to learn for the next time and it didn't happen again until this morning. The extra logging failed (:shakes fist:) but we were able to implicate this code in the process. The silver lining is that I'll be working on it again and might be able to get some speed improvements while were there. I'm not sure what the outage says about the schedule. I'll have to do some thinking about that. In the mean time it does say that we should keep the old search there as a backup. We've been able to fall back to it during the outages to minimize the suffering. Nik