On Thu, Oct 30, 2014 at 10:18 PM, MZMcBride <z(a)mzmcbride.com> wrote:
James Forrester wrote:
On 30 October 2014 09:53, Chad
<innocentkiller(a)gmail.com> wrote:
New hardware is in place and we've got plenty
of breathing room to wrap
up the migration to the new search engine.
Excellent news!
Indeed! Thanks to all who made this possible. An independent search engine
is an incredibly important piece of infrastructure that's now getting a
more appropriate level of attention and love. This is great and I'm
excited to see what we'll be able to (continue to) build on top of it.
I lost track of the discussion about the ability to run regular
expressions across wikitext. I see a large amount of opportunity in being
able to search through wikitext in real-time. There are plenty of findable
issues and errors in our articles and search is a key component in
improving the situation.
I have OK news and bad news on that front unfortunately.
We used to have brute force regex searches and they were pretty much
garbage. It was just too easy to write a search that would take minutes to
complete. That would cause the queue of other regex searches to get backed
up. And it'd timeout on the varnish side so the user, even if they had the
patience to wait for 5 minutes to get a result, wouldn't get one.
The OK news is that we cut over to using trigram accelerated regex searches
about a week ago. Its better. Almost usable but not quite. The worst
case run time is now 30 seconds. If you search for something that is rare
it'll probably come back in a few seconds. There are issues with
consistency when your request runs really long as well (
https://bugzilla.wikimedia.org/show_bug.cgi?id=72128). Far from perfect
but serviceable.
The bad news. We've had two Cirrus outages in the past week that I believe
are caused by the accelerator so its disabled and we're back to brute force
for now. The first outage was on Monday and we didn't have a clue what
caused it. We added logging to learn for the next time and it didn't
happen again until this morning. The extra logging failed (:shakes fist:)
but we were able to implicate this code in the process.
The silver lining is that I'll be working on it again and might be able to
get some speed improvements while were there.
I'm not sure what the outage says about the schedule. I'll have to do some
thinking about that. In the mean time it does say that we should keep the
old search there as a backup. We've been able to fall back to it during
the outages to minimize the suffering.
Nik