Fwd: [Wikitech-l] New search backend live on mediawiki.org - Wikidata-tech

28 Aug 2013

Did I say it'll take until next year before we have the new search 
infrastructure? Looks like I was wrong, it's being beta tested on mediawiki.org 
already. Time to check how well it works with wikibase, then!

Let's ask about that in the call tomorrow.

-- daniel

-------- Original-Nachricht --------
Betreff: [Wikitech-l] New search backend live on mediawiki.org
Datum: Wed, 28 Aug 2013 14:20:10 -0400
Von: Nikolas Everett &lt;neverett(a)wikimedia.org&gt;
Antwort an: Wikimedia developers &lt;wikitech-l(a)lists.wikimedia.org&gt;
An: Wikimedia developers &lt;wikitech-l(a)lists.wikimedia.org&gt;

Today we threw the big lever and turned on our new search backend at
mediawiki.org.  It isn't the default yet but it is just about ready for you
to try.  Here is what is we think we've improved:
1.  Templates are now expanded during search so:
1a.  You can search for text included in templates
1b.  You can search for categories included in templates
2.  The search engine is updated very quickly after articles change.
3.  A few funky things around intitle and incategory:
3a.  You can combine them with a regular query (incategory:kings peaceful)
3b.  You can use prefix searches with them (incategory:norma*)
3c.  You can use them everywhere in the query (roger incategory:normans)

What we think we've made worse and we're working on fixing:
1.  Because we're expanding templates some things that probably shouldn't
be searched are being searched.  We've fixed a few of these issues but I
wouldn't be surprised if more come up.  We opened Bug 53426 regarding audio
tags.
2.  The relative weighting of matches is going to be different.  We're
still fine tuning this and we'd appreciate any anecdotes describing search
results that seem out of order.
3.  We don't currently index headings beyond the article title in any
special way.  We'll be fixing that soon. (Bug 53481)
4.  Searching for file names or clusters of punctuation characters doesn't
work as well as it used to.  It still works reasonably well if you surround
your query in quotes but it isn't as good as it was.  (Bugs 53013 and 52948)
5.  "Did you mean" suggestions currently aren't highlighted at all and
sometimes we'll suggest things that aren't actually better. (Bugs 52286 and
52860)
6.  incategory:"category with spaces" isn't working. (Bug 53415)

What we've changed that you probably don't care about:
1.  Updating search in bulk is much more slow then before.  This is the
cost of expanding templates.
2.  Search is now backed by a horizontally scalable search backend that is
being actively developed (Elasticsearch) so we're in a much better place to
expand on the new solution as time goes on.

Neat stuff if you run your own MediaWiki:
CirrusSearch is much easier to install than our current search
infrastructure.

So what will you notice?  Nothing!  That is because while the new search
backend (CirrusSearch) is indexing we've left the current search
infrastructure as the default while we work on our list of bugs.  You can
see the results from CirrusSearch by performing your search as normal and
adding "&srbackend=CirrusSearch" to the url parameters.

If you notice any problems with CirrusSearch please file bugs directly for
it:
https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions…

Nik Everett
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l