Hi everyone,

tl;dr: The Mobile Apps Team is going to trial seamlessly surfacing full text search results instead of prefixsearch results in cases where prefixsearch does not give satisfactory results.

The Mobile Apps Team has received a lot of feedback that our search feature isn't the best. Our latest metrics confirm this; around 19% of queries give the user no results. Our working hypothesis is that this is because we use prefixsearch on article titles, which is very insensitive to typos and free-form text.

To help with this, we implemented full-text search. The user has two options for searching, prefixsearch and full text. You can see this example to see what these options look like.

However, the way we present the two options to users is suboptimal. There's no clear mental model for when one should be used compared to the other. The design team recommended that we simply present whichever result set is better for any given query. But how do we decide which result set is better? To validate this course of action, we audited which of the two options, prefixsearch or full text search, was better for a set of queries. The results are here.

The takehomes of our audit:

In cases where there are very few prefixsearch results (less than around 5), the full text results are just as good or better than the prefixsearch results. Often, this is because the "did you mean" functionality of the full text API helps the user out.
In cases where there are a good number of prefixsearch results, the prefixsearch results tend to be better than the fulltext results.

Here's what we're going to try:

Remove the buttons from the UI.
By default, use prefixsearch for searches.
If there are fewer than 5 prefixsearch results, show fulltext search results instead.

Metrics for success:

Higher search clickthrough (users finding what they need more)
Fewer number of queries give 0 results (users served more results)
Fewer number of queries per search session (users finding what they need faster)

The advantage of this experiment is that it's safe to fail: there is no actual UX change, so if we decide our solution isn't good enough, then we can rollout the fallback of surfacing the buttons without users thinking we're just endlessly tweaking the UI.

Please do get in touch if you have any questions!

Thanks,

Dan

Dan Garry

Associate Product Manager, Mobile Apps

Wikimedia Foundation