I've very excited about this. Where can we see updated search metrics
that don't require manual reports? I want to make sure that everyone
can see these.
--tomasz
On Fri, Dec 5, 2014 at 1:19 PM, Dan Garry <dgarry(a)wikimedia.org> wrote:
Hi everyone,
tl;dr: The Mobile Apps Team is going to trial seamlessly surfacing full text
search results instead of prefixsearch results in cases where prefixsearch
does not give satisfactory results.
The Mobile Apps Team has received a lot of feedback that our search feature
isn't the best. Our latest metrics confirm this; around 19% of queries give
the user no results. Our working hypothesis is that this is because we use
prefixsearch on article titles, which is very insensitive to typos and
free-form text.
To help with this, we implemented full-text search. The user has two options
for searching, prefixsearch and full text. You can see this example to see
what these options look like.
However, the way we present the two options to users is suboptimal. There's
no clear mental model for when one should be used compared to the other. The
design team recommended that we simply present whichever result set is
better for any given query. But how do we decide which result set is better?
To validate this course of action, we audited which of the two options,
prefixsearch or full text search, was better for a set of queries. The
results are here.
The takehomes of our audit:
In cases where there are very few prefixsearch results (less than around 5),
the full text results are just as good or better than the prefixsearch
results. Often, this is because the "did you mean" functionality of the full
text API helps the user out.
In cases where there are a good number of prefixsearch results, the
prefixsearch results tend to be better than the fulltext results.
Here's what we're going to try:
Remove the buttons from the UI.
By default, use prefixsearch for searches.
If there are fewer than 5 prefixsearch results, show fulltext search results
instead.
Metrics for success:
Higher search clickthrough (users finding what they need more)
Fewer number of queries give 0 results (users served more results)
Fewer number of queries per search session (users finding what they need
faster)
The advantage of this experiment is that it's safe to fail: there is no
actual UX change, so if we decide our solution isn't good enough, then we
can rollout the fallback of surfacing the buttons without users thinking
we're just endlessly tweaking the UI.
Please do get in touch if you have any questions!
Thanks,
Dan
--
Dan Garry
Associate Product Manager, Mobile Apps
Wikimedia Foundation
_______________________________________________
Mobile-l mailing list
Mobile-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l