Hi everyone,
*tl;dr: The Mobile Apps Team is going to trial seamlessly surfacing full text search results instead of prefixsearch results in cases where prefixsearch does not give satisfactory results.*
The Mobile Apps Team has received a lot of feedback that our search feature isn't the best. Our latest metrics confirm this; around 19% of queries give the user no results. Our working hypothesis is that this is because we use prefixsearch on article titles, which is very insensitive to typos and free-form text.
To help with this, we implemented full-text search. The user has two options for searching, prefixsearch and full text. You can see this example http://i.imgur.com/kxymYF4.png to see what these options look like.
However, the way we present the two options to users is suboptimal. There's no clear mental model for when one should be used compared to the other. The design team recommended that we simply present whichever result set is better for any given query. But how do we decide which result set is better? To validate this course of action, we audited which of the two options, prefixsearch or full text search, was better for a set of queries. The results are here https://docs.google.com/a/wikimedia.org/spreadsheets/d/142IoWVRox83zEPnEIHHhPWS3jSpSATRPdqWxnMU4fKs/edit#gid=0 .
The takehomes of our audit:
- In cases where there are very few prefixsearch results (less than around 5), the full text results are just as good or better than the prefixsearch results. Often, this is because the "did you mean" functionality of the full text API helps the user out. - In cases where there are a good number of prefixsearch results, the prefixsearch results tend to be better than the fulltext results.
Here's what we're going to try:
- Remove the buttons from the UI. - By default, use prefixsearch for searches. - If there are fewer than 5 prefixsearch results, show fulltext search results instead.
Metrics for success:
- Higher search clickthrough (users finding what they need more) - Fewer number of queries give 0 results (users served more results) - Fewer number of queries per search session (users finding what they need faster)
The advantage of this experiment is that it's safe to fail: there is no actual UX change, so if we decide our solution isn't good enough, then we can rollout the fallback of surfacing the buttons without users thinking we're just endlessly tweaking the UI.
Please do get in touch if you have any questions!
Thanks, Dan
I've very excited about this. Where can we see updated search metrics that don't require manual reports? I want to make sure that everyone can see these.
--tomasz
On Fri, Dec 5, 2014 at 1:19 PM, Dan Garry dgarry@wikimedia.org wrote:
Hi everyone,
tl;dr: The Mobile Apps Team is going to trial seamlessly surfacing full text search results instead of prefixsearch results in cases where prefixsearch does not give satisfactory results.
The Mobile Apps Team has received a lot of feedback that our search feature isn't the best. Our latest metrics confirm this; around 19% of queries give the user no results. Our working hypothesis is that this is because we use prefixsearch on article titles, which is very insensitive to typos and free-form text.
To help with this, we implemented full-text search. The user has two options for searching, prefixsearch and full text. You can see this example to see what these options look like.
However, the way we present the two options to users is suboptimal. There's no clear mental model for when one should be used compared to the other. The design team recommended that we simply present whichever result set is better for any given query. But how do we decide which result set is better? To validate this course of action, we audited which of the two options, prefixsearch or full text search, was better for a set of queries. The results are here.
The takehomes of our audit:
In cases where there are very few prefixsearch results (less than around 5), the full text results are just as good or better than the prefixsearch results. Often, this is because the "did you mean" functionality of the full text API helps the user out. In cases where there are a good number of prefixsearch results, the prefixsearch results tend to be better than the fulltext results.
Here's what we're going to try:
Remove the buttons from the UI. By default, use prefixsearch for searches. If there are fewer than 5 prefixsearch results, show fulltext search results instead.
Metrics for success:
Higher search clickthrough (users finding what they need more) Fewer number of queries give 0 results (users served more results) Fewer number of queries per search session (users finding what they need faster)
The advantage of this experiment is that it's safe to fail: there is no actual UX change, so if we decide our solution isn't good enough, then we can rollout the fallback of surfacing the buttons without users thinking we're just endlessly tweaking the UI.
Please do get in touch if you have any questions!
Thanks, Dan
-- Dan Garry Associate Product Manager, Mobile Apps Wikimedia Foundation
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
"If there are fewer than 5 prefixsearch results, show fulltext search results instead."
To be clear, in this case the full text results would append after the 5 prefix results.
On Dec 5, 2014, at 1:19 PM, Dan Garry dgarry@wikimedia.org wrote:
Hi everyone,
tl;dr: The Mobile Apps Team is going to trial seamlessly surfacing full text search results instead of prefixsearch results in cases where prefixsearch does not give satisfactory results.
The Mobile Apps Team has received a lot of feedback that our search feature isn't the best. Our latest metrics confirm this; around 19% of queries give the user no results. Our working hypothesis is that this is because we use prefixsearch on article titles, which is very insensitive to typos and free-form text.
To help with this, we implemented full-text search. The user has two options for searching, prefixsearch and full text. You can see this example to see what these options look like.
However, the way we present the two options to users is suboptimal. There's no clear mental model for when one should be used compared to the other. The design team recommended that we simply present whichever result set is better for any given query. But how do we decide which result set is better? To validate this course of action, we audited which of the two options, prefixsearch or full text search, was better for a set of queries. The results are here.
The takehomes of our audit: In cases where there are very few prefixsearch results (less than around 5), the full text results are just as good or better than the prefixsearch results. Often, this is because the "did you mean" functionality of the full text API helps the user out. In cases where there are a good number of prefixsearch results, the prefixsearch results tend to be better than the fulltext results. Here's what we're going to try: Remove the buttons from the UI. By default, use prefixsearch for searches. If there are fewer than 5 prefixsearch results, show fulltext search results instead. Metrics for success: Higher search clickthrough (users finding what they need more) Fewer number of queries give 0 results (users served more results) Fewer number of queries per search session (users finding what they need faster) The advantage of this experiment is that it's safe to fail: there is no actual UX change, so if we decide our solution isn't good enough, then we can rollout the fallback of surfacing the buttons without users thinking we're just endlessly tweaking the UI.
Please do get in touch if you have any questions!
Thanks, Dan
-- Dan Garry Associate Product Manager, Mobile Apps Wikimedia Foundation _______________________________________________ Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
On 5 December 2014 at 13:36, Monte Hurd mhurd@wikimedia.org wrote:
To be clear, in this case the full text results would append after the 5 prefix results.
Actually, why do we need to do that? I thought the audit showed pretty clearly that this wasn't necessary. :-)
Try a few queries and see; in almost every case, either the prefix results are in included in the full text search results, or they're actually really irrelevant.
Dan
Hello,
that sounds really great! Nice to see big improvements to app search in the last and future time :)
Is there a tracking bug/card for this to follow what’s going on?
Kind regards Florian
Von: mobile-l-bounces@lists.wikimedia.org [mailto:mobile-l-bounces@lists.wikimedia.org] Im Auftrag von Dan Garry Gesendet: Freitag, 5. Dezember 2014 22:20 An: mobile-l Betreff: [WikimediaMobile] [Apps] Simplified search? It's possible!
Hi everyone,
tl;dr: The Mobile Apps Team is going to trial seamlessly surfacing full text search results instead of prefixsearch results in cases where prefixsearch does not give satisfactory results.
The Mobile Apps Team has received a lot of feedback that our search feature isn't the best. Our latest metrics confirm this; around 19% of queries give the user no results. Our working hypothesis is that this is because we use prefixsearch on article titles, which is very insensitive to typos and free-form text.
To help with this, we implemented full-text search. The user has two options for searching, prefixsearch and full text. You can see this example to see what these options look like.
However, the way we present the two options to users is suboptimal. There's no clear mental model for when one should be used compared to the other. The design team recommended that we simply present whichever result set is better for any given query. But how do we decide which result set is better? To validate this course of action, we audited which of the two options, prefixsearch or full text search, was better for a set of queries. The results are here.
The takehomes of our audit: • In cases where there are very few prefixsearch results (less than around 5), the full text results are just as good or better than the prefixsearch results. Often, this is because the "did you mean" functionality of the full text API helps the user out. • In cases where there are a good number of prefixsearch results, the prefixsearch results tend to be better than the fulltext results. Here's what we're going to try: • Remove the buttons from the UI. • By default, use prefixsearch for searches. • If there are fewer than 5 prefixsearch results, show fulltext search results instead. Metrics for success: • Higher search clickthrough (users finding what they need more) • Fewer number of queries give 0 results (users served more results) • Fewer number of queries per search session (users finding what they need faster) The advantage of this experiment is that it's safe to fail: there is no actual UX change, so if we decide our solution isn't good enough, then we can rollout the fallback of surfacing the buttons without users thinking we're just endlessly tweaking the UI.
Please do get in touch if you have any questions!
Thanks, Dan