We've now got some preliminary data in about how effective our search is in
mobile apps. Note that this is only half a day's data, and it's sampled at
1%.
High-level metrics:
- *Number of searching sessions which end by tapping on a result:* 91%
- *Percentage of queries that give 0 results: *19%
- *Mean queries per searching session:* 3.8
- *Median user-perceived time taken to retrieve search results: *486ms
- *90th percentile of user-perceived time taken:* 898ms
My quick take-homes from this:
- 91% clickthrough seems pretty good.
- 19% of queries giving "no results" seems really bad.
- The combination of the above two means users are generally finding
what they need, but that it's a struggle to do it.
- User-perceived performance seems pretty good; 90% of our users get
search results within a second!
What I think our next steps are:
- Push out our new search improvements and see how these baselines
change.
- Try to tackle the "0 results" problem, which seems to be more of a
problem than clickthrough.
Spreadsheet containing my queries and processed data:
https://docs.google.com/a/wikimedia.org/spreadsheets/d/1syLgNygAS7Prxxg7RTI…
Dan
--
Dan Garry
Associate Product Manager, Mobile Apps
Wikimedia Foundation