We've now got some preliminary data in about how effective our search is in mobile apps. Note that this is only half a day's data, and it's sampled at 1%.
High-level metrics:
- *Number of searching sessions which end by tapping on a result:* 91% - *Percentage of queries that give 0 results: *19% - *Mean queries per searching session:* 3.8 - *Median user-perceived time taken to retrieve search results: *486ms - *90th percentile of user-perceived time taken:* 898ms
My quick take-homes from this:
- 91% clickthrough seems pretty good. - 19% of queries giving "no results" seems really bad. - The combination of the above two means users are generally finding what they need, but that it's a struggle to do it. - User-perceived performance seems pretty good; 90% of our users get search results within a second!
What I think our next steps are:
- Push out our new search improvements and see how these baselines change. - Try to tackle the "0 results" problem, which seems to be more of a problem than clickthrough.
Spreadsheet containing my queries and processed data: https://docs.google.com/a/wikimedia.org/spreadsheets/d/1syLgNygAS7Prxxg7RTIv...
Dan