We've now got some preliminary data in about how effective our search is in mobile apps. Note that this is only half a day's data, and it's sampled at 1%.

High-level metrics:

• Number of searching sessions which end by tapping on a result: 91%
  
• Percentage of queries that give 0 results: 19%
  
• Mean queries per searching session: 3.8
  
• Median user-perceived time taken to retrieve search results: 486ms
  
• 90th percentile of user-perceived time taken: 898ms

My quick take-homes from this:

• 91% clickthrough seems pretty good.
• 19% of queries giving "no results" seems really bad.
• The combination of the above two means users are generally finding what they need, but that it's a struggle to do it.
• User-perceived performance seems pretty good; 90% of our users get search results within a second!

What I think our next steps are:

• Push out our new search improvements and see how these baselines change.
• Try to tackle the "0 results" problem, which seems to be more of a problem than clickthrough.

Spreadsheet containing my queries and processed data: https://docs.google.com/a/wikimedia.org/spreadsheets/d/1syLgNygAS7Prxxg7RTIvK3vxwyVG-ZkjxtK0LSbU76w/edit

Dan

--

Dan Garry

Associate Product Manager, Mobile Apps

Wikimedia Foundation

_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l