It was great to meet you at IA yesterday, thanks for following up with this link to your work. Very interesting and coincides with our own work on using the completion suggester to replace the current prefix search used on-wiki.
Have you put any thought into normalizing page view data? One thing we have been trying to figure out (but on the back-burner as we focus on currently quarterly goals) is how best to integrate page views (
https://phabricator.wikimedia.org/T112681). Because we have to do this across many wiki's with a wide varience in page views, and we want to use the data not only for the completion suggester but also within our full text search results, we are thinking about normalizing the data down to a % of page views for that wiki over a time period. Possiblying taking in a larger time period of page views and weighting newer page views as more important than older page views. Additionally we are looking into if we should be batch loading page view information on a weekly basis, or if we can load page view data only when pages are edited (or some combination of the two). I've pinged david and trey with this and they might have some questions for you :)
We havn't yet put this into production because we wanted to integrate page view data into the scoring before running more tests. It looks quite promising based on your initial