This is great. Do you have any categories tracked that could be interesting
to break the position click-rates down by? eg: navigational vs. explorative
queries, SAT clicks (satisfied user's query intent) vs. DSAT clicks (not
satisfied), requery rate (how many times a user reformulates a new query in
a session), time-to-first-click, search session duration, user's
country/default language/# edits, length of query (# of query tokens), # of
query results, popular vs. uncommon queries, high scoring SERP vs. low
scoring SERP (or a proxy like Max BM25F of the top result), speller was
click vs. not clicked, category of page clicked on, popular pages vs.
rarely visited pages, etc.
This experiment running on Special:Search is nice as that page doesn't
automatically redirect when the query exactly matches a page.
You can measure the positional importance by setting up an A/B test where
you flip position 2 & 3. Also, a slowdown experiment would tell you the
impact of latency, and help focus engineering efforts towards precision, or
latency improvements.
--justin
On Tue, Feb 9, 2016 at 9:53 AM, Erik Bernhardson <ebernhardson(a)wikimedia.org
wrote:
2 | 34214 | 14.26%