The results of the A/B test have now been published:
https://meta.wikimedia.org/wiki/Research:Reducing_Zero_Result_Searches_Thro…
In summary, there was a statistically significant difference, with the
original configuration being better than the new configuration we tried.
However, practically speaking, that change was so small as to be basically
irrelevant, so we intend to stick with the original configuration.
Thanks,
Dan
On 7 August 2015 at 13:32, Pine W <wiki.pine(a)gmail.com> wrote:
Can we call you "Discovery Dan"? ;)
Thanks for working on improving search results.
Pine
On Fri, Aug 7, 2015 at 1:19 PM, Dan Garry <dgarry(a)wikimedia.org> wrote:
Hello!
As part of our goal to reduce the zero results rate
<
https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q1_Goals#Search
,
the Discovery Department is currently running an A/B test to try
different
parameters for the search suggester. We're
hoping that our new parameters
will give users more suggestions without decreasing their quality.
The reason we've chosen to tweak the suggestions is because of our recent
work <https://phabricator.wikimedia.org/T105202> to automatically run
queries for the user if they get zero results but have a suggestion. The
purpose of this A/B test is to determine whether this has significant
impact towards achieving our goal or not.
This is the first A/B test that the Discovery Department has run, so
we're
still ironing out the process. We hope to run
many more A/B tests in the
future.
For further information on this, please review the associated Phabricator
task <https://phabricator.wikimedia.org/T108103>.
If you have any questions, I'd be happy to answer them.
Thanks,
Dan
--
Dan Garry
Lead Product Manager, Discovery
Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
--
Dan Garry
Lead Product Manager, Discovery
Wikimedia Foundation