Hello!
As part of our goal to reduce the zero results rate https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q1_Goals#Search, the Discovery Department is currently running an A/B test to try different parameters for the search suggester. We're hoping that our new parameters will give users more suggestions without decreasing their quality.
The reason we've chosen to tweak the suggestions is because of our recent work https://phabricator.wikimedia.org/T105202 to automatically run queries for the user if they get zero results but have a suggestion. The purpose of this A/B test is to determine whether this has significant impact towards achieving our goal or not.
This is the first A/B test that the Discovery Department has run, so we're still ironing out the process. We hope to run many more A/B tests in the future.
For further information on this, please review the associated Phabricator task https://phabricator.wikimedia.org/T108103.
If you have any questions, I'd be happy to answer them.
Thanks, Dan
Can we call you "Discovery Dan"? ;)
Thanks for working on improving search results.
Pine
On Fri, Aug 7, 2015 at 1:19 PM, Dan Garry dgarry@wikimedia.org wrote:
Hello!
As part of our goal to reduce the zero results rate < https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q1_Goals#Search
,
the Discovery Department is currently running an A/B test to try different parameters for the search suggester. We're hoping that our new parameters will give users more suggestions without decreasing their quality.
The reason we've chosen to tweak the suggestions is because of our recent work https://phabricator.wikimedia.org/T105202 to automatically run queries for the user if they get zero results but have a suggestion. The purpose of this A/B test is to determine whether this has significant impact towards achieving our goal or not.
This is the first A/B test that the Discovery Department has run, so we're still ironing out the process. We hope to run many more A/B tests in the future.
For further information on this, please review the associated Phabricator task https://phabricator.wikimedia.org/T108103.
If you have any questions, I'd be happy to answer them.
Thanks, Dan
-- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
The results of the A/B test have now been published: https://meta.wikimedia.org/wiki/Research:Reducing_Zero_Result_Searches_Throu...
In summary, there was a statistically significant difference, with the original configuration being better than the new configuration we tried. However, practically speaking, that change was so small as to be basically irrelevant, so we intend to stick with the original configuration.
Thanks, Dan
On 7 August 2015 at 13:32, Pine W wiki.pine@gmail.com wrote:
Can we call you "Discovery Dan"? ;)
Thanks for working on improving search results.
Pine
On Fri, Aug 7, 2015 at 1:19 PM, Dan Garry dgarry@wikimedia.org wrote:
Hello!
As part of our goal to reduce the zero results rate <
https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q1_Goals#Search
,
the Discovery Department is currently running an A/B test to try
different
parameters for the search suggester. We're hoping that our new parameters will give users more suggestions without decreasing their quality.
The reason we've chosen to tweak the suggestions is because of our recent work https://phabricator.wikimedia.org/T105202 to automatically run queries for the user if they get zero results but have a suggestion. The purpose of this A/B test is to determine whether this has significant impact towards achieving our goal or not.
This is the first A/B test that the Discovery Department has run, so
we're
still ironing out the process. We hope to run many more A/B tests in the future.
For further information on this, please review the associated Phabricator task https://phabricator.wikimedia.org/T108103.
If you have any questions, I'd be happy to answer them.
Thanks, Dan
-- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org