On Fri, Feb 26, 2016 at 4:19 AM David Causse <dcausse(a)wikimedia.org> wrote:
yes this is not exactly what we expected :(
I guess it was too good to be true: reduce latency and improve quality at
the same time :)
On our side I'd say that perf was the main issue, Erik added a cache at
the backend-end level which seems to have a good impact.
Morelike queries are still routed to the new datacenter in dallas to
reduce stress on eqiad. We could maybe try to reroute them to eqiad and see
if caching is sufficient?
If it's the case I'd say that we don't need to run any A/B test.
Is it possible to analyze the correlation between the chosen article and
the presence of an image?
Rescoring options are slightly different for enwiki, is it possible to
have the detail for e.g. enwiki/frwiki/dewiki?
If it does not require huge effort on your side I'd say that you could run
another A/B test by disabling boostLinks, you just have to add
cirrusBoostLinks=no to api URL.
Le 26/02/2016 00:33, Erik Bernhardson a écrit :
ouch, that is not at all the result we were hoping for. Just goes to show
why we have to test these things and not just take a few examples that
perform badly in one set and look to do a better job with some different
options. Thanks for putting this together!
On Thu, Feb 25, 2016 at 3:06 PM, Dmitry Brant <dbrant(a)wikimedia.org>
As mentioned previously, the current version of the Android app contains
an A/B test where it presents "read more" suggestions to the user, based on
(a) the standard "morelike" query, or (b) the new "opening_text"
Here are the results from the last ~10 days of the test:
- The clickthrough rate using the default morelike query is (and has
been) around 15%.
- With the new opening_text query, the clickthrough rate decreases to
Therefore, it seems that the new query has a nontrivial negative effect
on CTR :(
We'll plan on removing this test in the next release of the app, but
we'll be happy to plug in a different or updated query, if it will be of
further use to Discovery.
(queries embedded as comments in the headers)
Senior Software Engineer / Product Owner (Android)
Mobile-l mailing list
Mobile-l mailing list