Hi Jan,

I don't know of documented examples (the A/B testing design depends on the question you want to answer). If you want to chat about this more, I'd be happy to brainstorm with you about your options. Message me off-list and we can set up a time if that's helpful.

Best, Leila

Leila Zia Senior Research Scientist Wikimedia Foundation

On Wed, Sep 7, 2016 at 12:16 AM, Jan Dittrich jan.dittrich@wikimedia.de wrote:

Hello Analytics,

A while ago I asked about the existence of any A/B-Testing Framework. I got to know (Thanks, Nuria!) that https://phabricator.wikimedia. org/T135762 is in preparation. However, I assume, that until this is in place, we need to use custom solutions which utilize EventLogging. Event logging itself is pretty clear to me, but not the splitting/cookie logic.

Could anybody link me some examples for such a self implemented way to show users their assigned content and, if they are not assigned to a group yet, to assign users to A/B… bins?

Jan

-- Jan Dittrich UX Design/ User Research

Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30 219 158 26-0 http://wikimedia.de

Imagine a world, in which every single human being can freely share in the sum of all knowledge. That‘s our commitment.

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

---

Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

On Wed, 7 Sep 2016 at 00:16 Jan Dittrich jan.dittrich@wikimedia.de wrote:

Hello Analytics,

A while ago I asked about the existence of any A/B-Testing Framework. I got to know (Thanks, Nuria!) that https://phabricator.wikimedia.org/T135762 is in preparation. However, I assume, that until this is in place, we need to use custom solutions which utilize EventLogging. Event logging itself is pretty clear to me, but not the splitting/cookie logic.

Could anybody link me some examples for such a self implemented way to show users their assigned content and, if they are not assigned to a group yet, to assign users to A/B… bins?

Here's the code for MW-VE to add a proportion of new accounts to have VE on by default, used when slowly ramping up deployment or for an A/B test: https://phabricator.wikimedia.org/diffusion/EVED/browse/master/VisualEditor.... — hope that helps.

J.