On Thu, Feb 21, 2013 at 5:15 PM, Steven Walling swalling@wikimedia.orgwrote:
We ended the week-long A/B test of the 'getting started' tour delivered to half the new English Wikipedians who chose a task from Special:GettingStarted after signing up. (Results forthcoming.)
Quick update: it turns out we needed to re-enable this split test until next Thursday, due to bug 45251.
Explanation for those interested in the challenges of A/B testing: this bug caused a subset of users to get logged as in the test group, but not actually delivered the tour. The group affected by this was big enough that it meant after removing them from the list, it was not a large enough dataset to declare any kind of statistical significance about what effect the tour had. Ori and Matt patched in a fix and deployed today along with some other bug fixes. I owe them both big time, and hopefully next week Dario and I will be able to declare some results about the impact of guided tours on editor activity.