An interesting piece of corporate communication on the topic was http://blog.okcupid.com/index.php/we-experiment-on-human-beings/
I've expanded the Meta-Wiki page a bit, including the following additions: * They number in the dozens and are usually documented in the Meta-Wiki [[Research]] namespace. * Their outcome is often not used for any concrete deliverable, such as a merged change to MediaWiki core PHP code or a peer reviewed paper. * Sometimes changes which are known to be potentially harmful, and would never (or hardly) pass standard code review, are deployed as "experiments" to bypass tougher public scrutiny. (This is also valid of fundraising banners, whose poor translations since 2011 are often actively damaging to the public opinion and understanding of Wikimedia projects.)
Nemo
P.s.:
MZMcBride, 31/10/2014 04:13:
Of course the stark reality is that A/B testing on users (typically readers, not editors) during the annual Wikimedia Foundation fundraiser has been a major component of the Wikimedia Foundation's growth.
In part that's a myth. The income has been increased simply by making the banners larger, brighter, naughtier and alarming (we're in danger, bla bla). Sometimes they take more space than is left to the article; sometimes they can't be dismissed.
Of course the stark reality is that A/B testing on users (typically
readers, not editors) during the annual Wikimedia Foundation fundraiser has been a major component of the Wikimedia Foundation's growth.
In part that's a myth. The income has been increased simply by making the banners larger, brighter, naughtier and alarming (we're in danger, bla bla). Sometimes they take more space than is left to the article; sometimes they can't be dismissed.
Hi Nemo - I can't agree with this at all. The banners from the 2013 campaign (the last I can readily find) are no bigger or scarier than those from 2011. On the whole they are much less interruptive, as they are displayed less consistently; and they attract far less third-party attention than the "Jimmy banners".
The increase in efficiency through the banner campaign has been truly remarkable!
If there was a way to get the same kind of result on (say) the number of new editors who stick around and contribute more that would be great.
Chris
Chris Keating, 02/11/2014 10:52:
Hi Nemo - I can't agree with this at all. The banners from the 2013 campaign (the last I can readily find) are no bigger or scarier than those from 2011. On the whole they are much less interruptive, as they are displayed less consistently; and they attract far less third-party attention than the "Jimmy banners".
The increase in efficiency through the banner campaign has been truly remarkable!
Are you able to provide data for any of these claims?
Nemo
On 19 November 2014 11:36, Federico Leva (Nemo) nemowiki@gmail.com wrote:
Chris Keating, 02/11/2014 10:52:
The increase in efficiency through the banner campaign has been truly remarkable!
Are you able to provide data for any of these claims?
If you take a moment to do the obvious thing and look on Meta, specifically at the obvious page ([[Fundraising]]), you'll see links to that exact thing.
Seriously, were you claiming Chris was lying? Were you seriously asserting you didn't think such data existed? They've been rigorously A/B testing the banners for the past several years.
- d.
David Gerard, 19/11/2014 13:12:
The increase in efficiency through the banner campaign has been truly remarkable!
Are you able to provide data for any of these claims?
If you take a moment to do the obvious thing and look on Meta, specifically at the obvious page ([[Fundraising]]), you'll see links to that exact thing.
I've read that page several times and I'm unable to provide the data you claim it contains. I've asked data 20 months ago and it's not been provided yet. https://meta.wikimedia.org/wiki/Talk:Fundraising_2012/Report
Neom
wikimedia-l@lists.wikimedia.org