An interesting piece of corporate communication on the topic was
http://blog.okcupid.com/index.php/we-experiment-on-human-beings/
I've expanded the Meta-Wiki page a bit, including the following additions:
* They number in the dozens and are usually documented in the Meta-Wiki
[[Research]] namespace.
* Their outcome is often not used for any concrete deliverable, such as
a merged change to MediaWiki core PHP code or a peer reviewed paper.
* Sometimes changes which are known to be potentially harmful, and would
never (or hardly) pass standard code review, are deployed as
"experiments" to bypass tougher public scrutiny. (This is also valid of
fundraising banners, whose poor translations since 2011 are often
actively damaging to the public opinion and understanding of Wikimedia
projects.)
Nemo
P.s.:
MZMcBride, 31/10/2014 04:13:
Of course the stark reality is that A/B testing on
users (typically
readers, not editors) during the annual Wikimedia Foundation fundraiser
has been a major component of the Wikimedia Foundation's growth.
In part that's a myth. The income has been increased simply by making
the banners larger, brighter, naughtier and alarming (we're in danger,
bla bla). Sometimes they take more space than is left to the article;
sometimes they can't be dismissed.