[Foundation-l] Donation Button Enhancement : Part 2

[Brian]

On Mon, Jul 20, 2009 at 6:04 PM, Robert Rohde <rarohde at gmail.com> wrote:

> On Mon, Jul 20, 2009 at 4:50 PM, Rand Montoya<rmontoya at wikimedia.org>
> wrote:
> > Wikimedians--
> >
> > As many of you know, last month we began work on exploring the
> > visibility of the donate button on all Wikimedia projects. After a long
> > comment period, we received many comments and many new ideas. Some of
> > these ideas we have incorporated into a new set of test buttons. Thank
> > you to everyone who took the time to evaluate Round 1 buttons. You can
> > see those discussions here:
> >
> http://meta.wikimedia.org/wiki/Fundraising_2009/Donation_buttons_upgrade/Round1
> >
> >
> > We have 4 designs that we will be testing on the Wikipedia:EN main skin
> > during August and the first part of September. We are going to evaluate
> > each button for one full week. This process will unfold over the next
> > two months.
> >
> > You can see the designs and timeline at this link:
> > http://meta.wikimedia.org/wiki/Fundraising_2009/Donation_buttons_upgrade
>
> Testing should be done in parallel, not in sequence.  History has
> demonstrated that donors have a tendency to respond disproportionately
> to "the new thing".  Which means that whatever button you test first
> will have an advantage over whichever one you test last.  Probably the
> easiest way to get a reasonable distribution is to vary which button
> people see based on their IP.
>
> -Robert Rohde



It's also necessary to control for seasonal traffic (and thus donation)
variations. I note that the first three button tests are at the end of
summer while the fourth coincides with the beginning of the school year. It
could be the case that there is no variation, or that the variation is
highly significant. Since nobody has looked there is no way to tell if the
test results are valid.