2009/7/21 Robert Rohde <rarohde(a)gmail.com>om>:
Testing should be done in parallel, not in sequence.
History has
demonstrated that donors have a tendency to respond disproportionately
to "the new thing". Which means that whatever button you test first
will have an advantage over whichever one you test last. Probably the
easiest way to get a reasonable distribution is to vary which button
people see based on their IP.
Or simply to randomise it entirely.
If either of those aren't possible for technical reasons, it might be
practical to rotate them - run each button for x many hours at a
stretch, rotating them so as to ensure they don't regularly go up at
the same time (of the day or of the week) and so that they get roughly
equal coverage.
--
- Andrew Gray
andrew.gray(a)dunelm.org.uk