On Tue, Jul 21, 2009 at 12:30 PM, Gregory Maxwell gmaxwell@gmail.comwrote:
On Mon, Jul 20, 2009 at 9:19 PM, Erik Moellererik@wikimedia.org wrote:
Indeed, that's the reasoning behind the proposed approach. We don't want it to typically be changing constantly for an individual user. Yes, a sequential run does introduce various problematic biases.
An IP-address based hack could work, but would need to take into account dynamic IP addresses and such, without introducing strange new biases of its own. We'll discuss a bit further - good ideas / algorithms welcome. :-)
For this the normal procedure is to give users a session cookie of some kind (either one handed out by the server or one just generated on the client) and base the selection on that.
For caching reasons I suppose you'd just want to do this all client side. Should work fine.
Alternatively, someone rigs up the front end caches to do this substitution based on IP at serving time. This would be non-trivial with squid. It would be much easier with varnish, alas.
In any case, I strongly agree with the argument against running them sequentially. Not only do you get the uncertainty from changing habits over time but later buttons will suffer from the influence of prior ones. Whatever can be done to avoid sequential testing should be done.
Didn't we do this kind of trial during the last fundraiser - with the messages at the top? They were rotated each new day weren't they? In any case, whatever we worked out for last time can't we use that method again?
wittylama.com/blog Peace, love & metadata Sent from Sydney, Nsw, Australia
foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l