On Tue, Jul 21, 2009 at 12:30 PM, Gregory Maxwell <gmaxwell(a)gmail.com>wrote;wrote:
On Mon, Jul 20, 2009 at 9:19 PM, Erik
Indeed, that's the reasoning behind the
proposed approach. We don't
want it to typically be changing constantly for an individual user.
Yes, a sequential run does introduce various problematic biases.
An IP-address based hack could work, but would need to take into
account dynamic IP addresses and such, without introducing strange new
biases of its own. We'll discuss a bit further - good ideas /
algorithms welcome. :-)
For this the normal procedure is to give users a session cookie of
some kind (either one handed out by the server or one just generated
on the client) and base the selection on that.
For caching reasons I suppose you'd just want to do this all client
side. Should work fine.
Alternatively, someone rigs up the front end caches to do this
substitution based on IP at serving time. This would be non-trivial
with squid. It would be much easier with varnish, alas.
In any case, I strongly agree with the argument against running them
sequentially. Not only do you get the uncertainty from changing habits
over time but later buttons will suffer from the influence of prior
ones. Whatever can be done to avoid sequential testing should be done.
Didn't we do this kind of trial during the last fundraiser - with the
messages at the top? They were rotated each new day weren't they? In any
case, whatever we worked out for last time can't we use that method again?
Peace, love & metadata
Sent from Sydney, Nsw, Australia
foundation-l mailing list