[Wikimedia-l] Fundraising updates?

[James Salsman]

Zack,

Thanks very much for helping me understand this.

> We can only do big multivariate tests for banner click rates.

The multivariate tests you ran in May and October list total
donations. What am I missing?

> For example, the new banners have about 30% the click rate of the old ones,
> but they make about 3 or 4 times as much money.

If you have any reason to believe that this is not because of the new
pull-down format, please let me know.

> To determine how well a banner message does for donations, we usually need a
> sample size between 500 and 5,000 donations per banner [requiring] from 30
> minutes to several hours to collect -- if we're only testing two banners at a time.

How about using a much quicker 90% confidence interval on a larger
number of tests, and then confirming the top, say ten performers, at
the 95% confidence level you've been using in subsequent tests?

>.... We tested tons of those banners. I think that we tested
> virtually every different (serious) theme that was suggested.

I'm sorry, but "tons" is not a number. You didn't even get a third of
them and a quick perusal shows that you missed hundreds of sincere
serious submissions.

> They all had BOTH far lower click rates and even lower donation rates

I believe using the word "far" is a factual mistake. I plotted a
sample distribution, and I know that the variance is more than enough
that the population would exceed the top of the sample. Moreover,
there were many discovered which came very close to the "Please read
an appeal from Jimmy" top performer in 2009 and 2010. Isn't that how
you discovered the "If everyone donated $x this fundraiser would be
over in y" which was the top performer for a time?

> usually by orders of magnitude.

That is technically true, but as they fit a lognormal distribution,
it's not surprising.

> Now we're pretty clear on why: A short slogan isn't enough to get people
> over all their questions about why they should support Wikipedia. More text
> was needed. In our marketing-slogan-obsessed culture, the idea that we'd
> have to present people with a long paragraph was very counterintuitive....
> As it turns out, Wikipedia users DO like to read -- and want all the facts
> before they donate.

I completely agree with this, but I note that the longer banners you
started with this month were comprised of three shorter statements,
one of which was an appeal statement.

Let me just cut to the chase and ask it this way: If you were to test
the remaining volunteer submissions in the appeal statement slot in
the existing top performing, fact-based, pull-down appeal,
(1) What would you need that you don't already have?
(2) How long would it take at the 90% confidence level?
(3) How can volunteers help make it easier and faster for you?