Phoebe writes:
This is a very small, self-selected sample; there would be
no harm or cost associated with turning it on for a much larger percentage (or all) of logged-in users on the top-ten languages, not just English or German alone, which both have peculiarities associated with being the largest Wikipedia communities.
Is there a version of the survey that does *not* entail a self-selected sample? The methodologist in me wants to know, because it seems to me that selection bias is inherent in any survey of this sort. (What's more, it seems fairly predictable in which direction that bias would skew results.)
--Mike
2009/3/4 Mike Godwin mnemonic@gmail.com:
Phoebe writes:
This is a very small, self-selected sample; there would be
no harm or cost associated with turning it on for a much larger percentage (or all) of logged-in users on the top-ten languages, not just English or German alone, which both have peculiarities associated with being the largest Wikipedia communities.
Is there a version of the survey that does *not* entail a self-selected sample? The methodologist in me wants to know, because it seems to me that selection bias is inherent in any survey of this sort. (What's more, it seems fairly predictable in which direction that bias would skew results.)
Well, you could block access to all Wikimedia sites until someone from your IP address has completely the survey. I wouldn't recommend it, though!
On Wed, Mar 4, 2009 at 11:08 AM, Mike Godwin mnemonic@gmail.com wrote:
Phoebe writes:
This is a very small, self-selected sample; there would be
no harm or cost associated with turning it on for a much larger percentage (or all) of logged-in users on the top-ten languages, not just English or German alone, which both have peculiarities associated with being the largest Wikipedia communities.
Is there a version of the survey that does *not* entail a self-selected sample? The methodologist in me wants to know, because it seems to me that selection bias is inherent in any survey of this sort. (What's more, it seems fairly predictable in which direction that bias would skew results.)
--Mike
This is true -- though I'm no statistician and I'd love to hear from those who are (Robert?) I have wrestled with this question before and I'm not sure there's any way to get a non-self-selected survey about anything on the projects due to anonymity concerns. However, I'm mostly concerned with:
* getting responses from more language communities * it seems there may be a difference between those who see the notice popup on their watchlists (the 5%) and those who chose to go to the survey to take it (people like me, who really care about this issue). For instance, we ran an item about the survey in yesterday's English-Wikipedia signpost; enough people read this that could cause a spike in responses. I'm not sure if this is a valid methodological concern or not -- or perhaps we are mostly interested in getting responses from people who really care? I'm not sure. * the short time frame arguably leaves out the class of editors who only log in occasionally; their responses may be different from the editing-every-day crowd due to a qualitative difference in participation. (Or not! Who knows).
Also, some clarity in what each of the options means would be good; the question about participation in foundation activities in particular seemed a bit vague. More to the point, while I agree it's interesting to know what infrequent-editors versus heavy-editors think about the question, how is participation level and thoughts on attribution going to be correlated? Do one's thoughts matter less if one is an infrequent editor? Etc.
One quick way to get some expert feedback about all these questions would be to submit the survey design to wiki-research-l, where the researchers hang out.
-- phoebe
wikimedia-l@lists.wikimedia.org