Thanks for your thoughtful reply to this survey, Laura. I support your inquiries.

On Tue, Apr 3, 2012 at 4:22 PM, Laura Hale laura@fanhistory.com wrote:

On Wed, Apr 4, 2012 at 8:53 AM, Audrey Abeyta audrey.abeyta@gmail.comwrote:

...

Dear Wikipedia contributors,

Your valuable opinions are needed regarding users' motivations to contribute to Wikipedia. This topic is currently investigated by Audrey Abeyta, an undergraduate student at the University of California, Santa Barbara. You can read a more detailed description of the project here: http://meta.wikimedia.org/wiki/Research:Motivations_to_Contribute_to_Wikiped...

Looking at this survey...

Is this intended for USAians only? I ask because of the following question:

What is your annual income?

• $0 - $8,700
• $8,701 - $35,350
• $35,351 - $85,650
• $85,651 - $178,650
• $178,651 - $388, 351
• More than $388,351

I assume the currency referenced in USD? If so, could you explicitly state this? I live in Australia, which is also on the dollar. New Zealand and Canada and Jamacia also use the dollar. The dollar is the name of their national currency, but they are not USD. Also, what are these numbers based on? Do they correlate to Cost of Living categories for AUD$ , JAM$, NZD$, CAD$? I do not see a question on the page with this question which asks which country and metro area a respondent is from. My Cost of Living in Canberra will be much different than say some one in Rockford Illinois.

If there are no controls for this, the question should probably be jettisoned as completely useless garbage data. I would also ask a question regarding native language speaking, WHAT Wikipedia a person contributes to, etc. A person from Canada who is bilingual and edits primarily edits French Wikipedia is probably going to be very different than an Indian from New Delhi who has a third language of English and is editing English Wikipedia... but the survey questions allow no distinguishing of this.

What sort of controls have been taken to insure the data represents a cross section of the Wikipedia population? And that say you don't have an over representation of women, when they make up only 10% of English Wikipedia's contributor base? Ditto with regular contributors who make over 1,000 a month, how will they be controlled for and not over represented when compared to users who make less than 100 edits a month?

-- mobile: 0412183663 twitter: purplepopple blog: ozziesport.com

---

Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l

On Wed, Apr 4, 2012 at 9:36 AM, Audrey Abeyta audrey.abeyta@gmail.comwrote:

Hi Laura,

Thank you for your feedback. You're absolutely correct: I should have specified that currency is in US dollars (I have now specified the currency in the question text). I do, however, have a question that asks about the respondent's country of residence. The questions in this questionnaire were adapted from Hars & Ou (2001), so I tried to deviate from their structure as little as possible.

I haven't read Hars & Ou. My research background is I probably best described as education, marketing and sociology based. (My dissertation topic is actually fundamentally about online research methods.)

When Hars & Ou did their work in 2001, were they conducting research in online communities? And were they dealing in global populations?

By not asking both language, country and metro area, by not allowing the expression of income in a local sense, you are creating a junk survey that will not be repeatable. If you look at the cost of living in Texas and compare it to Chicago, Illinois, there is a huge gulf. The cost of housing, of petrol, the local taxes, the cost of medical care, the local commodities in terms of food and clothing mean that $8,000 will go much, much further in Texas than they will in Chicago. In turn, the cost of living in Chicago will be cheap when compared to Sydney and Canberra. These will look a bit more reasonable when you compare the cost of living to say Tokyo or Moscow. $8,000 USD does not go very far in Chicago, Sydney, Canberra, Tokyo, Moscow when compared to Texas.

I would STRONGLY urge you to either put in a question that asks country and metro area, and then correct for this by adjusting for cost of living when doing your final results. If you can't do that, I would STRONGLY urge you to remove the question because the data will be completely meaningless. (Minimum wage in my territory is $17.78 USD.)

Your concerns regarding the over/underrepresentation of certain segments of the Wikipedia population are also well-founded. Because respondents are volunteers, I am aware that there may be a large sampling bias, which I will do my best to correct for during statistical analysis. Additionally, I will acknowledge this limitation in the discussion section of my thesis.

How will you do sampling correction? I don't see a language connection for one. The survey just says "Wikipedia", not "English Wikipedia" so I assume you're talking about all Wikipedias. If not, you will want to consider that my own response included experiences with Simple Wikipedia. You asked time spent editing Wikipedia, but did not ask the type of work done on the site, nor the volume of edits done, nor the status on Wikipedia. how are you going to correct for an over representation of English Wikipedia contributors, female contributors, the admin core, and power contributors?

This is hugely important. If you don't have questions for allowing for those connections, if you don't deliberately seek out minority responses but instead advertise to a select selecting population, your results will be fundamentally flawed and not repeatable. Given your research questions, I suspect if we both advertised this survey, we would get differences in answers that extremely different and STATISTICALLY significant.

The research design here just looks very, very poor and like there is very little done to correct for groups that may have an incentive to contribute versus occasional contributors who have less of an incentive to contribute and complete your survey.