On Wed, Apr 4, 2012 at 9:36 AM, Audrey Abeyta <audrey.abeyta(a)gmail.com>wrote;wrote:
Hi Laura,
Thank you for your feedback. You're absolutely correct: I should have
specified that currency is in US dollars (I have now specified the currency
in the question text). I do, however, have a question that asks about the
respondent's country of residence. The questions in this questionnaire were
adapted from Hars & Ou (2001), so I tried to deviate from their structure
as little as possible.
I haven't read Hars & Ou. My research background is I probably best
described as education, marketing and sociology based. (My dissertation
topic is actually fundamentally about online research methods.)
When Hars & Ou did their work in 2001, were they conducting research in
online communities? And were they dealing in global populations?
By not asking both language, country and metro area, by not allowing the
expression of income in a local sense, you are creating a junk survey that
will not be repeatable. If you look at the cost of living in Texas and
compare it to Chicago, Illinois, there is a huge gulf. The cost of
housing, of petrol, the local taxes, the cost of medical care, the local
commodities in terms of food and clothing mean that $8,000 will go much,
much further in Texas than they will in Chicago. In turn, the cost of
living in Chicago will be cheap when compared to Sydney and Canberra.
These will look a bit more reasonable when you compare the cost of living
to say Tokyo or Moscow. $8,000 USD does not go very far in Chicago,
Sydney, Canberra, Tokyo, Moscow when compared to Texas.
I would STRONGLY urge you to either put in a question that asks country and
metro area, and then correct for this by adjusting for cost of living when
doing your final results. If you can't do that, I would STRONGLY urge you
to remove the question because the data will be completely meaningless.
(Minimum wage in my territory is $17.78 USD.)
Your concerns regarding the over/underrepresentation of certain segments
of the Wikipedia population are also well-founded. Because respondents are
volunteers, I am aware that there may be a large sampling bias, which I
will do my best to correct for during statistical analysis. Additionally, I
will acknowledge this limitation in the discussion section of my thesis.
How will you do sampling correction? I don't see a language connection for
one. The survey just says "Wikipedia", not "English Wikipedia" so I
assume
you're talking about all Wikipedias. If not, you will want to consider
that my own response included experiences with Simple Wikipedia. You asked
time spent editing Wikipedia, but did not ask the type of work done on the
site, nor the volume of edits done, nor the status on Wikipedia. how are
you going to correct for an over representation of English Wikipedia
contributors, female contributors, the admin core, and power contributors?
This is hugely important. If you don't have questions for allowing for
those connections, if you don't deliberately seek out minority responses
but instead advertise to a select selecting population, your results will
be fundamentally flawed and not repeatable. Given your research questions,
I suspect if we both advertised this survey, we would get differences in
answers that extremely different and STATISTICALLY significant.
The research design here just looks very, very poor and like there is very
little done to correct for groups that may have an incentive to contribute
versus occasional contributors who have less of an incentive to contribute
and complete your survey.
--
twitter: purplepopple
blog:
ozziesport.com