Thanks for sharing this and notifying us on the list.
Overall, I am very supportive of additional attempts to do more rigorous
survey research on Wikipedians. Some questions that I think you could try
to address in the proposal:
- *Sampling*: You mention that you plan to stratify your sample based on
past edit history and recruit via talk page messages. However, beyond this
you say nothing about the logistics of subject sampling, recruitment, or
any approaches you will take to address the fact that conducting
representative surveys in online communities is very, very difficult. Can
you elaborate on this aspect of your study? In particular, how will your
approach address shortcomings in data and sample quality that have affected
of Wikipedia contributors?
- *Self-report measures of edit history: *Why ask the respondents to
self-report their edit histories (this kind of thing is notoriously hard to
do accurately) when you could ask them to provide their usernames or at
least link their usernames to their survey responses (since you're
recruiting via talk page messages anyway)?
- *Collaboration w related studies: *There are several other ongoing
efforts to survey wikipedians -- even at least one other one
<http://dl.acm.org/citation.cfm?id=2145264>(link to "official"
is gated, but other versions are available for free) focused on social
psychological concerns. Also, my impression is that the WMF is involved in
planning another editor survey in the near future. How will your approach
complement/extend/overlap with these other efforts? Will you make any
effort to collaborate with these ongoing studies? How will your study avoid
subject exhaustion -- especially among more active wikipedians who may find
themselves invited to participate in many surveys?
- *Missing measures and missing people:* Previous studies have shown that a
variety of additional factors may figure in shaping the participation
practices of Wikipedians as well as those who might edit Wikipedia but
choose not to do so. For example, in a recent paper
gated link, but I am also happy to provide copies to those who would like
access) that I co-wrote w Eszter Hargittai, we find that web use skills
are, in some ways, even more robust predictors of wikipedia contribution
than gender. There are many other examples of important measures that
predict participation in various ways as well, whether it be individual's
trust/caution attitudes, newcomer experiences, etc. Which of these measures
will you include? How will you ensure that you have included the most
important measures in this survey study since survey results are otherwise
quitre prone to omitted variable bias?
*Missing people and sampling on the dependent variable: *Maybe most
importantly, insofar as you say that you are interested in understanding
factors that determine who edits, you are selecting on the dependent
variable (wikipedia editing) by limiting your study to individuals who have
accounts on the encyclopedia and edit already. It strikes me as especially
egregious that you are requiring survey respondents to read and reply to
the survey recruitment materials via talk page message. This means that
precisely those individuals who participate least (and who would provide
your study with necessary variation on the outcome of interest) are the
least likely to respond and to be included in the study. As a result, I
fear that your findings will not speak to these questions effectively
unless you find an alternative method of sampling and recruitment.
I hope that these comments are helpful for you as you continue to refine
the study design. I really think you're pursuing a critical set of concerns
in this study and I am eager to see it succeed in the most effective way
On Sun, Apr 12, 2015 at 8:44 AM, Christina Shane-Simpson <
Hello Fellow Wiki Researchers,
I’ve recently posted a project proposal under the Inspire Campaign and
would love feedback from this community on the research proposal, *Characterization
of Editors on Wikipedia*:
In order to accurately explore the main goals of the Inspire Campaign,
we must be able to effectively characterize our community. Any
interventions that we develop should reflect and match the needs of the
target population, requiring a thorough understanding of the traits and
behaviors of our community of editors. As a direct extension of the recent
gender gap research on Wikipedia and to explore other potential areas of
inequality, we’d like to conduct another study that compares the traits of
the super-editor, the active editor (moderate editing), and the inactive
editor (infrequent edits).
The proposed project would use an online self-report survey that is posted
on editor talk pages. The research team has experience conducting online
surveys and will monitor responses on this survey to identify any potential
misuse of the survey (i.e. vandalism) and/or outliers in the data. This
entire project would only be implemented after an IRB approval from the
lead researcher's academic institution.
Thank you in advance for your assistance in developing this proposal!
The College of Staten Island and
The Graduate Center, CUNY