Dear Christina,
1 are you defining your super editors by total or recent edits? Whilst we have pretty good
editor retention amongst high edit count editors, even amongst those with over a 100,000
edits there are inactive and semi active editors.
2 how are you going to ensure that talkpage invites are only responded to by the targeted
editors?
3 have you considered emailing your survey? Yes that loses you at least the 30% who
haven't set an email, but you are much more likely to get your responses from the
intended target group, also it is quite an effective way to contact the inactive and
former editors who might not see a talkpage note.
4 What are you going to do to avoid trying to survey deceased Wikipedians? Especially with
talkpage notes.
5 how does one make requests to add other questions to your survey?
6 you mention using census categories to ask the ethnicity question, may one ask whose
census, Australia, Canada, India, the UK or the USA? Also are you intending to replicate
the census questions or base your questions literally on the census categories generated
from those questions?
Regards
Jonathan Cardy
On 12 Apr 2015, at 20:49, Christina Shane-Simpson
<christinam.shane(a)gmail.com> wrote:
Hello Aaron and Other Wiki Researchers,
Thank you for responding so quickly and thoroughly to my recent proposal! Many of your
concerns align with issues I’ve been discussing with my research team, so I’m glad to hear
that we’re overlapping in that sense. Apologies in advance for the length of the
following:
- - Sampling: I completely agree with your concerns in response to the
(relatively) recent revisit to the original Gender Gap results. As an exploratory study,
I don’t think we could accurately represent the entire Wikipedia community or make causal
inferences about the community as a whole due to the voluntary nature of the survey and
the potential for inaccuracies in self-reporting. However, I’m hoping that this
preliminary project could reveal a few new patterns that might be explored in greater
depth at a later date.
Based on the Wikipedia editor rankings, I’d planned to pull the top 20% of editors and
post on their Talk Pages, giving us the “super-editor” sample. Since the two remaining
samples are more difficult to recruit, I’m currently exploring the most effective way to
obtain a randomized sample of the active (moderate) and inactive editors (infrequent
edits) – this will likely be developed with the assistance of someone more skilled in
programming than myself. I’ve also been speaking with a statistician about alternative
methods, beyond propensity-matching, where we might account for response biases that are
likely to occur. However, I’d be very open to suggestions from this community about
effectively sampling from Wikipedia and methods you’ve used to account for biases common
in these surveys.
- - Self-Report Measures of Edit History: This would only serve to verify the
editor ranking and provide a more thorough context by which the editor feels he/she makes
contributions to the Wikipedia community. Since we’ll have usernames – via Talk Pages –
as you suggested, I’d like to explore actual editing behaviors given that we’d have the
resources to do so.
- - Collaboration: Participant fatigue is a huge concern with all of these online
surveys targeting active editors. I believe you’re correct that the WMF is planning
another editor survey, but I had hoped to provide some foundation for other themes that
might be explored in these larger surveys. The prior WMF surveys didn’t provide as much
depth as we might need to reveal any patterns in editing behaviors. I’ve also reached out
to a couple of other proposals, with similar interests, to determine whether we can
compliment each other’s efforts. I think these types of collaborations are very do-able
and may help us to limit the frequency of Wikipedia editor surveys.
- Missing Measures and People: I was able to access your article, so thank you
for linking it! I’ve been reviewing the literature to clarify variables (such as the web
use you identify) to determine which should be included in the survey. In order to keep
the survey at a reasonable length, I’d hoped to capture some of these editing barriers via
themes captured in the open-ended responses. This might be particularly relevant in the
context of editors’ perceived barriers, which might vary based on the aforementioned
traits. However, I agree that the study would likely benefit form some further
questioning about editing experiences and I’ll be adding this into the proposal.
- Missing People and Sampling: Your main concern also parallels the concerns of my
research team. I’ve been speaking with my team about potentially recruiting a passive
Wikipedia user sample that would serve as a comparison. It was my original hope that a
small incentive would encourage even the infrequent editors to complete the survey
measure, but in the event that they don’t we’ll need that comparison group. Our greatest
barrier would be matching the “pertinent” comparison sample characteristics with our
super-editors. I’m not sure that we can achieve this yet, but more to come as I explore
this option.
Thank you again Aaron for your thorough feedback! As I’ve been following this listserv,
I’m incredibly grateful that we have developed such a strong research-oriented Wikipedia
community.
Sincerely,
Christina
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l