On 17 July 2014 17:55, phoebe ayers <phoebe.wiki(a)gmail.com> wrote:
Part of the problem is a somewhat subtle demographic one: while
contributors to Wikipedia do turn over, so newer contributors will not
necessarily have seen lots of surveys, very heavy editors and admins (who
are often easier to identify) tend to be long-term participants who might
have been surveyed many times. Additionally, the people who follow mailing
lists, social media, etc. (or at least the people who speak up on those
channels) skew towards very-long-term contributors who have strong opinions
and have seen it all before. So, if you advertise your survey on the
mailing list, that's the population you get, and that's the feedback you
get. (But it's a catch-22; there's not really other obvious mass channels).
This is a really important insight, thanks for sharing it, Phoebe. It's
important to work out what the problem is that we're trying to solve before
we try solving it! If the key problem here is that Wikipedians need to be
protected from researchers constantly surveying them, and actually the
wide-ranging surveys are really rare these days, then maybe the problem is
with heavy editors and admins being constantly 'surveyed' (although I'm
guessing that this is not the only research method being used as I talk
about below).
Does anyone know whether this is actually a problem with editors these
days? I know that I have interviewed a bunch of editors over the years
without RCOM approval (some with RCOM approval) and I have only had good
experiences. Sure there were people who didn't want to be interviewed, but
they just ignored my requests - I'm not sure that they would say that they
were bothered enough that an entire process needed to be developed to
approve projects.
I think part of the problem here is that there is a bias towards particular
types of research projects in the way that RCOM was designed. I do both
quantitative and qualitative research on WP and the quantitative research
nowadays focuses mostly on capturing large-scale user actions using the API
or the dumps - I have a feeling that's why there are fewer surveys these
days - more researchers are using the data to conduct research and (right
now) that doesn't require any permissions beyond what is required by uni
ethics board (and all the problems that come with that!).
The projects I do as a qualitative researcher tend to be exploratory. I
will interview people on skype, for example, about their work on particular
articles before I know that I have a project. I could certainly develop a
proposal to RCOM but it would be so wide-ranging that I'm unsure what the
actual benefit was. I think that a much bigger problem is actually
developing community guidelines around ethical treatment of subjects who
don't often realise that their comments and interactions can be legally
(but, I believe not necessarily ethically) used without their permission (I
wrote something about my thoughts on this here [1]).
Basically, I think that we need to reassess what kinds of problems are the
most important ones right now that we want to solve rather than
resuscitating a process that was designed to address a specific type of
problem that was prevalent a long time ago. The new problems that I see
right now that a research community is best placed to solve are things like:
- developing community guidelines for the representation of editors'
identities in research (similar, perhaps to the AOIR guidelines [2]);
- finding ways of making responsible requests to the WMF for data that they
hold that might benefit research outside the WMF;
- developing opportunities for researchers to collaborate and share what
they're doing with the wider research community (as Kerry suggests).
[1]
http://ethnographymatters.net/blog/2013/06/27/onymous-pseudonymous-neither-…
[2]
http://aoir.org/reports/ethics2.pdf
Best,
Heather.
Anyway, this is a hard problem without super-obvious solutions, and not
one that there's a lot of models for -- very few online projects are
simultaneously as open with their data and as interesting for research
purposes!
best,
Phoebe
--
* I use this address for lists; send personal messages to phoebe.ayers
<at>
gmail.com *
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l