On 17 July 2014 17:55, phoebe ayers <phoebe.wiki@gmail.com> wrote:

Part of the problem is a somewhat subtle demographic one: while contributors to Wikipedia do turn over, so newer contributors will not necessarily have seen lots of surveys, very heavy editors and admins (who are often easier to identify) tend to be long-term participants who might have been surveyed many times. Additionally, the people who follow mailing lists, social media, etc. (or at least the people who speak up on those channels) skew towards very-long-term contributors who have strong opinions and have seen it all before. So, if you advertise your survey on the mailing list, that's the population you get, and that's the feedback you get. (But it's a catch-22; there's not really other obvious mass channels).

This is a really important insight, thanks for sharing it, Phoebe. It's important to work out what the problem is that we're trying to solve before we try solving it! If the key problem here is that Wikipedians need to be protected from researchers constantly surveying them, and actually the wide-ranging surveys are really rare these days, then maybe the problem is with heavy editors and admins being constantly 'surveyed' (although I'm guessing that this is not the only research method being used as I talk about below).

Does anyone know whether this is actually a problem with editors these days? I know that I have interviewed a bunch of editors over the years without RCOM approval (some with RCOM approval) and I have only had good experiences. Sure there were people who didn't want to be interviewed, but they just ignored my requests - I'm not sure that they would say that they were bothered enough that an entire process needed to be developed to approve projects.

I think part of the problem here is that there is a bias towards particular types of research projects in the way that RCOM was designed. I do both quantitative and qualitative research on WP and the quantitative research nowadays focuses mostly on capturing large-scale user actions using the API or the dumps - I have a feeling that's why there are fewer surveys these days - more researchers are using the data to conduct research and (right now) that doesn't require any permissions beyond what is required by uni ethics board (and all the problems that come with that!).

The projects I do as a qualitative researcher tend to be exploratory. I will interview people on skype, for example, about their work on particular articles before I know that I have a project. I could certainly develop a proposal to RCOM but it would be so wide-ranging that I'm unsure what the actual benefit was. I think that a much bigger problem is actually developing community guidelines around ethical treatment of subjects who don't often realise that their comments and interactions can be legally (but, I believe not necessarily ethically) used without their permission (I wrote something about my thoughts on this here [1]).

Basically, I think that we need to reassess what kinds of problems are the most important ones right now that we want to solve rather than resuscitating a process that was designed to address a specific type of problem that was prevalent a long time ago. The new problems that I see right now that a research community is best placed to solve are things like:

- developing community guidelines for the representation of editors' identities in research (similar, perhaps to the AOIR guidelines [2]);

- finding ways of making responsible requests to the WMF for data that they hold that might benefit research outside the WMF;

- developing opportunities for researchers to collaborate and share what they're doing with the wider research community (as Kerry suggests).

[1] http://ethnographymatters.net/blog/2013/06/27/onymous-pseudonymous-neither-or-both/

[2] http://aoir.org/reports/ethics2.pdf

Best,

Heather.

Anyway, this is a hard problem without super-obvious solutions, and not one that there's a lot of models for -- very few online projects are simultaneously as open with their data and as interesting for research purposes!

best,
Phoebe

--
* I use this address for lists; send personal messages to phoebe.ayers <at> gmail.com *

_______________________________________________
Wiki-research-l mailing list
Wiki-research-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l