Yes, indeed this is a quite interesting topic due to the methodological challenge (how to get a representative sample for your study) and the interference with the community.
I subscribe the proposal made by Alexander and Reid, and the rest of previous comments. Some additional context from our long past experience in this kind of initiatives with communities driven by volunteers.
The main problem for researchers in this area is that it is:
1) Difficult to find community members willing to participate.
2) Problematic to find them without interfering the normal cruise of life in the community (like "spamming" talk pages, no matter how good your initial intention was in doing it).
3) Even more important, avoiding to "bother" them more than a reasonable number of times. The first study with Apache community went on relatively well (25% or so of answers). The second attempt failed miserably. Volunteers want to spend their time maintaining Debian packages, dealing with Apache issues, providing new features to Gnome desktop apps, improving Wikipedia articles, etc. They don't want to spend too much time answering "yet another survey". And volunteer communities get burnt quite quickly in this sense, trust me.
An example of a rather participative survey has been the Wikipedia Global Survey (as far as it seems), but of course it took major support from WM Management Board itself. This could not be feasible for all the cases.
Honestly, despite our numerous previous experiences, I haven't got a perfect answer for this issues, but in the meantime, the opt-in experiments flag seems to be a good approach to try for.
Best,
F.
--- El jue, 4/6/09, Reid Priedhorsky reid@umn.edu escribió:
De: Reid Priedhorsky reid@umn.edu Asunto: Re: [Wiki-research-l] WIkipedia proposal: internal IRB for research Para: "Research into Wikimedia content and communities" wiki-research-l@lists.wikimedia.org Fecha: jueves, 4 junio, 2009 1:01 On 06/03/2009 05:29 PM, Alexander Foley wrote:
I think a better course of action might be to
establish a Wikipedia
Experiments subgroup of users who opt-in to
participate in
experiments, much like what Google does with its
experimental
features. You're limiting the sample quite a
bit, and quite possibly
only getting involved or heavily involved Wikipedia
users, but if your
core survey group is editors it would likely be
ideal.
I agree. I'd extend this notion a bit: my impression is that most people are more than happy to be solicited for studies (provided it doesn't happen too often). So I'd suggest two components:
- The opt-in defines specifically how frequently one can
be solicited (e.g. N times per year).
- The opt-in is widely pushed: highly visible on account
creation, and all existing users get one (1) invitation to opt in.
I think #2 is important because a de facto policy that only heavily involved Wikipedians participate in research would be severely limiting to the work we do.
Red
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
wiki-research-l@lists.wikimedia.org