I think a better course of action might be to establish a Wikipedia Experiments subgroup of users who opt-in to participate in experiments, much like what Google does with its experimental features. You're limiting the sample quite a bit, and quite possibly only getting involved or heavily involved Wikipedia users, but if your core survey group is editors it would likely be ideal.
Alex Foley
On Jun 3, 2009, at 3:22 PM, Andrea Forte wrote:
Interesting proposal!
It is one thing for the community to limit the kinds of correspondence that can happen on wiki or on mailing lists if the volume becomes a nuisance for editors, but difficult to imagine prohibiting researchers from contacting individuals directly to ask about their involvement in Wikipedia. I recommend not comparing such an effort to an IRB for that reason. However laudable the intentions, I'm sure many on this list are aware of the problems that bureaucratization of research ethics has sometimes created in social science research, so IRBs (created for medical research) may not be the best model to follow anyhow. :)
Although Wikipedia as a community cannot regulate human-subjects research ethics in the same way that a government-funded agency can, it can create a set of guidelines for on-site research-related communication. My personal recommendation would not be to immediately jump to policy making or review, but to encourage "good behavior" through best practices discussions like this one.
(I have personally never gotten *any* push back in my interview studies of Wikipedians. I also realized long ago that it's not a good idea to leave messages on talk pages because they are public communication and pretty much instantly break or at least weaken confidentiality clauses in my consent forms.)
Andrea
On Wed, Jun 3, 2009 at 5:45 PM, Joseph Reagle reagle@mit.edu wrote:
On Wednesday 03 June 2009, Aaron Halfaker wrote:
This proposal isn't really about the merits of any particular study. I only offered a link to the discussions about my most recent user study because I felt it was a good example of push-back from Wikipedia editors.
I think identifying the particular scenarios (i.e., push-back) would be a good thing within the proposal, and how the proposed solution might provide remedy. Some quick thoughts:
- This is a challenging problem. (My experience soliciting
participation for interviews on WP -- left on a Project page -- was that I got no responses! :) Leaving lots of messages on people's talk pages might have generated more attention, but obviously not all of it good. (I actually didn't make much use of interviews, and got the few I did through personal/f2f contacts rather than online solicitation.) 2. I would not call it a Wikipedia IRB. 3. I think it was reasonable for Wikipedians to object to the many dozens of messages left on User pages. 4. Katherine's messages were fairly good relative to what a IRB notice/consent form would look like, but I don't see any indication of IRB. The NICE notice could be much more specific. An example of an IRB approved solicitation I used can be seen here: http://reagle.org/joseph/2006/disp/9-consent-form.html 5. I expect that: (a) some ornery folks might always complain, even if there was consensus on a Village Research Pump, (b) the Foundation would not in any way want to indemnify the work being done by someone else.
That said, anything that helps researchers develop appropriate instruments, that furthers information within the community, and that is in keeping with policy and community sensibilities is a good thing.
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l