"it was difficult for contributors to tell if a survey was ethical, vetted"
Now, that's a problem of bad research design. Survey design 101 requires
that an invitation to the survey briefly discusses those issues. It all
boils down to the fact that many lazy or inadequately trained
researchers don't bother to do what is described at
https://en.wikipedia.org/wiki/Wikipedia:Ethically_researching_Wikipedia#Sur…
It is, unfortunately, not in our power to educate those researchers.
RCOM cannot do it, because most researchers will never find out it
exists, and will send invitations to their surveys or such ignoring any
required (or recommended) processes.
There's only one way that a body like RCOM could try to have some real
influence among serious Wikipedia researchers who at least have a decent
chance to finding out that it exists and what it does (like those of us
here). That is, if it had a carrot to go with the stick of (what,
exactly, I am still not sure - ban researchers accounts if they don't
follow RCOM procedure? Or just frown at them at WikiSym?). The carrot
could be a friendly user interface that would give a researcher an easy
way to sample population and send surveys to it, in exchange of jumping
through the hoops of whatever RCOM procedure creep becomes. People may
consider signing up for RCOM review or such if RCOM gives them something
of value in return. Until this happens, I don't except RCOM will become
more useful or visible than it has been for the past few years; in fact
I am predicting the continuation of its decline, as more and more people
realize its a toothless and basically unnecessary body.
--
Piotr Konieczny, PhD
http://hanyang.academia.edu/PiotrKonieczny
http://scholar.google.com/citations?user=gdV8_AEAAAAJ
http://en.wikipedia.org/wiki/User:Piotrus
On 7/18/2014 01:55, phoebe ayers wrote:
On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <hfordsa(a)gmail.com
<mailto:hfordsa@gmail.com>> wrote:
Agree with Kerry that we really need to have a more flexible
process that speaks to the main problem that (I think) RCOM was
started to solve i.e. that Wikipedians were getting tired of being
continually contacted by researchers to fill out *surveys*.
That's correct, afaik that was the original motivation, along with
some of the concerns that Lane/Nathan raised in the other list -- i.e.
that it was difficult for contributors to tell if a survey was
ethical, vetted, etc. Frankly, I think some long-term contributors
just felt jaded -- for a while it seemed there were lots of surveys
and studies to try to find out things that seemed intuitively obvious
if you were a participant in the community. I think Heather is right
that it seems like there have been fewer surveys in recent years, for
whatever reason.
Part of the problem is a somewhat subtle demographic one: while
contributors to Wikipedia do turn over, so newer contributors will not
necessarily have seen lots of surveys, very heavy editors and admins
(who are often easier to identify) tend to be long-term participants
who might have been surveyed many times. Additionally, the people who
follow mailing lists, social media, etc. (or at least the people who
speak up on those channels) skew towards very-long-term contributors
who have strong opinions and have seen it all before. So, if you
advertise your survey on the mailing list, that's the population you
get, and that's the feedback you get. (But it's a catch-22; there's
not really other obvious mass channels).
Anyway, this is a hard problem without super-obvious solutions, and
not one that there's a lot of models for -- very few online projects
are simultaneously as open with their data and as interesting for
research purposes!
best,
Phoebe
--
* I use this address for lists; send personal messages to phoebe.ayers
<at>
gmail.com <http://gmail.com> *
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l