"We regularly see requests to survey the most active Wikipedians about
their motivations to edit."
As I am in the top 100 most active Wikipedians, unless I am an outlier
for some reason, very few of those projects come to fruition, as I get
no more than 1-2 requests a year, at most.
"See this proposal[1] for an example of a study that was halted in
review due to the disruption it would have caused."
So, they would ask 500 people to take part in a 10 minute survey. A bit
long, but... so what? I expect they'd get a response ratio of about
10%, so they should contact the Top 5000. Still not seeing a problem.
Those who don't want, don't take part in the survey. It would be nice if
the researchers promised to do something constructive like improve
Wikipedia content, give out random prizes to contributors, or such to
"give back" to the community. Perhaps an idea to add to best practices,
but... where's that disruption? What am I missing?
--
Piotr Konieczny, PhD
Does anyone know whether this is actually a problem with editors
these days?
Yes. We regularly see requests to survey the most active Wikipedians
about their motivations to edit. These requests are problematic for
some very obvious reasons. See this proposal[1] for an example of a
study that was halted in review due to the disruption it would have
caused.
The projects I do as a qualitative researcher tend to be
exploratory. I will interview people on skype, for example, about
their work on particular articles before I know that I have a project.
Do you document your study on wiki and ask for feedback about
disruption before moving forward? Regardless of the process around
it, I think we might all agree that is good behavior for any research
activity. This might be obvious to you as someone who has been doing
ethnographic work in Wikimedia communities for a long time, but it is
apprently less obvious to more junior wiki researchers.
This good-faith documentation and discussion describes the whole RCom
subject recruitment process. You refer to RCom as "heavy weight", but
as far as I can tell, the weight is entirely on the RCom coordinator
-- a burden I'll gladly accept to help good research take place
without disruption. Researchers should have already documented their
research and prepared themselves to discuss the work with their
subjects before they arrive.
I don't know of a single study that has passed stalled in RCom's
process that has resulted in substantial disruption or stalled for
more than two weeks. I welcome you to provide counter examples.
I don't think [the CSCW workshop proposal] addresses the issue
unless there's something I'm missing (like an invitation, for example!
One of the ways that researchers can be supported is through groups
that help them socialize their research activities with community
members (and minimize disruption for community members). Despite the
tone of this conversation, we have been highly successful in this regard.
I think it would be nice if you could offer an invitation to the
researchers on this list
That's the plan. We're just getting to a point where we have a solid
idea of what we want to accomplish. An announcement will come soon.
Basically, I think that we need to reassess what kinds of problems
are the most important ones right now that we want to solve rather
than resuscitating a process that was designed to address a
specific type of problem that was prevalent a long time ago
As I pointed out previously, the subject recruitment process is alive
and does not need to be "resuscitated ". It is also solving a
relevant problem. I welcome Lane Rasberry (if he has time) to share
his substantial concerns about undocumented, undiscussed research
taking place on-wiki.
1.
https://meta.wikimedia.org/wiki/Research:Online_knowledge_sharing
2.
https://meta.wikimedia.org/wiki/Category:Uncategorized_support_requests
-Aaron
On Fri, Jul 18, 2014 at 1:27 AM, Heather Ford <hfordsa(a)gmail.com
<mailto:hfordsa@gmail.com>> wrote:
On 17 July 2014 22:37, Jonathan Morgan <jmorgan(a)wikimedia.org
<mailto:jmorgan@wikimedia.org>> wrote:
First, I wanted to highlight the important issue that Heather
raises here, because although it's a separate issue, it's an
important one:
On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford
<hfordsa(a)gmail.com <mailto:hfordsa@gmail.com>> wrote:
...
One immediate requirement that I've been talking to others
about is finding ways of making the case to the WMF as a
group of researchers for the anonymization of country
level data, for example. I've spoken to a few researchers
(and I myself made a request about a year ago that hasn't
been responded to) and it seems like some work is required
by the foundation to do this anonymisation but that there
are a few of us who would be really keen to use this data
to produce research very valuable to Wikipedia -
especially from smaller language versions/developing
countries. Having an official process that assesses how
worthwhile this investment of time would be to the
Foundation would be a great idea, I think, but right now
there seems to be a general focus on the research that the
Foundation does itself rather than enabling researchers
outside. I know how busy Aaron and Dario (and others in
the team) are so perhaps this requires a new position to
coordinate between researchers and Foundation resources?
As a community-run group, RCOM doesn't have any role in making
non-public data available to researchers. However, Aaron and I
are putting together a proposal for a workshop
<https://meta.wikimedia.org/wiki/Research:CSCW15_workshop>
that would address issues like this. That's work we're doing
in an official capacity, as opposed to the RCOM work, which is
volunteer.
Jonathan, it looks like this will be a great workshop and I think
CSCW is a great venue! but I don't think it addresses the issue
unless there's something I'm missing (like an invitation, for
example! ;) I see that the workshop is forward-facing but its aim
seems to be to work with a bunch of different communities like
Reddit and GalaxyZoo. What we need are better channels as
Wikipedia researchers to communicate our needs as researchers
operating outside the WMF. And preferably in a way that doesn't
require us to have to travel to Canada to a workshop to do it!
And, I offered it as a joke but it reminds me of a small, subtle
point, I think it would be nice if you could offer an invitation
to the researchers on this list to join the workshop and/or
workshop planning when you advertise the work you're doing on
this. I know it's a wiki and anyone could probably join, but I
feel like there is enormous possibility for the group represented
here to feel involved and recognised, and I, for one, would like
to be invited sometimes.. to the fun stuff, that is, not just the
hard, arduous stuff :)
Best,
Heather.
On RCOM more generally... I think clarifying the role of the
committee, and getting a larger and more diverse set of people
involved, might help make RCOM work. But as Aaron can attest,
it is difficult to get people to agree on what RCOMs role
should be, let alone get them to work for RCOM.
I've been involved with RCOM for a while, albeit not very
actively. Unfortunately, I think that the fact that the only
people who "review" requests /happen to be*/ WMF staffers
contributes to confusion about RCOM's role and it's authority.
IMO, if RCOM or any other subject recruitment review process
is to succeed, we need:
* more wiki-researchers who are willing to regularly
participate in both peer review /and/ in developing better
process guidelines and standards (it's really just Aaron
right now)
* more /Wikipedians/ who are willing to do the same
* some degree of buy-in from the Wikimedia community as a
whole. RCOM needs legitimacy. But where, and from whom?
Subject recruitment is a global concern, but the proposed
subject recruitment process is focused on en-wiki (mostly
because that's where most of the relevant research
activities /that we are aware of/ are happening). How to
make RCOM more global?
RCOM is in a tough spot right now. We can't force researchers
to submit their proposals, or abide by the
suggestions/recommendations/decisions/whatever that result
from their review. But because we /look like /an official
body, it's easy to blame us for failing to prevent disruptive
research (if you're a community member), for "rubber stamping"
research that we like (ditto), or for drowning research in red
tape (if you're a wiki-researcher).
- J
*we were wiki-researchers first!
Heather Ford
Oxford Internet Institute <http://www.oii.ox.ac.uk>
Doctoral Programme
EthnographyMatters <http://ethnographymatters.net> |
Oxford Digital Ethnography Group
<http://www.oii.ox.ac.uk/research/projects/?id=115>
http://hblog.org <http://hblog.org/> | @hfordsa
<http://www.twitter.com/hfordsa>
On 17 July 2014 08:49, Kerry Raymond
<kerry.raymond(a)gmail.com <mailto:kerry.raymond@gmail.com>>
wrote:
Yes, I meant the community/communities of WMF. But the
authority of the community derives from WMF, which
chooses to delegate such matters. I think that
“advise” is a good word to use.
Kerry
------------------------------------------------------------------------
*From:*Amir E. Aharoni
[mailto:amir.aharoni@mail.huji.ac.il
<mailto:amir.aharoni@mail.huji.ac.il>]
*Sent:* Thursday, 17 July 2014 5:37 PM
*To:* kerry.raymond(a)gmail.com
<mailto:kerry.raymond@gmail.com>; Research into
Wikimedia content and communities
*Subject:* Re: [Wiki-research-l] discussion about
wikipedia surveys
WMF does not "own" me as a contributor;
it does not
decide who can and cannot recruit me for whatever
purposes.
I don't think that it really should be about WMF. The
WMF shouldn't enforce anything. The community can
formulate good practices for researchers and _advise_
community members not to cooperate with researchers
who don't follow these practices. Not much more is needed.
--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore
2014-07-17 8:24 GMT+03:00 Kerry Raymond
<kerry.raymond(a)gmail.com
<mailto:kerry.raymond@gmail.com>>:
Just saying here what I already put on the Talk page:
I am a little bothered by the opening sentence
"This page documents the process that researchers
must follow before asking Wikipedia contributors
to participate in research studies such as
surveys, interviews and experiments."
WMF does not "own" me as a contributor; it does
not decide who can and cannot recruit me for
whatever purposes. What WMF does own is its
communication channels to me as a contributor and
WMF has a right to control what occurs on those
channels. Also I think WMF probably should be
concerned about both its readers and its
contributors being recruited through its channels
(as either might be being recruited). I think this
distinction should be made, e.g.
"This page documents the process that researchers
must follow if they wish to use Wikipedia's
(WMF's?) communication channels to recruit people
to participate in research studies such as
surveys, interviews and experiments. Communication
channels include its mailing lists, its Project
pages, Talk pages, and User Talk pages [and
whatever else I've forgotten]."
If researchers want to recruit WPians via non-WMF
means, I don’t think it’s any business of WMF’s.
An example might be a researcher who wanted to
contact WPians via chapters or thorgs; I would
leave it for the chapter/thorg to decide if they
wanted to assist the researcher via their
communication channels.
Of course, the practical reality of it is that
some researchers (oblivious of WMF’s concerns in
relation to recruitment of WPians to research
projects) will simply use WMF’s channels without
asking nicely first. Obviously we can remove such
requests on-wiki and follow up any email requests
with the commentary that this was not an approved
request. In my category of [whatever else I’ve
forgotten], I guess there are things like Facebook
groups and any other social media presence.
Also to be practical, if WMF is to have a process
to vet research surveys, I think it has to be
sufficiently fast and not be overly demanding to
avoid the possibility of the researcher giving up
(“too hard to deal with these people”) and simply
spamming email, project pages, social media in the
hope of recruiting some participants regardless.
That is, if we make it too slow/hard to do the
right thing, we effectively encourage doing the
wrong thing. Also, what value-add can we give them
to reward those who do the right thing? It’s nice
to have a carrot as well as a stick when it comes
to onerous processes J
Because of the criticism of “not giving back”,
could we perhaps do things to try to make the
researcher feel part of the community to make
“giving back” more likely? For example, could we
give them a slot every now and again to talk about
their project in the R&D Showcase? Encourage them
to be on this mailing list. Are we at a point
where it might make sense to organise a Wikipedia
research conference to help build a research
community? Just thinking aloud here …
Kerry
------------------------------------------------------------------------
*From:*wiki-research-l-bounces@lists.wikimedia.org
<mailto:wiki-research-l-bounces@lists.wikimedia.org>
[mailto:wiki-research-l-bounces@lists.wikimedia.org
<mailto:wiki-research-l-bounces@lists.wikimedia.org>]
*On Behalf Of *Aaron Halfaker
*Sent:* Thursday, 17 July 2014 6:59 AM
*To:* Research into Wikimedia content and communities
*Subject:* Re: [Wiki-research-l] discussion about
wikipedia surveys
RCOM review is still alive and looking for new
reviewers (really, coordinators). Researchers can
be directed to me or Dario
(dtaraborelli(a)wikimedia.org
<mailto:dtaraborelli@wikimedia.org>) to be
assigned a reviewer. There is also a proposed
policy on enwiki that could use some eyeballs:
https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment
On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva
(Nemo) <nemowiki(a)gmail.com
<mailto:nemowiki@gmail.com>> wrote:
phoebe ayers, 16/07/2014 19:21:
(Personally, I think the answer should be to
resuscitate RCOM, but
that's easy to say and harder to do!)
IMHO in the meanwhile the most useful thing folks
can do is subscribing
to the feed of new research pages:
<https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom&hidebots=1&hideredirs=1&limit=500&offset=&namespace=202>
It's easier to build a functioning RCOM out of an
active community of
"reviewers", than the other way round.
Nemo
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
<mailto:Wiki-research-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
<mailto:Wiki-research-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
<mailto:Wiki-research-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
<mailto:Wiki-research-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
Jonathan T. Morgan
Learning Strategist
Wikimedia Foundation
User:Jmorgan (WMF)
<https://meta.wikimedia.org/wiki/User:Jmorgan_%28WMF%29>
jmorgan(a)wikimedia.org <mailto:jmorgan@wikimedia.org>
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
<mailto:Wiki-research-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
<mailto:Wiki-research-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l