Does anyone know whether this is actually a problem with editors these
days?
Yes. We regularly see requests to survey the most active Wikipedians about
their motivations to edit. These requests are problematic for some very
obvious reasons. See this proposal[1] for an example of a study that was
halted in review due to the disruption it would have caused.
The projects I do as a qualitative researcher tend to be exploratory. I
will interview people on skype, for example, about
their work on particular
articles before I know that I have a project.
Do you document your study on wiki and ask for feedback about disruption
before moving forward? Regardless of the process around it, I think we
might all agree that is good behavior for any research activity. This
might be obvious to you as someone who has been doing ethnographic work in
Wikimedia communities for a long time, but it is apprently less obvious to
more junior wiki researchers.
This good-faith documentation and discussion describes the whole RCom
subject recruitment process. You refer to RCom as "heavy weight", but as
far as I can tell, the weight is entirely on the RCom coordinator -- a
burden I'll gladly accept to help good research take place without
disruption. Researchers should have already documented their research and
prepared themselves to discuss the work with their subjects before they
arrive.
I don't know of a single study that has passed stalled in RCom's process
that has resulted in substantial disruption or stalled for more than two
weeks. I welcome you to provide counter examples.
I don't think [the CSCW workshop proposal] addresses the issue unless
there's something I'm missing (like an
invitation, for example!
One of the ways that researchers can be supported is through groups that
help them socialize their research activities with community members (and
minimize disruption for community members). Despite the tone of this
conversation, we have been highly successful in this regard.
I think it would be nice if you could offer an invitation to the
researchers on this list
That's the plan. We're just getting to a point where we have a solid idea
of what we want to accomplish. An announcement will come soon.
Basically, I think that we need to reassess what kinds of problems are the
most important ones right now that we want to solve
rather than
resuscitating a process that was designed to address a specific type of
problem that was prevalent a long time ago
As I pointed out previously, the subject recruitment process is alive and
does not need to be "resuscitated ". It is also solving a relevant
problem. I welcome Lane Rasberry (if he has time) to share his substantial
concerns about undocumented, undiscussed research taking place on-wiki.
1.
https://meta.wikimedia.org/wiki/Research:Online_knowledge_sharing
2.
https://meta.wikimedia.org/wiki/Category:Uncategorized_support_requests
-Aaron
On Fri, Jul 18, 2014 at 1:27 AM, Heather Ford <hfordsa(a)gmail.com> wrote:
On 17 July 2014 22:37, Jonathan Morgan
<jmorgan(a)wikimedia.org> wrote:
First, I wanted to highlight the important issue
that Heather raises
here, because although it's a separate issue, it's an important one:
On Thu, Jul 17, 2014 at 2:38 AM, Heather Ford <hfordsa(a)gmail.com> wrote:
...
One immediate requirement that I've been talking to others about is
finding ways of making the case to the WMF as a group of researchers for
the anonymization of country level data, for example. I've spoken to a few
researchers (and I myself made a request about a year ago that hasn't been
responded to) and it seems like some work is required by the foundation to
do this anonymisation but that there are a few of us who would be really
keen to use this data to produce research very valuable to Wikipedia -
especially from smaller language versions/developing countries. Having an
official process that assesses how worthwhile this investment of time would
be to the Foundation would be a great idea, I think, but right now there
seems to be a general focus on the research that the Foundation does itself
rather than enabling researchers outside. I know how busy Aaron and Dario
(and others in the team) are so perhaps this requires a new position to
coordinate between researchers and Foundation resources?
As a community-run group, RCOM doesn't have any role in making non-public
data available to researchers. However, Aaron and I are putting together a
proposal for a workshop
<https://meta.wikimedia.org/wiki/Research:CSCW15_workshop> that would
address issues like this. That's work we're doing in an official capacity,
as opposed to the RCOM work, which is volunteer.
Jonathan, it looks like this will be a great workshop and I think CSCW is
a great venue! but I don't think it addresses the issue unless there's
something I'm missing (like an invitation, for example! ;) I see that the
workshop is forward-facing but its aim seems to be to work with a bunch of
different communities like Reddit and GalaxyZoo. What we need are better
channels as Wikipedia researchers to communicate our needs as researchers
operating outside the WMF. And preferably in a way that doesn't require us
to have to travel to Canada to a workshop to do it!
And, I offered it as a joke but it reminds me of a small, subtle point, I
think it would be nice if you could offer an invitation to the researchers
on this list to join the workshop and/or workshop planning when you
advertise the work you're doing on this. I know it's a wiki and anyone
could probably join, but I feel like there is enormous possibility for the
group represented here to feel involved and recognised, and I, for one,
would like to be invited sometimes.. to the fun stuff, that is, not just
the hard, arduous stuff :)
Best,
Heather.
On RCOM more generally... I think clarifying the role of the committee,
and getting a larger and more diverse set of people involved, might help
make RCOM work. But as Aaron can attest, it is difficult to get people to
agree on what RCOMs role should be, let alone get them to work for RCOM.
I've been involved with RCOM for a while, albeit not very actively.
Unfortunately, I think that the fact that the only people who "review"
requests *happen to be** WMF staffers contributes to confusion about
RCOM's role and it's authority. IMO, if RCOM or any other subject
recruitment review process is to succeed, we need:
- more wiki-researchers who are willing to regularly participate in
both peer review *and* in developing better process guidelines and
standards (it's really just Aaron right now)
- more *Wikipedians* who are willing to do the same
- some degree of buy-in from the Wikimedia community as a whole. RCOM
needs legitimacy. But where, and from whom? Subject recruitment is a global
concern, but the proposed subject recruitment process is focused on en-wiki
(mostly because that's where most of the relevant research activities *that
we are aware of* are happening). How to make RCOM more global?
RCOM is in a tough spot right now. We can't force researchers to submit
their proposals, or abide by the
suggestions/recommendations/decisions/whatever that result from their
review. But because we *look like *an official body, it's easy to blame
us for failing to prevent disruptive research (if you're a community
member), for "rubber stamping" research that we like (ditto), or for
drowning research in red tape (if you're a wiki-researcher).
- J
*we were wiki-researchers first!
Heather Ford
Oxford Internet Institute <http://www.oii.ox.ac.uk> Doctoral Programme
EthnographyMatters <http://ethnographymatters.net> | Oxford Digital
Ethnography Group <http://www.oii.ox.ac.uk/research/projects/?id=115>
http://hblog.org | @hfordsa <http://www.twitter.com/hfordsa>
On 17 July 2014 08:49, Kerry Raymond <kerry.raymond(a)gmail.com> wrote:
Yes, I meant the community/communities of WMF.
But the authority of
the community derives from WMF, which chooses to delegate such matters. I
think that “advise” is a good word to use.
Kerry
------------------------------
*From:* Amir E. Aharoni [mailto:amir.aharoni@mail.huji.ac.il]
*Sent:* Thursday, 17 July 2014 5:37 PM
*To:* kerry.raymond(a)gmail.com; Research into Wikimedia content and
communities
*Subject:* Re: [Wiki-research-l] discussion about wikipedia surveys
> WMF does not "own" me as a contributor; it does not decide who can
and cannot recruit me for whatever purposes.
I don't think that it really should be about WMF. The WMF shouldn't
enforce anything. The community can formulate good practices for
researchers and _advise_ community members not to cooperate with
researchers who don't follow these practices. Not much more is needed.
--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore
2014-07-17 8:24 GMT+03:00 Kerry Raymond <kerry.raymond(a)gmail.com>om>:
Just saying here what I already put on the Talk page:
I am a little bothered by the opening sentence "This page documents the
process that researchers must follow before asking Wikipedia contributors
to participate in research studies such as surveys, interviews and
experiments."
WMF does not "own" me as a contributor; it does not decide who can and
cannot recruit me for whatever purposes. What WMF does own is its
communication channels to me as a contributor and WMF has a right to
control what occurs on those channels. Also I think WMF probably should be
concerned about both its readers and its contributors being recruited
through its channels (as either might be being recruited). I think this
distinction should be made, e.g.
"This page documents the process that researchers must follow if they
wish to use Wikipedia's (WMF's?) communication channels to recruit people
to participate in research studies such as surveys, interviews and
experiments. Communication channels include its mailing lists, its Project
pages, Talk pages, and User Talk pages [and whatever else I've forgotten]."
If researchers want to recruit WPians via non-WMF means, I don’t think
it’s any business of WMF’s. An example might be a researcher who wanted to
contact WPians via chapters or thorgs; I would leave it for the
chapter/thorg to decide if they wanted to assist the researcher via their
communication channels.
Of course, the practical reality of it is that some researchers
(oblivious of WMF’s concerns in relation to recruitment of WPians to
research projects) will simply use WMF’s channels without asking nicely
first. Obviously we can remove such requests on-wiki and follow up any
email requests with the commentary that this was not an approved request.
In my category of [whatever else I’ve forgotten], I guess there are things
like Facebook groups and any other social media presence.
Also to be practical, if WMF is to have a process to vet research
surveys, I think it has to be sufficiently fast and not be overly demanding
to avoid the possibility of the researcher giving up (“too hard to deal
with these people”) and simply spamming email, project pages, social media
in the hope of recruiting some participants regardless. That is, if we make
it too slow/hard to do the right thing, we effectively encourage doing the
wrong thing. Also, what value-add can we give them to reward those who do
the right thing? It’s nice to have a carrot as well as a stick when it
comes to onerous processes J
Because of the criticism of “not giving back”, could we perhaps do
things to try to make the researcher feel part of the community to make
“giving back” more likely? For example, could we give them a slot every now
and again to talk about their project in the R&D Showcase? Encourage them
to be on this mailing list. Are we at a point where it might make sense to
organise a Wikipedia research conference to help build a research
community? Just thinking aloud here …
Kerry
------------------------------
*From:* wiki-research-l-bounces(a)lists.wikimedia.org [mailto:
wiki-research-l-bounces(a)lists.wikimedia.org] *On Behalf Of *Aaron
Halfaker
*Sent:* Thursday, 17 July 2014 6:59 AM
*To:* Research into Wikimedia content and communities
*Subject:* Re: [Wiki-research-l] discussion about wikipedia surveys
RCOM review is still alive and looking for new reviewers (really,
coordinators). Researchers can be directed to me or Dario (
dtaraborelli(a)wikimedia.org) to be assigned a reviewer. There is also
a proposed policy on enwiki that could use some eyeballs:
https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment
On Wed, Jul 16, 2014 at 11:16 AM, Federico Leva (Nemo) <
nemowiki(a)gmail.com> wrote:
phoebe ayers, 16/07/2014 19:21:
> (Personally, I think the answer should be to resuscitate RCOM, but
> that's easy to say and harder to do!)
IMHO in the meanwhile the most useful thing folks can do is subscribing
to the feed of new research pages:
<
https://meta.wikimedia.org/w/index.php?title=Special:NewPages&feed=atom…
>
It's easier to build a functioning RCOM out of an active community of
"reviewers", than the other way round.
Nemo
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
--
Jonathan T. Morgan
Learning Strategist
Wikimedia Foundation
User:Jmorgan (WMF) <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)>
jmorgan(a)wikimedia.org
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l