Hi all,
I am a bit late in the game, but since so many questions were raised about RCom, its
scope, its goals, the source of its authority etc. and I helped coordinate it in the early
days I thought I’d chime in to clear some confusion.
Is RCom an official WMF body or a group of volunteers?
RCom was created as a volunteer body to help design policies and best practices around
research on Wikimedia projects. People who joined the committee did so on a volunteer
basis and with a variety of interests by responding to a call for participation issued by
WMF. Despite the fact that the original initiative came from WMF, its membership almost
entirely consisted of non-WMF researchers and community members (those of us who are now
with Wikimedia had no affiliation with the Foundation when RCom was launched [1]). RCom
work was and remains 100% volunteer-driven, even for those of us who are full-time
employees of the Foundation.
Is RCom a body regulating subject recruitment?
No, subject recruitment was only one among many areas of interest identified by its
participants [2]
Is RCom still alive?
RCom stopped working a while ago as a group meeting on a regular basis to discuss joint
initiatives. However, it spawned a large number of initiatives and workgroups that are
still alive and kicking, some of which have evolved into other projects that are now only
loosely associated with RCom. These include reviewing subject recruitment requests, but
also the Research Newsletter, which has been published monthly for the last 3 years;
countless initiatives in the area of open access; initiatives to facilitate Wikimedia data
documentation and data discoverability; hackathons and outreach events aimed at bringing
together researchers and Wikimedia contributors. Subject recruitment reviews and
discussions are still happening, and I believe they provide a valuable service when you
consider that they are entirely run by a microscopic number of volunteers. I don’t think
that the alternative between “either RCom exists and it functions effectively or reviews
should immediately stop” is well framed or even desirable, for the reasons that I explain
below.
What’s the source of RCom’s authority in reviewing subject recruitment requests?
Despite the perception that one of RCom’s duties would be to provide formal approval for
research projects, it was never designed to do so and it never had the power to enforce
formal review decisions. Instead, it was offered as a volunteer support service in an
effort to help minimize disruption, improve the relevance of research involving Wikimedia
contributors, sanity check the credentials of the researchers, create collaborations
between researchers working on the same topic. The lack of community or WMF policies to
back subject recruitment caused in the past few years quite some headaches, particularly
in those cases in which recruitment attempts were blocked and referred to the RCom in
order to “obtain formal approval”. The review process itself was meant to be as inclusive
as possible and not restricted to RCom participants and researchers having their proposal
reviewed were explicitly invited to address any questions or concerns raised by community
members on the talk page. I totally agree that the way in which the project templates and
forms were designed needs some serious overhaul to remove any indication of a binding
review process or a commitment for reviews to be delivered within a fixed time frame. I
cannot think of any example in which the review process discriminated some type of
projects (say qualitative research) in favor of other types of research, but I am sure
different research proposals attracted different levels of participation and interest in
the review process. My recommendation to anyone interested in designing future subject
recruitment processes is to focus on a lightweight review process open to the largest
possible number of community members but backed by transparent and enforceable policies.
It’s a really hard problem and there is simply no obvious silver bullet solution that can
be found without some experimentation and fault tolerance.
What about requests for private data?
Private data and technical support requests from WMF are a different story: they were
folded into the list of frequently asked questions hosted on the RCom section of Meta, but
by definition they require a direct and substantial involvement from the Foundation: (1)
they involve WMF as the legal entity that would be held liable for disclosing data in
breech of its privacy policies and (2) they involve paid staff resources and need to be
prioritized against a lot of other requests. There are now dedicated sections on private
data on the Wikimedia Privacy Policy [4] and Data Retention guidelines [5]. Many people,
including myself and other members of the Foundation’s Analytics team, believe that we
should try and collect the minimum amount of private data that we need in order to operate
and study our projects and make all those types of aggregate/sanitized data that we can
retain indefinitely publicly available to everyone under open licenses. We’ve already
started a process to do so and to ensure that more data (for example, data collected via
site instrumentation [6]) be exposed via Labs or other APIs, in the respect of our users’
privacy.
How can we incentivize researchers to “give back” to the community?
In the early days we drafted a set of requirements [3] to make sure we could get back as
much as possible from research involving WMF resources. It’s been hard to implement these
requirements without policies to enforce them. The suggestion of having more researchers
apply for a slot at the Research Showcase to present their work is an excellent idea that
we should consider. In general, the Research team at WMF is always interested in hearing
about incentives to drive more interest towards actionable research on Wikimedia
projects.
Dario
[1]
https://meta.wikimedia.org/w/index.php?title=Research:Committee&oldid=2…
[2]
https://meta.wikimedia.org/wiki/Research:Committee/Areas_of_interest
[3]
https://meta.wikimedia.org/wiki/Research:WMF_support
[4]
https://meta.wikimedia.org/wiki/Privacy_policy
[5]
https://meta.wikimedia.org/wiki/Data_retention_guidelines
[6]
http://meta.wikimedia.org/wiki/Research:Schemas
On Jul 29, 2014, at 6:49 PM, Aaron Halfaker <aaron.halfaker(a)gmail.com> wrote:
Either [RCOM is] functioning or its not, surely?
Well, I explained that there are functioning sub-committees still. In other words, there
are initiatives that RCOM started that are alive and successful, but we no longer
coordinate as a larger group. I don't know how else to explain it. I guess you could
say that RCOM is still functioning and that we no longer require/engage in group meetings.
As Kerry noted earlier on, the policy as it stands [1] says that researchers
"must" obtain approval through the process described. If the wording now needs
to be changed to "ought to" then surely this requires more consensus than your
single message here?
That's a proposed policy. Until it is passed by consensus, the "must" is a
proposed term. I think that it should be "must", but until that consensus is
reached, I'll continue to say that it "ought to".
Regarding researchers stating what should be regulated, I think there is a big difference
between deciding what should be regulated and being involved in the discussion of *how* it
should be regulated. Hence why I welcome participation. What I'm saying is that you
have a vested interest in not being regulated, but I'd still like to discuss how your
activities can be regulated effectively & efficiently. Does that make sense?
b) Pine suggested a board decision on this earlier one to obtain clarity and I supported
this but it was met with silence, which is why I followed up.
I welcome you to raise it to them. I don't think it is worth their time, but they
might disagree.
But what is clear is that clarification is required - especially on the distribution of
tasks between Foundation employees, the research community and Wikimedia editors. And this
is *especially* true for people outside this list.
I think that the proposed policy on English Wikipedia does that quite well. That's
why I directed people there. Also, again, I am not working on RCOM or subject recruitment
as a WMF employee. I do this in my volunteer time. This is true of all of RCOM who
happen to also be staff.
if you want process to be more clearly documented, you also have to address people like
Poitr who would rather not have processes described in detail. When you guys work out how
clearly you want a process to be described, please let me know. I'm tired of
re-spec'ing processes. This is the third iteration.
If the policy is incorrectly described on the policy pages, then someone from RCom (or
whatever it is now called) should be the one to change this - preferably with some
discussion.
Heather, that is a proposed policy page on English Wikipedia. It is not part of RCOM.
It would render RCOM irrelevant for subject recruitment concerns. That's why I
started it. I don't think that RCOM/WMF/researchers should own subject recruitment
review. I think the community being studied should own it and that RCOM/WMF/researchers
should participate.
Also, I am not your employee. This is my volunteer time. I don't have much of it,
so I focus on keeping the system running -- and it is -- and improving the system -- which
is the proposal I linked to. If you want something done and other volunteers don't
have time to do it. Do it yourself.
-Aaron
On Tue, Jul 29, 2014 at 10:23 AM, Heather Ford <hfordsa(a)gmail.com> wrote:
On Tue, Jul 29, 2014 at 10:25 AM, Aaron Halfaker <aaron.halfaker(a)gmail.com> wrote:
RCOM is not functioning as a complete group anymore.
I'm a little confused why this wasn't made clear right at the beginning of this
thread e.g. when others suggested this might be the case and you refuted them? Also,
I'm not sure what 'functioning as a complete group' actually means. Either its
functioning or its not, surely?
However, we split into sub-committees while we were still a functioning group. The
subject recruitment sub-committee and newsletter sub-committees are performing vital
functions still.
I never stated that research recruiting needs RCOM approval. I definitely said that it
"ought to" have RCOM approval.
So, does that mean that is what the policy *ought to* be now? And do you believe that
this should this be the way that the policy gets decided? Because it isn't right now
as far as I can see. As Kerry noted earlier on, the policy as it stands [1] says that
researchers "must" obtain approval through the process described. If the wording
now needs to be changed to "ought to" then surely this requires more consensus
than your single message here?
re. the comment that I (and the other researchers?) on this list shouldn't be the
ones to decide what the regulation should be, I disagree on two counts. a) It seems on the
one hand that you want this to be "self-regulation" i.e. you invited researchers
on this list to join R-COM at the beginning of this thread, but that you don't think
that the researchers here should be able to determine what to regulate. I know that
you're looking for an inclusive process but you can't have it both ways: if we are
going to help regulate, then we need to at least help decide how to regulate. b) Pine
suggested a board decision on this earlier one to obtain clarity and I supported this but
it was met with silence, which is why I followed up.
There are also more than two "review coordinators" (not not
"reviewers") -- it's just that DarTar and I have accepted the burden of
distributing work. When people are busy, we often coordinate the reviews ourselves.
I can understand your frustration; I really can! I know that you've done a lot of
really great, prior work on this and I don't think any of us are saying that we need
to throw the baby out with the bathwater. But what is clear is that clarification is
required - especially on the distribution of tasks between Foundation employees, the
research community and Wikimedia editors. And this is *especially* true for people outside
this list.
I welcome your edits to make it clear that review is optional. As you might imagine, I
have plenty of work to do and I appreciate your good-faith collaboration on improving our
research documentation.
I'm frustrated by this response. If the policy is incorrectly described on the policy
pages, then someone from RCom (or whatever it is now called) should be the one to change
this - preferably with some discussion. I find it frustrating that WMF employees are often
the ones who make the final policy pronouncements but then tell others to implement it.
And if we don't do the work, then we're apparently not assuming good faith.
This is a great opportunity to rejuvenate the process; hopefully it will eventually be
seen that way :)
Best,
Heather.
[1]
https://en.wikipedia.org/wiki/Wikipedia:Research_recruitment
-Aaron
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
_______________________________________________
Wiki-research-l mailing list
Wiki-research-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wiki-research-l