Hi Erik,
the review of this request from Harvard/Sciences Po has been particularly painful and I
want to make sure that, much as we have good measures in place to protect our editor
community, we do not alienate researchers with indefinite approval requests that last for
months.
Part of the problem with this request is that it never directly came under the RCom's
radar until recently (a separate discussion had already started months ago between the
researchers, some other members of the Foundation and the community). The process they
went through, however, has been exceptionally long and it has already seen several rounds
of feedback/approval from both WMF and RCom members. As Jerome notes: "we addressed
and complied with all the comments and requests we received, either from the
Foundation's staff or the community itself, according to our available technical means
and information. The process of revising our research protocol alone has lasted for the
last 6 months, including a 2 weeks engaging discussion with the community on the admin
board, some Foundation_I preliminary annoucements, an RCom review, and extensive contacts
with WMF staff, including you, Steven, Philippe and (to a somewhat lesser extend) Bryony
and Zack. The situation from our side is becoming really costly, both on the human side
because it is difficult for our team to see the study cancelled again at this stage and
financially because we invested a lot in designing those successive coherent research
protocols"
Because of the particular status of this request and its future implications (using
CentralNotice for recruitment), I think it's acceptable to have a longer discussion
(and we discussed this issue during the last RCom meeting). I don't think it's
reasonable, on the other hand, to put this specific request on hold again for an
indefinite amount of time. I should highlight that I learned about the Wikimedia Germany
survey only days ago and in an accidental way and there is no public documentation
available at the moment on the timeframe/recruitment methods of this initiative, while the
Sciences Po team has been documenting and revising their request for months. I appreciate
that our communication about how to handle requests for subject recruitment has not been
very functional so far, but we should make sure that all requests (including those from
chapters and from the Foundation itself) are documented and reviewed by the RCom to avoid
unnecessary conflicts.
Due to the exceptional circumstances (i.e. the approval for this request being withdrawn
again 24h before it's scheduled target date), I am happy to put some extra time and
effort into this and help the team meet our new requirements so we can give them a
definite target date AND approval to run their study in a timely way.
Jonathan, I'll ask the team to reply on your request for comments on the planned
follow-up study if they haven't done so yet.
Best,
Dario
On Jul 4, 2011, at 10:29 AM, Erik Moeller wrote:
Hi folks,
I apologize I've not been able to pay closer attention to this
project; I didn't realize that it was running on such a tight
timeline. There are a number of issues with the Harvard survey.
For now I've set the banner campaign to "2012" and ask that we do not
run this until we've resolved the issues below. Again, I'm sorry for
the last minute notice, but this entire project has moved forward
fairly rapidly since it was first discussed here, and I'm afraid we
may have to put it on hold for several weeks.
1) This page:
http://meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and…
as well as the current banner suggests that participants will be
directed to a "login page". That login page is in fact a wiki signup
sheet on meta.
If this is, per the emails going around earlier, still intended to be
a survey with 2,000 participants, that's an absolutely hackish way to
go about verifying participant eligibility. Everything I can see seems
to indicate that it is so. Directing thousands of people to edit a
wiki signup sheet is _not_ a good idea (potential for edit conflicts,
vandalism, accidental IP address disclosure, etc., as well as reducing
likelihood that people will go through with the whole process).
Whatever this is trying to do, there are better ways to do it.
We can't go live with this if this is indeed the intent of this page.
It's too much of a hack, sorry.
2) From the people organizing the survey, I'd appreciate a working
link or copy of the actual survey questionnaire. The one shared
previously no longer works.
3) I'm also concerned about the massive banner with the Wikipedia
globe that is used for this survey in the current draft banner. This
banner suggests a very strong endorsement by WMF/Wikipedia for this
survey, when most people in WMF have probably never heard of the
project. Moreover, we don't want to assume responsibility for
technical problems, problems with the survey design, lateness of
evaluation or publication, etc. We have to choose a design and
approach that's appropriate to a survey undertaken by a third party.
A less prominent banner with the logos of the institutions undertaking
the survey would be my preference.
4) Relatedly, as a point of clarification, has this banner been coded
(as the previous editor survey) to only show once to each editor? If
not, again, I'm concerned about the intensity and the risk of
overexposing our editors to this invitation.
5)
There's a major scheduling conflict between the Harvard survey and a
survey that Wikimedia Germany has been preparing. The survey Wikimedia
Germany has been working on is a survey concerning editor health. The
project predates most of the work done in organizing and making
visible Wikimedia research, and unfortunately has no Meta wiki page --
but was planned to be deployed in July.
Again, we have to be careful with the risk of oversurveying and have
to stagger and schedule our surveys.
All in all, my preference would be to postpone the Harvard survey to
August or September, to resolve the aforementioned issues fully.
Erik
--
Erik Möller
Deputy Director, Wikimedia Foundation
Support Free Knowledge:
http://wikimediafoundation.org/wiki/Donate
_______________________________________________
RCom-l mailing list
RCom-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l