Hi Erik,
the review of this request from Harvard/Sciences Po has been particularly painful and I want to make sure that, much as we have good measures in place to protect our editor community, we do not alienate researchers with indefinite approval requests that last for months.
Part of the problem with this request is that it never directly came under the RCom's radar until recently (a separate discussion had already started months ago between the researchers, some other members of the Foundation and the community). The process they went through, however, has been exceptionally long and it has already seen several rounds of feedback/approval from both WMF and RCom members. As Jerome notes: "we addressed and complied with all the comments and requests we received, either from the Foundation's staff or the community itself, according to our available technical means and information. The process of revising our research protocol alone has lasted for the last 6 months, including a 2 weeks engaging discussion with the community on the admin board, some Foundation_I preliminary annoucements, an RCom review, and extensive contacts with WMF staff, including you, Steven, Philippe and (to a somewhat lesser extend) Bryony and Zack. The situation from our side is becoming really costly, both on the human side because it is difficult for our team to see the study cancelled again at this stage and financially because we invested a lot in designing those successive coherent research protocols"
Because of the particular status of this request and its future implications (using CentralNotice for recruitment), I think it's acceptable to have a longer discussion (and we discussed this issue during the last RCom meeting). I don't think it's reasonable, on the other hand, to put this specific request on hold again for an indefinite amount of time. I should highlight that I learned about the Wikimedia Germany survey only days ago and in an accidental way and there is no public documentation available at the moment on the timeframe/recruitment methods of this initiative, while the Sciences Po team has been documenting and revising their request for months. I appreciate that our communication about how to handle requests for subject recruitment has not been very functional so far, but we should make sure that all requests (including those from chapters and from the Foundation itself) are documented and reviewed by the RCom to avoid unnecessary conflicts.
Due to the exceptional circumstances (i.e. the approval for this request being withdrawn again 24h before it's scheduled target date), I am happy to put some extra time and effort into this and help the team meet our new requirements so we can give them a definite target date AND approval to run their study in a timely way.
Jonathan, I'll ask the team to reply on your request for comments on the planned follow-up study if they haven't done so yet.
Best, Dario
On Jul 4, 2011, at 10:29 AM, Erik Moeller wrote:
Hi folks,
I apologize I've not been able to pay closer attention to this project; I didn't realize that it was running on such a tight timeline. There are a number of issues with the Harvard survey.
For now I've set the banner campaign to "2012" and ask that we do not run this until we've resolved the issues below. Again, I'm sorry for the last minute notice, but this entire project has moved forward fairly rapidly since it was first discussed here, and I'm afraid we may have to put it on hold for several weeks.
- This page:
http://meta.wikimedia.org/wiki/Research:Dynamics_of_Online_Interactions_and_...
as well as the current banner suggests that participants will be directed to a "login page". That login page is in fact a wiki signup sheet on meta.
If this is, per the emails going around earlier, still intended to be a survey with 2,000 participants, that's an absolutely hackish way to go about verifying participant eligibility. Everything I can see seems to indicate that it is so. Directing thousands of people to edit a wiki signup sheet is _not_ a good idea (potential for edit conflicts, vandalism, accidental IP address disclosure, etc., as well as reducing likelihood that people will go through with the whole process). Whatever this is trying to do, there are better ways to do it.
We can't go live with this if this is indeed the intent of this page. It's too much of a hack, sorry.
- From the people organizing the survey, I'd appreciate a working
link or copy of the actual survey questionnaire. The one shared previously no longer works.
- I'm also concerned about the massive banner with the Wikipedia
globe that is used for this survey in the current draft banner. This banner suggests a very strong endorsement by WMF/Wikipedia for this survey, when most people in WMF have probably never heard of the project. Moreover, we don't want to assume responsibility for technical problems, problems with the survey design, lateness of evaluation or publication, etc. We have to choose a design and approach that's appropriate to a survey undertaken by a third party.
A less prominent banner with the logos of the institutions undertaking the survey would be my preference.
- Relatedly, as a point of clarification, has this banner been coded
(as the previous editor survey) to only show once to each editor? If not, again, I'm concerned about the intensity and the risk of overexposing our editors to this invitation.
There's a major scheduling conflict between the Harvard survey and a survey that Wikimedia Germany has been preparing. The survey Wikimedia Germany has been working on is a survey concerning editor health. The project predates most of the work done in organizing and making visible Wikimedia research, and unfortunately has no Meta wiki page -- but was planned to be deployed in July.
Again, we have to be careful with the risk of oversurveying and have to stagger and schedule our surveys.
All in all, my preference would be to postpone the Harvard survey to August or September, to resolve the aforementioned issues fully.
Erik
Erik Möller Deputy Director, Wikimedia Foundation
Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate
RCom-l mailing list RCom-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/rcom-l