WSC, thanks for your note, which gives me the opportunity to clarify some gaps in the
current RCom review procedure.
There are two separate considerations that apply to this project specifically:
1) the researcher asked us whether the integration of the questionnaire into features WMF
is currently developing (such as MoodBar) was feasible from a technical standpoint. I
consulted my colleagues in the product team as the question of available resources and
technical feasibility needs to be settled on their side independently of the RCom's
assessment of the proposal. I could have replied privately about this request but I
thought it'd be useful to make my response public to allow other RCom members to
review it.
2) the researcher shared with me a list of questions that he is planning to use for this
study and specifically asked me not to circulate this document publicly. He's
comfortable about sharing this document individually with RCom members as long as it's
not posted on a publicly indexed mailing list.
This addresses two separate questions you're asking:
• is there some information that would be inappropriate to post on public fora as part of
the review of a research proposal?
• are there any parts of a research proposal whose assessment is of exclusive WMF
competence and not of the community or the Research Committee?
The answer to both questions is yes.
There are many cases I can think of in which a public forum is not appropriate for
reviewing research proposals. On top of the Wiki Anxiety case (a research protocol that
the researcher doesn't want to be publicly circulated until the research is actually
run), there are other cases I've come across that fall under the same category.
Authors of grant proposals who seek support from WMF in the form of, say, a non-committal
letter of endorsement won't be able to publicly share the contents of the proposal
while it's being evaluated as this would undermine the funder's decision. We might
decide that RCom provides no support to projects that cannot be fully and transparently
documented on Meta at the time of the review, but that would certainly leave us with a
bunch of undergrad surveys as the only kind of collaboration requests we receive and I am
not sure this is the direction we want to take. I think it's legitimate for a
researcher to ask that under specific conditions a proposal (or parts thereof) should not
be immediately disclosed until it's thoroughly reviewed AND funded. This probably
means that we should use tools like OTRS to manage the parts of a request that cannot be
discussed publicly for the above reasons. In the specific case of the Wiki Anxiety
protocol, I am happy to share the annexes of the proposal with any of you, please let me
know off-list if you wish to read it.
The second consideration is that there are multiple cases in which a proposal (or parts
thereof) needs to be assessed by WMF independently and prior to RCom's own assessment.
These include:
• projects that require the allocation of technical resources or engineer time from WMF;
• projects that require access to private data (in ways that are compliant with our
privacy policy) and engineer supervision on this process;
• projects that require a review from our Legal team;
This doesn't mean bypassing RCom but ensuring that the project meets the minimum
technical requirements to be implemented (without which it's pointless for the
researcher to invest further effort).
I am sorry to hear you felt disconnected from the review process, I think your feedback on
this and other projects is extremely helpful, but please let me know if the above
distinction answers your concerns.
The question of reviewing some aspects of a project via private channels is definitely an
issue that deserves further discussion and I'd like to hear how RCom members feel
about it.
Dario
On Sep 8, 2011, at 1:24 PM, WereSpielChequers wrote:
Hi Dario,
I asked if I could see the questionnaire. None of my talk threads have been responded to,
but you've added a new thread that includes the sentence "I don't know
whether you would be using only a subset of the questions listed in the Wiki
Anxiety/Usability Inventory (a document that was shared privately with WMF), but the
questionnaire in its current state (46+22 questions)". So I now know how long it is,
but it has been shared privately and so presumably I can't have it.
Which leaves me feeling somewhat disconnected from the process.
I don't know whether the researcher has been asked if the survey can be shared with
WMF volunteers on RCOM, or why the secrecy.
Currently our emails are logged on the Internet for all to see, so I'm not sure how
we handle the situation where someone is willing to engage with just the WMF. I suppose
moving to OTRS resolves that, though it does leave me wondering how we maintain
transparency to the community. I suppose they key thing will be to set some boundaries as
to what we are and are not prepared to do in Private.
WSC
On 8 September 2011 20:21, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote:
I don't understand, that was my feedback to the researcher based on discussions I had
with other product people at WMF (regarding technical aspects of the proposed integration
with MoodBar, which are of direct WMF competence)
Dario
On Sep 8, 2011, at 12:16 PM, WereSpielChequers wrote:
Apparently he's answered my query, but only
to the WMF
http://meta.wikimedia.org/wiki/Research_talk:Investigating_editing_anxiety_…
That rather puts me in an uncomfortable position, is there any point in my getting
involved in this?
WSC
On 7 September 2011 21:55, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote:
I think he's aware of the talk page as he made the very first edit
http://meta.wikimedia.org/w/index.php?title=Research_talk:Investigating_edi…
I'll ping him again if we don't hear from him in the coming days
Thanks
Dario
On Sep 7, 2011, at 1:02 PM, WereSpielChequers wrote:
Hi Dario,
Do we need to tell him there is a talkpage?
WSC
On 7 September 2011 20:29, Dario Taraborelli <dtaraborelli(a)wikimedia.org> wrote:
FYI Ben has updated the project description with further information about the sample and
methods.
Thanks WSC for the initial feedback,
Dario
On Aug 30, 2011, at 11:47 AM, Dario Taraborelli wrote:
A new project has been added to the Research
directory:
http://meta.wikimedia.org/wiki/Research:Investigating_editing_anxiety_in_ne…
The PI – Benjamin Cowan – is an HCI researcher based in Edinburgh and I had a few
conversations with him on this project over the last weeks.
This research sounds very timely and relevant to WMF's work on new user engagement (I
am cc'ing the Summer of Research list), but we need to figure out how to best handle
the recruitment of participants.
One option we discussed would be to (temporarily) integrate a questionnaire into the new
MoodBar feature [1], but I need to discuss this internally with the tech people.
In the meantime your feedback is very welcome. I told Ben we need more information, for
example on the target sample size. He'll be adding more details in the coming days.
Dario
[1]
http://www.mediawiki.org/wiki/MoodBar
_______________________________________________
RCom-l mailing list
RCom-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l
_______________________________________________
RCom-l mailing list
RCom-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l
_______________________________________________
RCom-l mailing list
RCom-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l
_______________________________________________
RCom-l mailing list
RCom-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l
_______________________________________________
RCom-l mailing list
RCom-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l
_______________________________________________
RCom-l mailing list
RCom-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/rcom-l