[RCom-l] Public vs private review of support requests

Fri Sep 9 00:35:54 UTC 2011

WSC, thanks for your note, which gives me the opportunity to clarify some gaps in the current RCom review procedure.

There are two separate considerations that apply to this project specifically:

1) the researcher asked us whether the integration of the questionnaire into features WMF is currently developing (such as MoodBar) was feasible from a technical standpoint. I consulted my colleagues in the product team as the question of available resources and technical feasibility needs to be settled on their side independently of the RCom's assessment of the proposal. I could have replied privately about this request but I thought it'd be useful to make my response public  to allow other RCom members to review it.

2) the researcher shared with me a list of questions that he is planning to use for this study and specifically asked me not to circulate this document publicly. He's comfortable about sharing this document individually with RCom members as long as it's not posted on a publicly indexed mailing list.

This addresses two separate questions you're asking:

• is there some information that would be inappropriate to post on public fora as part of the review of a research proposal?
• are there any parts of a research proposal whose assessment is of exclusive WMF competence and not of the community or the Research Committee?

The answer to both questions is yes.

There are many cases I can think of in which a public forum is not appropriate for reviewing research proposals. On top of the Wiki Anxiety case (a research protocol that the researcher doesn't want to be publicly circulated until the research is actually run), there are other cases I've come across that fall under the same category. Authors of grant proposals who seek support from WMF in the form of, say, a non-committal letter of endorsement won't be able to publicly share the contents of the proposal while it's being evaluated as this would undermine the funder's decision. We might decide that RCom provides no support to projects that cannot be fully and transparently documented on Meta at the time of the review, but that would certainly leave us with a bunch of undergrad surveys as the only kind of collaboration requests we receive and I am not sure this is the direction we want to take. I think it's legitimate for a researcher to ask that under specific conditions a proposal (or parts thereof) should not be immediately disclosed until it's thoroughly reviewed AND funded.  This probably means that we should use tools like OTRS to manage the parts of a request that cannot be discussed publicly for the above reasons. In the specific case of the Wiki Anxiety protocol, I am happy to share the annexes of the proposal with any of you, please let me know off-list if you wish to read it.

The second consideration is that there are multiple cases in which a proposal (or parts thereof) needs to be assessed by WMF independently and prior to RCom's own assessment. These include:
• projects that require the allocation of technical resources or engineer time from WMF;
• projects that require access to private data (in ways that are compliant with our privacy policy) and engineer supervision on this process;
• projects that require a review from our Legal team;
This doesn't mean bypassing RCom but ensuring that the project meets the minimum technical requirements to be implemented (without which it's pointless for the researcher to invest further effort).

I am sorry to hear you felt disconnected from the review process, I think your feedback on this and other projects is extremely helpful, but please let me know if the above distinction answers your concerns. 
The question of reviewing some aspects of a project via private channels is definitely an issue that deserves further discussion and I'd like to hear how RCom members feel about it.

Dario

On Sep 8, 2011, at 1:24 PM, WereSpielChequers wrote:

> Hi Dario,
> 
> I asked if I could see the questionnaire. None of my talk threads have been responded to, but you've added a new thread that includes the sentence "I don't know whether you would be using only a subset of the questions listed in the Wiki Anxiety/Usability Inventory (a document that was shared privately with WMF), but the questionnaire in its current state (46+22 questions)".  So I now know how long it is, but it has been shared privately and so presumably I can't have it.
> 
> Which leaves me feeling somewhat disconnected from the process.
> 
> I don't know whether the researcher has been asked if the survey can be shared with WMF volunteers on RCOM, or why the secrecy.
> 
> Currently our emails are logged on the Internet for all to see, so I'm not sure how we handle the situation where someone is willing to engage with just the WMF. I suppose moving to OTRS resolves that, though it does leave me wondering how we maintain transparency to the community. I suppose they key thing will be to set some boundaries as to what we are and are not prepared to do in Private.
> 
> 
> WSC
> 
> 
> 
> On 8 September 2011 20:21, Dario Taraborelli <dtaraborelli at wikimedia.org> wrote:
> I don't understand, that was my feedback to the researcher based on discussions I had with other product people at WMF (regarding technical aspects of the proposed integration with MoodBar, which are of direct WMF competence)
> 
> Dario
> 
> On Sep 8, 2011, at 12:16 PM, WereSpielChequers wrote:
> 
>> Apparently he's answered my query, but only to the WMF http://meta.wikimedia.org/wiki/Research_talk:Investigating_editing_anxiety_in_new_users
>> 
>> That rather puts me in an uncomfortable position, is there any point in my getting involved in this?
>> 
>> WSC
>> 
>> On 7 September 2011 21:55, Dario Taraborelli <dtaraborelli at wikimedia.org> wrote:
>> I think he's aware of the talk page as he made the very first edit
>> 
>> http://meta.wikimedia.org/w/index.php?title=Research_talk:Investigating_editing_anxiety_in_new_users&oldid=2866546
>> 
>> I'll ping him again if we don't hear from him in the coming days
>> 
>> Thanks
>> Dario
>> 
>> On Sep 7, 2011, at 1:02 PM, WereSpielChequers wrote:
>> 
>>> Hi Dario,
>>> 
>>> Do we need to tell him there is a talkpage?
>>> 
>>> WSC
>>> 
>>> On 7 September 2011 20:29, Dario Taraborelli <dtaraborelli at wikimedia.org> wrote:
>>> FYI Ben has updated the project description with further information about the sample and methods.
>>> Thanks WSC for the initial feedback,
>>> 
>>> Dario
>>> 
>>> On Aug 30, 2011, at 11:47 AM, Dario Taraborelli wrote:
>>> 
>>> > A new project has been added to the Research directory:  http://meta.wikimedia.org/wiki/Research:Investigating_editing_anxiety_in_new_users
>>> >
>>> > The PI – Benjamin Cowan – is an HCI researcher based in Edinburgh and I had a few conversations with him on this project over the last weeks.
>>> > This research sounds very timely and relevant to WMF's work on new user engagement (I am cc'ing the Summer of Research list), but we need to figure out how to best handle the recruitment of participants.
>>> > One option we discussed would be to (temporarily) integrate a questionnaire into the new MoodBar feature [1], but I need to discuss this internally with the tech people.
>>> >
>>> > In the meantime your feedback is very welcome. I told Ben we need more information, for example on the target sample size. He'll be adding more details in the coming days.
>>> >
>>> > Dario
>>> >
>>> > [1] http://www.mediawiki.org/wiki/MoodBar
>>> 
>>> 
>>> _______________________________________________
>>> RCom-l mailing list
>>> RCom-l at lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/rcom-l
>>> 
>>> _______________________________________________
>>> RCom-l mailing list
>>> RCom-l at lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/rcom-l
>> 
>> 
>> _______________________________________________
>> RCom-l mailing list
>> RCom-l at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/rcom-l
>> 
>> 
>> _______________________________________________
>> RCom-l mailing list
>> RCom-l at lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/rcom-l
> 
> 
> _______________________________________________
> RCom-l mailing list
> RCom-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/rcom-l
> 
> 
> _______________________________________________
> RCom-l mailing list
> RCom-l at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/rcom-l

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wikimedia.org/pipermail/rcom-l/attachments/20110908/0328e76b/attachment.htm 