WSC, thanks for your note, which gives me the opportunity to clarify some gaps in the current RCom review procedure.
There are two separate considerations that apply to this project specifically:
1) the researcher asked us whether the integration of the questionnaire into features WMF is currently developing (such as MoodBar) was feasible from a technical standpoint. I consulted my colleagues in the product team as the question of available resources and technical feasibility needs to be settled on their side independently of the RCom's assessment of the proposal. I could have replied privately about this request but I thought it'd be useful to make my response public to allow other RCom members to review it.
2) the researcher shared with me a list of questions that he is planning to use for this study and specifically asked me not to circulate this document publicly. He's comfortable about sharing this document individually with RCom members as long as it's not posted on a publicly indexed mailing list.
This addresses two separate questions you're asking:
• is there some information that would be inappropriate to post on public fora as part of the review of a research proposal?
• are there any parts of a research proposal whose assessment is of exclusive WMF competence and not of the community or the Research Committee?
The answer to both questions is yes.
There are many cases I can think of in which a public forum is not appropriate for reviewing research proposals. On top of the Wiki Anxiety case (a research protocol that the researcher doesn't want to be publicly circulated until the research is actually run), there are other cases I've come across that fall under the same category. Authors of grant proposals who seek support from WMF in the form of, say, a non-committal letter of endorsement won't be able to publicly share the contents of the proposal while it's being evaluated as this would undermine the funder's decision. We might decide that RCom provides no support to projects that cannot be fully and transparently documented on Meta at the time of the review, but that would certainly leave us with a bunch of undergrad surveys as the only kind of collaboration requests we receive and I am not sure this is the direction we want to take. I think it's legitimate for a researcher to ask that under specific conditions a proposal (or parts thereof) should not be immediately disclosed until it's thoroughly reviewed AND funded. This probably means that we should use tools like OTRS to manage the parts of a request that cannot be discussed publicly for the above reasons. In the specific case of the Wiki Anxiety protocol, I am happy to share the annexes of the proposal with any of you, please let me know off-list if you wish to read it.
The second consideration is that there are multiple cases in which a proposal (or parts thereof) needs to be assessed by WMF independently and prior to RCom's own assessment. These include:
• projects that require the allocation of technical resources or engineer time from WMF;
• projects that require access to private data (in ways that are compliant with our privacy policy) and engineer supervision on this process;
• projects that require a review from our Legal team;
This doesn't mean bypassing RCom but ensuring that the project meets the minimum technical requirements to be implemented (without which it's pointless for the researcher to invest further effort).
I am sorry to hear you felt disconnected from the review process, I think your feedback on this and other projects is extremely helpful, but please let me know if the above distinction answers your concerns.
The question of reviewing some aspects of a project via private channels is definitely an issue that deserves further discussion and I'd like to hear how RCom members feel about it.
Dario