Hi all, Thank you for your feedback. I take your comments as a sign of genuine care and I'm happy to engage and learn with you how we can do better. (Note: I'm responding to all lists, though some of the feedback has been sent only to wikimedia-l.)

* Galder, Gereon, Xavier, Gnangarra, and Andy: thank you for your feedback.

* Andy, I'll respond to your comment first. We do not require the work to be published under a free license for us to consider it for the award. However, if the work is shortlisted, we reach out to the authors, tell them that it's shortlisted, and it can be considered for the award if the work is at least made publicly available. At that point, we also encourage the authors to publish under a free license and share with them a few ways they may be able to (even if the work is published somewhere already with restrictions). The issue of licenses is on top of our mind and we actively look for ways to push for more Wikimedia research work to be published under free licenses.

* I am going to share with you some of my thoughts, and a possible improvement we can make in the process.

** Let's try to keep things simple to be able to improve things together. This is not a case of "WMF did x". The idea of the award was created in the Research team, and both last year and this year, we've been grateful to have the support of researchers outside of WMF for it. (Aaron Shaw (Northwestern University), and Benjamin Mako Hill (U. of Washington)). I take full responsibility for the execution of the award and I can take your feedback and see where we can improve the process. :)

** In order to be able to improve the process, I should share more details about how we do the search for the publications first. We have multiple sources for searching for research published in a given year: 1. The nomination process we shared on this thread. 2. Research publications shared in WikiResearch twitter account. 3. External research search engines and repositories for different fields: we use scholar.google.com, dblp.org and more.

To give you a sense of the distribution of scholarly publications we identified last year from each of the above sources: 11 nominations and 170+ research publications through the twitter account and external searches. The award chairs (2 people; this year it is going to be Mako and I) reviewed all identified publications. We discussed every publication at varying depth depending on the result of our initial reviews.

** Knowing the process, there are at least a few ways I think the process must be improved. I'm sure now that you see more you can critique even more. :) I proactively share with you some of them here: ::* I need to have an easychair account to nominate. That can/must change (but to what? we want these nominations to be private, and we need a way to be able to process them efficiently because we're only 2 people. We are considering openreview.net for the future years because they're open source; but they still have other limitations. For this year, easychair it is.). ::* We need more people on the committee: both for workload sharing, and also including more perspectives. (This is /a lot/ to ask of researchers. I'm grateful that Mako and Aaron have supported us in the past.) ::* We need other non-English sources to source community research. (WikiResearch is primarily in English and about research published in English.) ::* The shared language of reviewers is assumed to be English. If we are going to at scale consider other languages, then we need a way that this group of people can converse on academic topics with one another without having to share a language.

** I also understand the reality of the resources available to me and our team. I understand the importance of working on multiple fronts with regards to the research community (Wikimedia Research Funds, Wiki Workshop, global research competitions, research showcases, monthly office hours, talks and presentations, formal collaborations, and more). I believe in the importance of motivation (and we have seen a very good momentum around the award idea from last year's run). We need to do many things, with very limited resources; Our values and ideals are important and we have to attempt to hold them all as we make decisions. In practice, sometimes we can't meet all the ambitions we have. We need to make trade-offs. What is important is to be aware, to listen, to try to improve, and to be honest.

I will leave you with the above and I commit to talk with Mako to consider ways to open up the process for more languages to be included (in 2021 or in 2022+; I can't promise changes for the 2021 process.). One of us will write back here with what we decide to do.

Thanks, Leila

On Sat, Jan 8, 2022 at 5:42 AM Andy Mabbett andy@pigsonthewing.org.uk wrote:

On Fri, 7 Jan 2022 at 19:48, Leila Zia lzia@wikimedia.org wrote:

...

=Eligibility criteria=

...

• The publication must be available in English.

I echo others' concerns about this.

I'm equally concerned that, while WMF regard being in English as essential for one of their awards, they do not regard the use of an open licence as a requirement.

-- Andy Mabbett @pigsonthewing https://pigsonthewing.org.uk _______________________________________________ Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org