Hi all, Thank you for your feedback. I take your comments as a sign of
genuine care and I'm happy to engage and learn with you how we can do
better. (Note: I'm responding to all lists, though some of the
feedback has been sent only to wikimedia-l.)

* Galder, Gereon, Xavier, Gnangarra, and Andy: thank you for your feedback.

* Andy, I'll respond to your comment first. We do not require the work
to be published under a free license for us to consider it for the
award. However, if the work is shortlisted, we reach out to the
authors, tell them that it's shortlisted, and it can be considered for
the award if the work is at least made publicly available. At that
point, we also encourage the authors to publish under a free license
and share with them a few ways they may be able to (even if the work
is published somewhere already with restrictions). The issue of
licenses is on top of our mind and we actively look for ways to push
for more Wikimedia research work to be published under free licenses.

* I am going to share with you some of my thoughts, and a possible
improvement we can make in the process.

** Let's try to keep things simple to be able to improve things
together. This is not a case of "WMF did x". The idea of the award was
created in the Research team, and both last year and this year, we've
been grateful to have the support of researchers outside of WMF for
it. (Aaron Shaw (Northwestern University), and Benjamin Mako Hill (U.
of Washington)). I take full responsibility for the execution of the
award and I can take your feedback and see where we can improve the
process. :)

** In order to be able to improve the process, I should share more
details about how we do the search for the publications first. We have
multiple sources for searching for research published in a given year:
1. The nomination process we shared on this thread.
2. Research publications shared in WikiResearch twitter account.
3. External research search engines and repositories for different
fields: we use scholar.google.com, dblp.org and more.

To give you a sense of the distribution of scholarly publications we
identified last year from each of the above sources: 11 nominations
and 170+ research publications through the twitter account and
external searches. The award chairs (2 people; this year it is going
to be Mako and I) reviewed all identified publications. We discussed
every publication at varying depth depending on the result of our
initial reviews.

** Knowing the process, there are at least a few ways I think the
process must be improved. I'm sure now that you see more you can
critique even more. :) I proactively share with you some of them here:
::* I need to have an easychair account to nominate. That can/must
change (but to what? we want these nominations to be private, and we
need a way to be able to process them efficiently because we're only 2
people. We are considering openreview.net for the future years because
they're open source; but they still have other limitations. For this
year, easychair it is.).
::* We need more people on the committee: both for workload sharing,
and also including more perspectives. (This is /a lot/ to ask of
researchers. I'm grateful that Mako and Aaron have supported us in the
past.)
::* We need other non-English sources to source community research.
(WikiResearch is primarily in English and about research published in
English.)
::* The shared language of reviewers is assumed to be English. If we
are going to at scale consider other languages, then we need a way
that this group of people can converse on academic topics with one
another without having to share a language.

** I also understand the reality of the resources available to me and
our team. I understand the importance of working on multiple fronts
with regards to the research community (Wikimedia Research Funds, Wiki
Workshop, global research competitions, research showcases, monthly
office hours, talks and presentations, formal collaborations, and
more). I believe in the importance of motivation (and we have seen a
very good momentum around the award idea from last year's run). We
need to do many things, with very limited resources; Our values and
ideals are important and we have to attempt to hold them all as we
make decisions. In practice, sometimes we can't meet all the ambitions
we have. We need to make trade-offs. What is important is to be aware,
to listen, to try to improve, and to be honest.

I will leave you with the above and I commit to talk with Mako to
consider ways to open up the process for more languages to be included
(in 2021 or in 2022+; I can't promise changes for the 2021 process.).
One of us will write back here with what we decide to do.

Thanks,
Leila

On Sat, Jan 8, 2022 at 5:42 AM Andy Mabbett <andy@pigsonthewing.org.uk> wrote:
>
> On Fri, 7 Jan 2022 at 19:48, Leila Zia <lzia@wikimedia.org> wrote:
>
> > =Eligibility criteria=
>
> > * The publication must be available in English.
>
> I echo others' concerns about this.
>
> I'm equally concerned that, while WMF regard being in English as
> essential for one of their awards, they do not regard the use of an
> open licence as a requirement.
>
> --
> Andy Mabbett
> @pigsonthewing
> https://pigsonthewing.org.uk
> _______________________________________________
> Wiki-research-l mailing list -- wiki-research-l@lists.wikimedia.org
> To unsubscribe send an email to wiki-research-l-leave@lists.wikimedia.org
_______________________________________________
Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l
Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/OOQOYTEXTIUEFZNU636I3UMQLFBK7A7H/
To unsubscribe send an email to wikimedia-l-leave@lists.wikimedia.org