In Estonia, I think we have always published the names of jury members (or in some cases, their wiki names) in all of the article and image competitions we've had over the last 10+ years. At least any exceptions don't come to my mind. So I don't see a problem with the expectation that jury members should be clearly brought out.
It becomes even more important on the international level. Like with Wiki Science Competition we have even published the results on how have jury members voted. Like: https://commons.wikimedia.org/wiki/Commons:Wiki_Science_Competition_2019/Winners/People_in_Science/round_2 Not that it needs to be that specific, but still... With the local version of the science photo competition, it has even been common that all the jury physically comes together to review the files and discuss them. It really helps to use the competence of the jury (I'm "bit" critical to relying only on casting votes, as this clearly works against the images that are actually more special). Occasionally it has happened that one of the jury members can't participate, but I haven't taken the name out for that and he/she has always had the possibility to provide feedback via email. And with the physical gathering is also obvious, that there can't be any numerical values to be published later except to say that "jury reached a consensus" or to bring out what the jury members thought about some images.What has happened a few times (can remember 3) over the years is that the results have been so weak or one-sided, that I didn't assemble a jury [with smaller competitions that may take place at the same time the articles/photos are being collected] or I not used it (as there are so clear winners that it would a total waste of time to do so). That is: setting up a fancy jury only makes sense if there would be results worthy of that jury. It there would only be a few not-so-good files, then the jury members may not want to return the next year. And when it is the first time to organize something it may not be clear on what may be the result. As privacy issues can easily be avoided, then the only possible edge case that comes to my mind is the possibility that the results may end up being too weak to justify some decent pre-announced jury... but with WLM I'd guess there should anyway be like at least 500 images to choose from whatever participating country, so that should not be a problem.
Correction: when looking at the list of ca 70 competitions we've had then there wasn't jury listed here: https://www.miljonpluss.ut.ee/24h/ (even thou there still was something in the press release)Regards
Ivo Kruusamägi_______________________________________________Kontakt attolippip (<attolippip@gmail.com>) kirjutas kuupäeval L, 3. oktoober 2020 kell 20:25:_______________________________________________Last year we did publish an overview of how the evaluation/selection process was organised (in Ukrainian): http://wlm.org.ua/top123_2019/ while introducing top 123 pictures of the last round (as it was published before announcing our top 10). I think it also helped to engage people on social media, as we asked our followers to submit their guesses about top 10 from these 123 in the comments :)On Sat, Oct 3, 2020, 19:56 effe iets anders <effeietsanders@gmail.com> wrote:Hi Mykola,I would encourage WLX Jury Tool to follow suit and provide a similar option to give easy openness to its users. There is no obligation to use either or any tool. This overview is just a convenient way to make it easier for organizers to be transparent.Lodewijk_______________________________________________On Sat, Oct 3, 2020 at 9:49 AM Mykola Kozlenko <mycola-k@ukr.net> wrote:Hi Lodewijk,As we intend to use WLX Jury Tool and not Montage in Ukraine, what is expected from us?We always publish the list of our jury members here: http://wlm.org.ua/juri/ , you can see all our jury members between 2012 and 2019. The 2020 list is still being finalised as we are still waiting for final confirmations, but we will also publish it as we did in previous years.
On the other side, as we have 47.5K images in Ukraine, we will have to organise multiple rounds, including checking the images before submitting them to the jury (as no jury is able to review that many images) and possibly a final call between jury members to distribute places among the top photos. We are happy to explain our selection process but we may not have the same details as for the teams using Montage.Best regardsMykola (NickK)WLM Ukraine team--- Оригінальне повідомлення ---
Від кого: "effe iets anders" <effeietsanders@gmail.com>
Дата: 1 жовтня 2020, 00:49:56Hi all,After getting feedback both on- and offlist, we arrived at the following: the Montage developers will make a page available to each jury coordinator, that they can copy and paste to a wiki page to share their process settings easily. That way it's easy to be transparent. At some point in the future, we will likely make this public by default.I do express the expectation that each national coordinator will publish their jury members (either real life name or username). It may well be that we ask you to submit this list of jurors when you submit your nominees (although we will not publish it on your behalf, that is your responsibility).Warmly,LodewijkAll great suggestions.In the past the WLM international team has also maintained a database of jurors to be backup. We had very little requests in recent years though, probably because so many people know organizers form other countries. If you're stuck, I would suggest to either ask someone you already know, or to ask this list. Most likely you have a response within 48h. But don't wait until the last minute, if you can avoid it. Jurors typically like a week to get things done (if the country isn't huge).LodewijkThank you Yaroslav, I will send you a separate email now!Having the WLE team setting up a pool of jurors like that was so helpful Anton. Yes, I could find the names and email other WLM organisers, but this system took that extra work away from me - which was hugely appreciated, especially in our first year!The international WLE team this year has tried to create a database of jurors from different countries for local contests. We had a Google form for these purposes (https://forms.gle/Pj61adjgYiE6Jn687) & asked local teams whether they needed help with jury.We haven't estimated the results yet, since the local selection process is still ongoing, but it seems to have worked well.Best RegardsAnton ProtsiukProject Manager at Wikimedia Ukraine_______________________________________________YaroslavIf WLM Ireland is in September I can help as a juror. If it is in October this could be more difficult because I am also in the jury for Russia, and there is typically a lot of work there.Generally, asking around (may be also on Commons) typically helps. In the past I have been on jury for a few different countries. Somebody would just approach me and ask.If I can make it, I agree._______________________________________________Hi, Rebecca!I have organized multiple WLM/WLE editions in Latvia. We have used Montage in most of them.After experiencing problems with jurors dropping out or being known at the last moment, we created new Wikimedia accounts just for the jury and sent the passwords to them.Montage allows some editing of jurors after the round has started but results can be unexpected.Mārtiņš_______________________________________________Hi Lodewijk,As a slight aside to this, and perhaps something that might help ease some of the worries around judging processes in other countries, would it be an idea to have a pool of international WLM Wikimedia jurors that could help judge other countries? Here in Ireland we have done this with Malta, exchanging jurors over the past few years, and for our first WLE this year we had Axel from Sweden be on our jury. You get the benefit of a fellow Wikimedian who understands the whole process, and who could bring some different expertise or perspective to a country's WLM. Having someone who is completely unfamiliar with your local built heritage means they can assess the images with a different take than someone who knows them very well.After 6 years, we have found it harder to recruit a jury from our pool of active Wikimedians and relevant expert judges from the arts and architecture sectors in Ireland. I know I would really appreciate it if we could "borrow" a juror from another Wikimedia group (in the past we have had some UK help with this too with jurors). We have suffered from jurors dropping out of the process at the last minute or after judging has begun (which results in having to restart rounds in Montage), generally it has been those who are not Wikimedians who perhaps did not fully understand the commitment when they agreed. Not only is it frustrating, it's very stressful. It may be less of an issue this year, given that the deadline for submitting to the international jury won't be at the end of October.Hope that helps clarify some of the issues some of the smaller countries can face over the years of WLM!Thanks,RebeccaHi Alexander,As for the status of Montage requests: I suggest that you start a separate thread on that, and would like to leave this to the maintainers to respond to.As for publishing the settings: I was imagining some kind of log-style publication, not a near write-up. This won't be pretty, but it will allow people to figure out how it worked out in practice. If we follow a logical naming convention, people should be able to puzzle it together. Ideally, the national organizers also publish their process on the website, but this log would be a way to verify that. But I accept your note that we may need to add a context explaining that more process may happen before/after this tool is used.Lodewijk_______________________________________________Dear Lodewijk,_______________________________________________We also intend to publish by default the settings of the montage jury tool, and the number of photos in each round that the national competitions have used. We're debating whether there should be an opt-out for this year.This is unrealistic, because jury process may involve several campaigns that are later merged together (in Russia, we do it all the time in order to meet your submission deadline). Moreover, some of the photos can be accepted for the next round within Montage but excluded later on if we find that they do not depict cultural heritage. The end result is that any number you take from Montage will not match the number of photos that we publish (e.g., as a short-list). This will only lead to confusion and won't be of any use for anyone.
Since you mentioned Montage, let me also ask when two important pull requests, which were done by one of our team members, are going to be merged into the code:
https://github.com/hatnote/montage/pull/169
https://github.com/hatnote/montage/pull/175
These are really, really important fixes. Without them I would have a problem creating new Montage campaigns in October.
Sincerely,
AlexanderOn 9/2/2020 11:49 PM, effe iets anders wrote:Hi all,over the past years, we have had various requests to encourage national organizers to be transparent in their judging processes and who sits on their jury. Most of the national organizers are currently transparent about this already. In the past weeks/month, more conversation around this has continued with some concerns (valid or not) on certain jury processes.In this light, the international team intends to institute a new expectation for national organizers, namely to publish the members of their jury (be it their username or real life name) at some point. We have not figured out the practical details yet, but I can imagine that while we encourage publication on the website, we would ask national organizers to add a list of jury members to their submission to the international jury - which we then will publish as well.We also intend to publish by default the settings of the montage jury tool, and the number of photos in each round that the national competitions have used. We're debating whether there should be an opt-out for this year.We will of course apply at least the same level of transparency to the international jury.Before we make this decision, I would like to ask for feedback on this, and whether there are edge cases we should consider where such transparency would be harmful. I'll take 1 week to gather some feedback on this, and then we'll make a final decision. You can respond to this on this mailing list, or privately to me.Warmly,Lodewijk_______________________________________________ Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org--_______________________________________________
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org
--_______________________________________________
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org_______________________________________________ Wiki Loves Monuments mailing list WikiLovesMonuments@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments http://www.wikilovesmonuments.org
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org