preselection actually doesn't happen in all countries, so I feel I have to break this chorus :)

In the Netherlands we have always had a system where every jury member had to see every image submitted (more or less). We learned that professional photographers actually are very well able to handle this kind of job, especially if they have some kind of editing function too. We learned that they can process actually thousands of images per hour - so if they are willing to spend a few hours and are helped by the right equipment (jury tool!) they can process a /lot/. For reference, the Netherlands had roughly 12k, 13k and 6k submissions in the past years. We asked the jury members to come up with their top-50/100, added those together and then discussed them during a real life jury meeting where the final decisions were made.

In other countries, this was undoable, and the list was split up in a way that each image was initially seen by one jury member (I believe this is the system used in Germany). They would then select a small number from their own stack, and together this would get a second round treatment. 

Basically, what you see in all countries is that there are multiple rounds. First there is a round where, one way or another, the big stack is reduced to a much smaller number. Usually between 100 and 1000, depending on your starting set (Poland is probably an exception - they needed a round in between). Then, there us another round to reduce the set further to somewhere between 10 and 100 - which makes it comprehensible for the human brain. From this point on, you can start discussing the order the images have to be in. 

Each of these rounds can take multiple forms. They can be a small group of people (or even one person) making quick decisions yes/no, it can crowdsourced (community) or it can be the jury itself. 

I am myself no fan of the system where a single person makes the decision in any round. I feel that there should /always/ be at least two people involved - otherwise the personal taste plays too much a role. This means twice the amount of work, but I think it is worth it because it means less very good images falling through the cracks. (if I dismiss one picture, that should not mean that it doesn't make a chance anywhere). 

I think all the models offered above are now supported in the jury tool - which allows a multi-round system setup, where each round can use a different selection method ranging from binary single person decision to crowdsourced rating. 

Whatever method or setup you choose, I advise a few things:
1) make sure that you have your 10 nominations for the international round submitted well in time! The deadline is 31 October (UTC), and this deadline is NOT FLEXIBLE! Keep some space before that date, to be able to manage sickness or hit-by-bus-situations. After this deadline, the nominations will be closed, and the international jury starts their work. We will send more specific instructions about this when that time approaches.
2) Appoint a non-voting jury moderator that will manage the process and poke people. This can be a staff member in your case. 
3) Set up a temporary mailing list, where each juror and the jury moderator is subscribed to. 
4) describe the jury process publicly, and describe your deliberations (including reasons why the top-10/20/30 was selected) in a jury report. 
5) Finally, make sure that your 10 nominations are submitted before the deadline. (Yes, this is mentioned twice, because it would be a pity if after all the work, your photos are not considered for the international round!)

Best,
Lodewijk

2013/8/26 Andrea Zanni <zanni.andrea84@gmail.com>
Wikimedia Italia had a pre-jury who selected 500 pics for the Jury.
We did not use the judging tool, as it would be better (more reliable, more formal) to have every picture seen and evaluated from multiple persons (and not just one).
If the judging tool could be set up like that, it would be *very* useful also for different workflows.

Aubrey   


On Mon, Aug 26, 2013 at 10:50 PM, Tomasz Ganicz <polimerek@gmail.com> wrote:
In case of Poland - we had over 50000 pictures to select - in fact the
highest number of all local WLM contests ;-) Our jury consisted of 3
wikimedians and 4 external experts. In fact only wikimedians were
engaged in first selection. It was organized spontaneously by
secretary of the jury. He has collected lists of pictures in
alphabetical order and put it on his discussion page of Wikimedia
Polska wiki:

http://pl.wikimedia.org/wiki/U%C5%BCytkownik:Karol007/WLZ

He created general rules of selecting pictures - which was almost
purely technical. As it was too much work for 3 people he asked
several trusted wikimedians to help the jury (including me ;-) ). On
average it resulted in selecting aroun 6% of the pictures - but it was
still a lot (around 3000). In the second round - each member of the
jury was asked to select no more than 100 out of the pre-selected
3000. It resulted in around 500 pictures. From this - each jury member
was asked to select top 12. It resulted in around 60 finalist. The
discussion and final selection of 10 best from 60 finalist was made on
real-life meeting which took several hours of hard discussion ;-)



2013/8/26 Michael Maggs <Michael@maggs.name>:
> I'd be interested to hear from countries that have experience from previous years how you pre-filtered the submissions for the jury.   As this is the first year for the UK, we have no idea how many entries we will get, but it may well be too many for out three-person jury to handle.
>
> I know that some countries have used special panels for pre-filtering, and others have used the Wiki community.    What were your experiences, and what technical means did you use to help with the filtering?
>
> Any insights on how best to handle this would be much appreciated.
>
> Michael
>
>
>
>
> _______________________________________________
> Wiki Loves Monuments mailing list
> WikiLovesMonuments@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
> http://www.wikilovesmonuments.org



--
Tomek "Polimerek" Ganicz
http://pl.wikimedia.org/wiki/User:Polimerek
http://www.ganicz.pl/poli/
http://www.cbmm.lodz.pl/work.php?id=29&title=tomasz-ganicz


_______________________________________________
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org