preselection actually doesn't happen in all countries, so I feel I have to
break this chorus :)
In the Netherlands we have always had a system where every jury member had
to see every image submitted (more or less). We learned that professional
photographers actually are very well able to handle this kind of job,
especially if they have some kind of editing function too. We learned that
they can process actually thousands of images per hour - so if they are
willing to spend a few hours and are helped by the right equipment (jury
tool!) they can process a /lot/. For reference, the Netherlands had roughly
12k, 13k and 6k submissions in the past years. We asked the jury members to
come up with their top-50/100, added those together and then discussed them
during a real life jury meeting where the final decisions were made.
In other countries, this was undoable, and the list was split up in a way
that each image was initially seen by one jury member (I believe this is
the system used in Germany). They would then select a small number from
their own stack, and together this would get a second round treatment.
Basically, what you see in all countries is that there are multiple rounds.
First there is a round where, one way or another, the big stack is reduced
to a much smaller number. Usually between 100 and 1000, depending on your
starting set (Poland is probably an exception - they needed a round in
between). Then, there us another round to reduce the set further to
somewhere between 10 and 100 - which makes it comprehensible for the human
brain. From this point on, you can start discussing the order the images
have to be in.
Each of these rounds can take multiple forms. They can be a small group of
people (or even one person) making quick decisions yes/no, it can
crowdsourced (community) or it can be the jury itself.
I am myself no fan of the system where a single person makes the decision
in any round. I feel that there should /always/ be at least two people
involved - otherwise the personal taste plays too much a role. This means
twice the amount of work, but I think it is worth it because it means less
very good images falling through the cracks. (if I dismiss one picture,
that should not mean that it doesn't make a chance anywhere).
I think all the models offered above are now supported in the jury tool -
which allows a multi-round system setup, where each round can use a
different selection method ranging from binary single person decision to
crowdsourced rating.
Whatever method or setup you choose, I advise a few things:
1) make sure that you have your 10 nominations for the international round
submitted well in time! The deadline is 31 October (UTC), and this deadline
is NOT FLEXIBLE! Keep some space before that date, to be able to manage
sickness or hit-by-bus-situations. After this deadline, the nominations
will be closed, and the international jury starts their work. We will send
more specific instructions about this when that time approaches.
2) Appoint a non-voting jury moderator that will manage the process and
poke people. This can be a staff member in your case.
3) Set up a temporary mailing list, where each juror and the jury moderator
is subscribed to.
4) describe the jury process publicly, and describe your deliberations
(including reasons why the top-10/20/30 was selected) in a jury report.
5) Finally, make sure that your 10 nominations are submitted before the
deadline. (Yes, this is mentioned twice, because it would be a pity if
after all the work, your photos are not considered for the international
round!)
Best,
Lodewijk
2013/8/26 Andrea Zanni <zanni.andrea84(a)gmail.com>
Wikimedia Italia had a pre-jury who selected 500 pics
for the Jury.
We did not use the judging tool, as it would be better (more reliable,
more formal) to have every picture seen and evaluated from multiple persons
(and not just one).
If the judging tool could be set up like that, it would be *very* useful
also for different workflows.
Aubrey
On Mon, Aug 26, 2013 at 10:50 PM, Tomasz Ganicz <polimerek(a)gmail.com>wrote;wrote:
In case of Poland - we had over 50000 pictures to
select - in fact the
highest number of all local WLM contests ;-) Our jury consisted of 3
wikimedians and 4 external experts. In fact only wikimedians were
engaged in first selection. It was organized spontaneously by
secretary of the jury. He has collected lists of pictures in
alphabetical order and put it on his discussion page of Wikimedia
Polska wiki:
http://pl.wikimedia.org/wiki/U%C5%BCytkownik:Karol007/WLZ
He created general rules of selecting pictures - which was almost
purely technical. As it was too much work for 3 people he asked
several trusted wikimedians to help the jury (including me ;-) ). On
average it resulted in selecting aroun 6% of the pictures - but it was
still a lot (around 3000). In the second round - each member of the
jury was asked to select no more than 100 out of the pre-selected
3000. It resulted in around 500 pictures. From this - each jury member
was asked to select top 12. It resulted in around 60 finalist. The
discussion and final selection of 10 best from 60 finalist was made on
real-life meeting which took several hours of hard discussion ;-)
2013/8/26 Michael Maggs <Michael(a)maggs.name>me>:
I'd be interested to hear from countries that
have experience from
previous years how you pre-filtered the submissions for the
jury. As this
is the first year for the UK, we have no idea how many entries we will get,
but it may well be too many for out three-person jury to handle.
I know that some countries have used special panels for pre-filtering,
and others
have used the Wiki community. What were your experiences, and
what technical means did you use to help with the filtering?
Any insights on how best to handle this would be much appreciated.
Michael
_______________________________________________
Wiki Loves Monuments mailing list
WikiLovesMonuments(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org
--
Tomek "Polimerek" Ganicz
http://pl.wikimedia.org/wiki/User:Polimerek
http://www.ganicz.pl/poli/
http://www.cbmm.lodz.pl/work.php?id=29&title=tomasz-ganicz
_______________________________________________
Wiki Loves Monuments mailing list
WikiLovesMonuments(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org
_______________________________________________
Wiki Loves Monuments mailing list
WikiLovesMonuments(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org