I'll suggest that we won't effect much of anything this year with this conversation, so let's leave it for after the contest is over. I'll ask everybody to keep a journal on their jury procedures this year so we can improve them next year. A couple of issues with Plotonides comments first: Nobody is gaming the system by self-nominating - that was built in as a compromise in the beginning. I'd take it out next year. The "positive version" of Kaldari's tool would output the file names of favorite pix to a special sandbox page for further processing. The only problems that I have with Nuno's tool is that I don't know how to output data, or how to scale it up for general use. That's because I'm not a coder, not because of any fault in the software. Anonymity in the screening process is probably necessary. Without some anonymity things might get very tricky and emotional. I'll readily admit that I've made some mistakes in screening - if you go through 1,000s of pix fairly quickly how could there not be mistakes? and I think most people will understand this. Except when somebody puts forth his special "baby" to the public ... I'll admit to 2 types of mistakes - Type I (the most important) removing pix that shouldn't have been removed at this stage. I'll admit to a 1% error rate here, and if you look at the numbers - the implications are truly horrendous. Type II errors - leaving in pix that shouldn't have been left in - I'm not really very concerned about . I'll admit to a 10% error rate here. If somebody were to complain about his baby being removed when it shouldn't have been, and then compares it to some Type II errors - if I were publicly identified, all I could do is give the seemingly lame excuse "everybody makes mistakes" But of course everybody does. I think with "crowd sourcing" or just crowd voting, we have the chance to make this contest something very, very special. Everything that's put in front of the final jury may not be perfect, but at the very least, it could be "crowd pleasing." A voting system for masses of photos - I've come to believe - is a very hard technical project. Perhaps, if we can define exactly what we want, we could get the Foundation to work on this for us before next year. I'll suggest a 3 stage general process. 1. Review for obvious technical problems (probably 2 strikes and you're out, rather than 1) 2. Crowd voting to get down to a limited number of crowd pleasing photos, and 3. A final round of judging by highly qualified judges. After all photography is an art, and experts must play a role. Hope this helps (for next year) Pete Message: 8 Date: Sat, 15 Sep 2012 00:25:14 +0200 From: Platonides <platonides(a)gmail.com> To: Wiki Loves Monuments Photograph Competition <wikilovesmonuments(a)lists.wikimedia.org> Subject: Re: [Wiki Loves Monuments] jury tools Message-ID: <5053AECA.5090809(a)gmail.com> Content-Type: text/plain; charset=ISO-8859-1 On 13/09/12 05:05, Peter Ekman wrote: One drawback of this is accountability. There's no trace of who voted a picture through the app (categories are changed by the bot). Another problem is that authors are adding themselves the categories nominating the images they just uploaded: https://commons.wikimedia.org/w/index.php?title=File:Comac_Church.jpg&d… https://commons.wikimedia.org/w/index.php?title=File%3ADramatic_Obelisk.jpg… https://commons.wikimedia.org/w/index.php?title=File:Protestant_Children%27… https://commons.wikimedia.org/w/index.php?title=File:Bellows_House_Missoula… (these are all different users tricking the system, random images provided by the tool) You mean something like Nuno tool?