It seems to me that you can't possibly give a jury a 1,000 photos and expect them to come up with anything reasonable. That type of system would also drive away quality jurors - the best jurors simply wouldn't have time for all that. And if we're talking about 10,000 photos, it just gets worse. There has to be some sort of pre-screening, whether we like it or not.
A couple of suggestions for pre-screening:
1. Let the photographer decide which of his photos is best - say 1 for the entire contest or 1 for each day he/she uploads.
2. Have a contest each day, with a each photographer who uploaded that day nominating a single photo, and letting the community vote (I'd say +1 for each photo you like) then after a few days a selected screener from the community selects 2 or 3 photos from the group that has the highest score. After 30 days, you'd have 60-90 photos that the jury can deal with, each photog would have had the chance to nominate his best photos (multiple times), the community would have their say, and the screeners would not have to deal with 1,000s of photos.
Pete Ekman
User:Smallbones
Date: Fri, 22 Jun 2012 00:16:01 +0200
From: Lodewijk <lodewijk@effeietsanders.org>
To: Wiki Loves Monuments Photograph Competition
<wikilovesmonuments@lists.wikimedia.org>
Subject: Re: [Wiki Loves Monuments] jury process - easy and neat?
Message-ID:
<CACf6BesGfNFEVuXoQapbahbTmCiyMh0FdAOgZpGBungyENpZzA@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Of course if you only use this tool as a means to get the best 1% of the
pictures... it doesn't have to be as precise. Just have the final selection
in a second round, with zero scores - then vote again for those or have a
discussion on the mailing list.
Personally, I think every country can decide its own process - so lets make
the tool somewhat dynamic, but not shoot down major options. Last year
Russia had a public voting, and it seems they were happy with it. I'm no
big fan of it, but who am I to stop it? :)
Lodewijk
2012/6/22 Platonides <platonides@gmail.com>
> On 21/06/12 18:29, aude wrote:
> > We have a technical volunteer "intern" helping us in DC and this is
> > something he wants to work on.
> >
> > I'm not exactly sure how the process worked last year, but we're
> > thinking either integrating something into the WordPress site so that
> > people can browse photos (pulled from Commons, like InstantCommons) and
> > rate them. A public voting phase could help narrow the selection for
> > the jury, which could also make use of the tool.
> >
> > Thoughts? suggestions? brilliant ideas?
>
> Each instance used its own system. Nuno made a web application for
> voting, from which you could start. There is also a toolserver tool
> which facilitates downloading all the files from a category.
> Last year we provided zips with all the images which our jury
> downloaded. Then they chose those they deemed the best, which went to a
> second round.
> If you're joining the jury in one room (with multiple computers), it may
> be simpler if they are provided directly a copy of the images in an
> external drive. Remember that not all your jury members will be tech-savvy.
>
> If I were designing such system, I'd make it an interface where the jury
> would go giving a mark from 1 to 10 to each photo. But not actually
> restricted to that. So after 200 photos, it could give a 12 if needed
> (instead of recalculating all previous votes), or even provide marks
> with decimal points.
> The interface itself would be just a (zoomable) gallery of the photos
> that he didn't review yet (plus auxiliar pages, to view the best ranked
> by you, change a vote, etc.).
>
> What would be interesting is that they could be using it from the first
> day, so instead of reviewing all files after the competition closes,
> they could keep up with the upload rate.
> As far as a 10 given on Sep 1st is the same as one given on 30th Sep
> (which is easy by things like moving the bar higher up to a 14), it'd be
> equivalent.
> You then fetch the N most ranked from each member to next round, so
> different scales aren't a problem.
>
>
> As for a public voting, I don't think it would work. You would need each
> person to review a significant number of files, otherwise the noise
> given by each different reviewer (a 10 by me could be an 8 by you). And
> you won't be getting volunteers to review thousands of photos. Only the
> jury will do that, because they agreed to. (Obviously, anyone is able to
> volunteer to be jury. We were discussing on wlm-iberconf ml giving a
> jury for another country and getting one).
>
> What I had thought as a possibility for involving the public was to
> allow it to choose a number of photos that pass to the next round (just
> as each jury does), thus ensuring they get attention. But that won't the
> jury task of having to view all of them.
>
> Regards
>
> _______________________________________________
> Wiki Loves Monuments mailing list
> WikiLovesMonuments@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
> http://www.wikilovesmonuments.eu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/wikilovesmonuments/attachments/20120622/b8116f22/attachment.html>
------------------------------
_______________________________________________
WikiLovesMonuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
End of WikiLovesMonuments Digest, Vol 18, Issue 35
**************************************************