Hi Pete,

excellent questions. But I suggest we seperate the technical from the more social ones. Because the social ones mostly depend on what exact setup you choose for your jury process and are very country specific (so probably rather something to ask yourself internally). 

I didn't consider myself the usecase scenario yet where the judging would start during the contest - so that is interesting. I automatically assumed that it would only start on Oct 1, which means that you only feed in a single amount of pictures once. If you want to feed it chunks of images over time, I guess a text file input (one every day) works better technically? 

The box where the local jury moderator explains things is a nice idea - could perhaps be an intro/help screen? 

Lodewijk


2013/8/19 Peter Ekman <pdekman@gmail.com>
RE: Sylvain's comments and practical "user experience"or sociological considerations.

A) the tool looks great!
B) Will it be able to handle 1,000 entries per day?  Or a better question - What will it take to make it work well with 1,000 entries per day?
- technically, I'd think one upload (perhaps of just one day's worth of entries each day) would seem to be needed.  If each reviewer needs to upload 30,000 images each time they use it, it could be pretty difficult.
-technically, can it be set so that new images (or perhaps randomly selected images) are judged by different reviewers.  We wouldn't want the same 500 images being at the front of the line that get judged by 10 different people, but the other 29,500 don't get seen at all.
-sociologically, how to deal with folks who go through and select the photos they uploaded and rate poorly other very good photos (gaming)?
-what happens to photos not rated in cases there are too many photos and not enough reviewers?  Are they just left behind?
-is "rate as you go" a possibility, or would you have to wait until the end of the contest?
-presentation, could you include a box where the local designer (who selects the various options) explains how they work, i.e. what are the mechanics for a photo to get to the next round.

Perhaps these aren't the easiest questions, and maybe some can be ignored for the time being, but I'd think the success of the system will depend on questions like this being addressed.

Sincerely,
Pete
User:Smallbones

Message: 4
Date: Mon, 19 Aug 2013 16:06:39 +0200
From: Sylvain Boissel <sylvainboissel@gmail.com>
To: Wiki Loves Monuments Photograph Competition
        <wikilovesmonuments@lists.wikimedia.org>
Subject: Re: [Wiki Loves Monuments] [Jury Tool] Technical Details
Message-ID:
        <CAPVch_tAwmOw9XZ0YKPZpT0XDyPOG2qwC-yx0nsFSfHHQe7ydw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Well, except for the part where my server's PHP version was too old to use
Composer, the installation went smoothly, the steps are very clear.

But looking at the config part and I'd appreciate some explanations about
what the options mean : how do the rounds system work, and the binary
scoring (if two jury members review a file, one saying "good" and the other
"bad", what does it do ?), etc.

For this part :
Commons category
Plain text file

I guess the "plain text file" is a one-shot import ? But what about the
"Commons Category" ? Do we have to import everything at once or can the
jury make the first round of evaluation "on the fly" during the contest ?
In France, we had 27 000 photos to evaluate. In 20 days, it is very short
so if the jury has 50 days (assuming they start sorting out the photos the
first day of the contest) its better for them.

And I want to thank you for creating the tool, it looks awesome !

Best regards,
Sylvain.

_______________________________________________
Wiki Loves Monuments mailing list
WikiLovesMonuments@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikilovesmonuments
http://www.wikilovesmonuments.org