That would indeed be better, but it would require a huge amount of extra work for PC chairs and a much more sophisticated software than EasyChair...

On Fri, Feb 5, 2016 at 11:32 AM, Gnangarra <gnangarra@gmail.com> wrote:
Wouldnt it be better to have redacted the anomaly and aggregated to two remaining scores rather then three scores because other the review is meaningless as it the anomaly in place and therefore create the wasted effort, where an 8 and 3 occurs if the third score is a 10 then the 3 is the anomaly and the average should be 9 where the third score is 4 then the 8 is the anomaly and the average is 3.5 





On 5 February 2016 at 18:19, Michele Lavazza <michele.lavazza@gmail.com> wrote:
@Andrew pleas take a look at https://wikimania2016.wikimedia.org/wiki/Programme for "the proportion, appropriateness or overall relationship of user digest presentations, critical issues or discussions".

@Marc-André thank you for your remarks, this is just to say that in the end there was no need to implement the so-called "WMF quota" because less than 10 WMF staff submission were among the top 40.

@Those who have doubts about the consistency of our implementation of a third evaluation in those cases where the first two were strongly divergent: this was a technically arduous thing, since clearly assigning a third review presupposes having two already. Having a deadline to respect, we tried to make sure to have the first two reviews for each submission a few days before the deadline (which proved impossible because of personal difficulties of some members of our international reviewing team). We would then add the third reviews when needed. This still was a heavy bulk of extra work, so we decided not to request the extra review for those submissions whose average score after the first two reviews was too low to make it possible for a third review to make the relevant submission competitive for acceptance. (For example: if the top 50 submissions, with 2 reviews or 3 where needed, already had an average score higher than 7.5, we didn't assign a third reviewer to a submission whose first two scores were 3 and 8, because, even if the third review were a 10, the average would still be lower than that 7.5).

I hope I answered a few of the most urgent questions. Thank you all for expressing you perplexities, criticism, or questions. I realise some aspects of the double blind peer review process have been suboptimal and we're trying to make sure both that the conference sessions will be great nevertheless, and that our experience is useful for the future Wikimanias.

Thank you very much,
yours,

Michele

On Thu, Feb 4, 2016 at 11:38 PM, Lodewijk <lodewijk@effeietsanders.org> wrote:
Hi all,

quick update about the discussions track: I'm coordinating that together with a few others, and this will probably be a bit later in time that most of the other sessions. This is a continuation of the Discussion Room of 2014/2015, and focuses on roundtabe discussions of 40-45 minutes each on a specific topic. That is mostly because there's less preparation required for those discussions, so we love to have them decided a bit later and have them be more 'hot and current'. We hope to open improve the descriptions of that soon though, and open suggestions for that too. 

We're currently planning to formally open discussion suggestions on Feb 25 or a bit before. But in the mean time, if you're afraid to forget to submit it by then, feel free to leave behind ideas here: https://wikimania2016.wikimedia.org/w/index.php?title=Talk:Discussions&action=edit&redlink=1 (Shani's list of points sounds good in general, although proposals with less information are also welcome for roundtable discussions)


Best,
Lodewijk


On Thu, Feb 4, 2016 at 9:32 PM, Shani <shani.even@gmail.com> wrote:
Following Andrew's comments, here's what I know -- 

1 - User Digest is just one session of about 30 minutes. Liaisons pick the speakers to that. It's supposed to give a review of the thematic subject - GLAM, EDU, etc. 

2 - I'm posting here what I've sent to the Cultural Partners Mailing List -

​"I've just updated the GLAM part on the program liaison page on Meta, putting there everything from our joint google doc. 

Now that the "critical issues" submission part is over, it's high time to submit your suggestions to all the other aspects of our GLAM track, if you haven't done so thus far. This includes suggestions for: 
* Discussions
* Workshops / Training
* Posters
* Lightning talks
* Anything else we might have forgotten

Some of you have already contacted me privately about discussions and workshops, so please feel free to update the relevant part on meta. 
Try to keep it in the same format as suggested below, so it's easier to follow - 
* Title:
* Purpose:
* Target audience:
* Length:
* Max number of people (only if there is a limitation on your part):
* Facilitator(s):
* any other detail that will help others get a sense of the workshop and what you want to achieve.
Please see an example I posted on behalf of Barbara Fischer -- building the GLAM KIT library. "

In other words, use the liaison page for now, till the organizing team has the separate pages ready. 
- Follow the format suggest, so it's cohesive and easier to follow. 
- Show your support to proposals, the wiki-way. The organizing team will take that into consideration.
- When the organizing team opens submissions for the remaining parts -- submit! 

Hope that helps, 
Shani. 

On Thu, Feb 4, 2016 at 7:39 PM, Marc A. Pelletier <marc@uberbox.org> wrote:
On 2016-02-04 3:22 AM, WereSpielChequers wrote:
Hope Montreal manages something a bit better,

 I don't know about "better", nor do I think it quite fair to slam the 2016 team either for what was clearly intended to be an attempt to improve the process - even if some of the results appear suboptimal in retrospect.

FWIW, the Montreal team is keeping a close eye on the experiments being done by the Italian team - no doubt there will be a valuable set of lessons learned and we may be able to translate some of the things that worked well into improvements to future Wikimanias.

As for the programme selection, we are gunning for a process that splits about 30% invited, 40% community CFP, and 30% unconference-style, with the selection process for the CFP being very close to past years (i.e.: public review on-wiki).  We also don't intend to make a distinction between submissions by Foundation staff and the other community members, though we expect that many presentations that would have been proposals by staff will end up being invited directly by the programming committee leaving more "slots" available to the CFP.

-- Marc



_______________________________________________
Wikimania-l mailing list
Wikimania-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimania-l


_______________________________________________
Wikimania-l mailing list
Wikimania-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimania-l



_______________________________________________
Wikimania-l mailing list
Wikimania-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimania-l



_______________________________________________
Wikimania-l mailing list
Wikimania-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimania-l




--
GN.
President Wikimedia Australia
WMAU: http://www.wikimedia.org.au/wiki/User:Gnangarra
Photo Gallery: http://gnangarra.redbubble.com


_______________________________________________
Wikimania-l mailing list
Wikimania-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimania-l