Wouldnt it be better to have redacted the anomaly and
aggregated to two
remaining scores rather then three scores because other the review is
meaningless as it the anomaly in place and therefore create the wasted
effort, where an 8 and 3 occurs if the third score is a 10 then the 3 is
the anomaly and the average should be 9 where the third score is 4 then the
8 is the anomaly and the average is 3.5
On 5 February 2016 at 18:19, Michele Lavazza <michele.lavazza(a)gmail.com>
wrote:
@Andrew pleas take a look at
https://wikimania2016.wikimedia.org/wiki/Programme for "the proportion,
appropriateness or overall relationship of user digest presentations,
critical issues or discussions".
@Marc-André thank you for your remarks, this is just to say that in the
end there was no need to implement the so-called "WMF quota" because less
than 10 WMF staff submission were among the top 40.
@Those who have doubts about the consistency of our implementation of a
third evaluation in those cases where the first two were strongly
divergent: this was a technically arduous thing, since clearly assigning a
third review presupposes having two already. Having a deadline to respect,
we tried to make sure to have the first two reviews for each submission a
few days before the deadline (which proved impossible because of personal
difficulties of some members of our international reviewing team). We would
then add the third reviews when needed. This still was a heavy bulk of
extra work, so we decided not to request the extra review for those
submissions whose average score after the first two reviews was too low to
make it possible for a third review to make the relevant submission
competitive for acceptance. (For example: if the top 50 submissions, with 2
reviews or 3 where needed, already had an average score higher than 7.5, we
didn't assign a third reviewer to a submission whose first two scores were
3 and 8, because, even if the third review were a 10, the average would
still be lower than that 7.5).
I hope I answered a few of the most urgent questions. Thank you all for
expressing you perplexities, criticism, or questions. I realise some
aspects of the double blind peer review process have been suboptimal and
we're trying to make sure both that the conference sessions will be
great nevertheless, and that our experience is useful for the future
Wikimanias.
Thank you very much,
yours,
Michele
On Thu, Feb 4, 2016 at 11:38 PM, Lodewijk <lodewijk(a)effeietsanders.org>
wrote:
Hi all,
quick update about the discussions track: I'm coordinating that together
with a few others, and this will probably be a bit later in time that most
of the other sessions. This is a continuation of the Discussion Room of
2014/2015 <https://wikimania2015.wikimedia.org/wiki/Discussion_Room>,
and focuses on roundtabe discussions of 40-45 minutes each on a specific
topic. That is mostly because there's less preparation required for those
discussions, so we love to have them decided a bit later and have them be
more 'hot and current'. We hope to open improve the descriptions of that
soon though, and open suggestions for that too.
We're currently planning to formally open discussion suggestions on Feb
25 or a bit before. But in the mean time, if you're afraid to forget to
submit it by then, feel free to leave behind ideas here:
https://wikimania2016.wikimedia.org/w/index.php?title=Talk:Discussions&…
(Shani's list of points sounds good in general, although proposals with
less information are also welcome for roundtable discussions)
Best,
Lodewijk
On Thu, Feb 4, 2016 at 9:32 PM, Shani <shani.even(a)gmail.com> wrote:
Following Andrew's comments, here's what
I know --
1 - User Digest is just one session of about 30 minutes. Liaisons pick
the speakers to that. It's supposed to give a review of the thematic
subject - GLAM, EDU, etc.
2 - I'm posting here what I've sent to the Cultural Partners Mailing
List -
*"*I've just updated the GLAM part on the program liaison page on Meta
<https://meta.wikimedia.org/wiki/Wikimania_2016_bids/Esino_Lario/Program/Liaisons#GLAM>,
putting there everything from our joint google doc.
Now that the "critical issues" submission part is over, *it's high
time to submit your suggestions *to all the other aspects of our GLAM
track, if you haven't done so thus far. This includes suggestions for:
** Discussions*
** Workshops / Training*
** Posters*
** Lightning talks*
* *Anything else we might have forgotten*
Some of you have already contacted me privately about *discussions *
and *workshops,* so please feel free to update the relevant part on
meta.
Try to keep it in the same format as suggested below, so it's easier to
follow -
* Title:
* Purpose:
* Target audience:
* Length:
* Max number of people (only if there is a limitation on your part):
* Facilitator(s):
* any other detail that will help others get a sense of the workshop
and what you want to achieve.
Please see an example I posted on behalf of Barbara Fischer -- building
the GLAM KIT library
<https://meta.wikimedia.org/wiki/Wikimania_2016_bids/Esino_Lario/Program/Liaisons#Building_the_GLAM_KIT_library>
.* "*
In other words*, use the liaison page for now,* till the organizing
team has the separate pages ready.
- Follow the format suggest, so it's cohesive and easier to follow.
- Show your support to proposals, the wiki-way. The organizing team
will take that into consideration.
- When the organizing team opens submissions for the remaining parts --
submit!
Hope that helps,
Shani.
On Thu, Feb 4, 2016 at 7:39 PM, Marc A. Pelletier <marc(a)uberbox.org>
wrote:
> On 2016-02-04 3:22 AM, WereSpielChequers wrote:
>
>> Hope Montreal manages something a bit better,
>>
>
> I don't know about "better", nor do I think it quite fair to slam the
> 2016 team either for what was clearly intended to be an attempt to improve
> the process - even if some of the results appear suboptimal in retrospect.
>
> FWIW, the Montreal team is keeping a close eye on the experiments
> being done by the Italian team - no doubt there will be a valuable set of
> lessons learned and we may be able to translate some of the things that
> worked well into improvements to future Wikimanias.
>
> As for the programme selection, we are gunning for a process that
> splits about 30% invited, 40% community CFP, and 30% unconference-style,
> with the selection process for the CFP being very close to past years
> (i.e.: public review on-wiki). We also don't intend to make a distinction
> between submissions by Foundation staff and the other community members,
> though we expect that many presentations that would have been proposals by
> staff will end up being invited directly by the programming committee
> leaving more "slots" available to the CFP.
>
> -- Marc
>
>
>
> _______________________________________________
> Wikimania-l mailing list
> Wikimania-l(a)lists.wikimedia.org
>
https://lists.wikimedia.org/mailman/listinfo/wikimania-l
>
_______________________________________________
Wikimania-l mailing list
Wikimania-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimania-l
_______________________________________________
Wikimania-l mailing list
Wikimania-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimania-l
_______________________________________________
Wikimania-l mailing list
Wikimania-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimania-l
_______________________________________________
Wikimania-l mailing list
Wikimania-l(a)lists.wikimedia.org