On 4 February 2016 at 14:50, Lydia Pintscher Lydia.Pintscher@wikimedia.de wrote:
I got a very similar comment assuming I knew little about Wikidata. As its product manager...
Anyone who has been burdened with doing these sorts of reviews will feel some sympathy for those giving the feedback. It is easy to upset a lot of people if the process is not well thought out. Where there are marking discrepancies, the workflow should mean it goes to another independent reviewer and there is a meeting (like 2 minutes in a Hangout discussion) where there is final agreement on the rating/mark *and* the feedback that should be given.
Even without discrepancies in marks, feedback needs to be positive and supportive, this is all volunteers giving their time after all, not postgrads getting critical essay feedback. That means the workflow also needs to include regular checks and team meetings to talk about how to best ensure marks and feedback remains consistent, even when the experience and viewpoints of the reviewers may be highly varied.
Lots of lessons to be summarized for later, and probably a need to consider whether now is a good time put up your hands and formally admit to problems in consistency. Asking submitters to give their feedback and suggestions on-wiki, even if is too late to change any decision, was a good response.
Fae