Thanks Lodewijk for answering my questions. I don't find your feedback moot and it's actually quite helpful. From what you're saying, it sounds like opening up feedback to those who reported data would help to solidify the content of the report before pushing the announcement publicly. We have 8 more program reports to publish and I'm starting to think of ways we might include a window for this kind of feedback, but I would need to check with the rest of the team and our timelines to know what is feasible. We have been extra busy these last few months.
Also, to make a clarification, we don't assume that all Wiki Loves Monuments are the same. The metrics we collect are fairly broad and not exhaustive so that we can first know the collective impact of the program and then we can dig deeper and learn about the contests in greater detail afterward. In the coming months, we will be doing one-on-one interviews with organizers to surface the processes and goals of several photo contests, and learn what works and what doesn't in different contexts. This would be the opportunity to explore these assumptions and questions.
Thanks again for your helpful suggestions, Edward
On Thu, May 7, 2015 at 9:46 PM, Lodewijk lodewijk@effeietsanders.org wrote:
Hi Edward,
Thanks for the questions. The Wiki Loves Monuments mailing list would have made a very logical starting place to ask for initial feedback. But also sending an email to the people who shared their data with you to work with in the first place, or people who worked on internal evaluations in these projects before.
The feeling has been created that right now, the 'damage is done': the report is published, you have done all you could to make sure that all community members are as much aware as possible of what you consider the conclusions. That means that any feedback now, becomes somewhat moot. We have seen this before with Foundation publications (i.e. statistics on the chapters), once it is announced to the community at large, feedback often doesn't get incorporated any more (I hope this time it does!) and even if it is, the "facts" already found their place into other publications like the signpost. Asking feedback is most valuable *before* you announce it, and proactively. You could (even better) consider involving those stakeholders even earlier in the process, which makes it less of a black box.
I strongly believe that it would improve the quality of the work you do. Still some of the basic flaws will remain due to the basic setup of the evaluation framework (assumptions that all WLM are comparable etc.) but others could be managed better.
Best, Lodewijk
On Fri, May 8, 2015 at 3:47 AM, Edward Galvez egalvez@wikimedia.org wrote:
Hi Lodewijk,
Thanks for your feedback about the process. It's been very valuable.
I have a few follow up questions below:
Sure, the team did reach out in the collection phase - after all,
without
the data such evaluation would be impossible. But after that, the conclusions were drafted and shared with the wide community, rather
than
with the stakeholders involved to discuss interpretation.
Can you say more about which stakeholders? Do you have ideas how we might include them in the future, for example, through the Wiki Loves Monuments mailing list, or were you thinking in some other way?
Either way, all communication seemed to be aimed to announce the
evaluation, rather than to ask active input on whether the analysis
made
sense, whether there were misunderstandings, etc. But maybe you have
had
a
lot of follow-up discussions with the people you collected data from
on a
1-to-1 level, which would be admirable.
We tried to encourage input and questions through the next steps and in
the
talk page, but it sounds like this might not have been enough. How do you think we can do this better next time? Anything specific that stands out
to
you, beyond sharing with stakeholders beforehand?
Thanks so much, Edward
Again, I do appreciate the effort, I don't agree with the approach and process.
Best, Lodewijk _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org <
https://meta.wikimedia.org/wiki/Mailing_lists/GuidelinesWikimedia-l@lists.wi...
Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
-- Edward Galvez Program Evaluation Associate Wikimedia Foundation _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe
Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe