Thanks Lodewijk for answering my questions. I don't find your feedback moot
and it's actually quite helpful. From what you're saying, it sounds like
opening up feedback to those who reported data would help to solidify the
content of the report before pushing the announcement publicly. We have 8
more program reports to publish and I'm starting to think of ways we might
include a window for this kind of feedback, but I would need to check with
the rest of the team and our timelines to know what is feasible. We have
been extra busy these last few months.
Also, to make a clarification, we don't assume that all Wiki Loves
Monuments are the same. The metrics we collect are fairly broad and not
exhaustive so that we can first know the collective impact of the program
and then we can dig deeper and learn about the contests in greater detail
afterward. In the coming months, we will be doing one-on-one interviews
with organizers to surface the processes and goals of several photo
contests, and learn what works and what doesn't in different contexts. This
would be the opportunity to explore these assumptions and questions.
Thanks again for your helpful suggestions,
Edward
On Thu, May 7, 2015 at 9:46 PM, Lodewijk <lodewijk(a)effeietsanders.org>
wrote:
Hi Edward,
Thanks for the questions. The Wiki Loves Monuments mailing list would have
made a very logical starting place to ask for initial feedback. But also
sending an email to the people who shared their data with you to work with
in the first place, or people who worked on internal evaluations in these
projects before.
The feeling has been created that right now, the 'damage is done': the
report is published, you have done all you could to make sure that all
community members are as much aware as possible of what you consider the
conclusions. That means that any feedback now, becomes somewhat moot. We
have seen this before with Foundation publications (i.e. statistics on the
chapters), once it is announced to the community at large, feedback often
doesn't get incorporated any more (I hope this time it does!) and even if
it is, the "facts" already found their place into other publications like
the signpost. Asking feedback is most valuable *before* you announce it,
and proactively. You could (even better) consider involving those
stakeholders even earlier in the process, which makes it less of a black
box.
I strongly believe that it would improve the quality of the work you do.
Still some of the basic flaws will remain due to the basic setup of the
evaluation framework (assumptions that all WLM are comparable etc.) but
others could be managed better.
Best,
Lodewijk
On Fri, May 8, 2015 at 3:47 AM, Edward Galvez <egalvez(a)wikimedia.org>
wrote:
Hi Lodewijk,
Thanks for your feedback about the process. It's been very valuable.
I have a few follow up questions below:
> Sure, the team did reach out in the collection phase - after all,
without
> the data such evaluation would be
impossible. But after that, the
> conclusions were drafted and shared with the wide community, rather
than
with the
stakeholders involved to discuss interpretation.
Can you say more about which stakeholders? Do you have ideas how we might
include them in the future, for example, through the Wiki Loves Monuments
mailing list, or were you thinking in some other way?
Either way, all communication seemed to be aimed to announce the
> evaluation, rather than to ask active input on whether the analysis
made
> sense, whether there were misunderstandings,
etc. But maybe you have
had
a
> lot of follow-up discussions with the people you collected data from
on a
1-to-1
level, which would be admirable.
We tried to encourage input and questions through the next steps and in
the
talk page, but it sounds like this might not have
been enough. How do you
think we can do this better next time? Anything specific that stands out
to
you, beyond sharing with stakeholders
beforehand?
Thanks so much,
Edward
Again, I do appreciate the effort, I don't
agree with the approach and
process.
Best,
Lodewijk
_______________________________________________
Wikimedia-l mailing list, guidelines at:
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l(a)lists.wikimedia.org
<
https://meta.wikimedia.org/wiki/Mailing_lists/GuidelinesWikimedia-l@lists.w…
Unsubscribe:
https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
<mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
--
Edward Galvez
Program Evaluation Associate
Wikimedia Foundation
_______________________________________________
Wikimedia-l mailing list, guidelines at:
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l(a)lists.wikimedia.org
Unsubscribe:
https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
<mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
_______________________________________________
Wikimedia-l mailing list, guidelines at:
https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines
Wikimedia-l(a)lists.wikimedia.org
Unsubscribe:
https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
<mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>