Hi Jan,

A few years ago, the Grants (now Community Engagement) dept. at WMF put together a set of metrics designed to measure the impact of different kinds of grant-funded and/or "programmatic" events and activities within the movement. Many of these metrics were built into the Wikimetrics tool, developed by the Analytics team to help individuals and groups measure and report the impact of their activities themselves.

The metrics we developed aren't perfect, but they were based on research and intended to be accessible and provide coverage for a wide variety of activities related to the Wikimedia movement. So they represent a framework in that respect. To learn more about them, you might ask members of the Learning & Evaluation team. 

- J

On Sat, Feb 4, 2017 at 7:34 AM, Jan Dittrich <jan.dittrich@wikimedia.de> wrote:

Hello Analytics,

It seems to easily happen that a well meant metrics is proposed, but that this is actually not sure to be correlated with something value or (at least with my skills) super  hard to measure technically [1].

So I wanted to create some easy-to-graspmaterials, maybe to put them on a poster or in a video… whatever.

Sadly I found a lot of warnings about "vanity metrics" online (recommending $ made instead) but no framework, research or case studies on organizational impact and understanding of metrics and how colleagues can be helped to be empowered to do a basic evaluation of metrics.

So you have any framework, research or case studies that could help us to teach and empower to evaluate metrics and their use?

Here is what I found useful so far:

- https://en.wikipedia.org/wiki/DIKW_pyramid and similar frameworks like https://randomblatherdotcom.files.wordpress.com/2014/04/bmust7rcuaa6ueo-jpg-large.jpeg

- For the need of organizational sensemaking: Weick, Sensemaking in Organizations

- And, not directly related but providing some inspiration: Robust methods and visualizations by Tuckey, decision making under uncertainty by Gingerenzer.

Jan

[1] In which case I prefer no metric at all, and investing into qualitative research instead, till the problem space is better understood, but if metrics are already there it often stays…
--

Jan Dittrich

UX Design/ User Research

Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0)30 219 158 26-0
http://wikimedia.de

Imagine a world, in which every single human being can freely share in the sum of all knowledge. That‘s our commitment.

Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics

--

Jonathan T. Morgan

Senior Design Researcher

Wikimedia Foundation

User:Jmorgan (WMF)

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics