[teampractices] interesting read on the problems with KPIs

[Joel Aufrecht]

TPG has been discussing this in the context of how we, as a team that does
process assistance and consulting, can quantify our value.  A few
highlights from our discussions:

"if you can't measure it, you can't manage it." is not true.  "if you can't
measure it, you can't prove it" captures the kernel of truth, but begs the
question of what needs to be proved by whom to whom.

The "Compstat" effect, in which anyone who has a metric-based incentive is
thus incented to influence the metric, not necessarily the underlying
reality the metric is intended to measure.  I.e., hire and fire police by
numbers and crime statistics will become unreliable; fire teachers based on
student test performance and test-taking will become less reliable.

It's also important to differentiate between self-accountability and
external accountability.  People measure their own performance to get a
truer understanding of their own work and avoid various biases; this is
subject to different problems and trade-offs than measuring work for
someone else's judgment.




*--Joel Aufrecht*
Team Practices Group
Wikimedia Foundation

On Mon, Dec 14, 2015 at 2:00 PM, Dan Duvall <dduvall at wikimedia.org> wrote:

> Thanks for sharing, indeed! The tone is charged but, if I'm being honest,
> I share many of the same sentiments and in no more eloquent terms. I've
> always considered them a pretty cruel imposition for people working at an
> operational level, people that have already pledged to give nearly half—and
> in many cases, it's much closer to all—of their waking life to the
> Wikimedia movement.
>
> On Mon, Dec 14, 2015 at 12:43 PM, Kevin Smith <ksmith at wikimedia.org>
> wrote:
>>
>> Hopefully we (as an org) are aiming for KPI's that measure actual
>> end-user value and impact ("outcomes") as opposed to measuring our own
>> internal amounts of work ("outputs"), or worse (e.g. "how much time an
>> employee spends in the bathroom").
>>
>
> Therein lays the crux of the problem for me: Our KPIs have been employed
> completely backwards. We didn't start by defining KPIs for organizational
> impact (likely because we lack organizational initiatives and strategy to
> begin with) or for intermediary goals (there are no intermediaries of
> 'undefined'). Instead, our ED mandated that teams use KPIs to measure their
> own individual conceptions of success which, in the absence of any similar
> accountabilities measures at the C-level, amounted to saying "prove your
> worth to us." Leaving teams to formulate their own models of success
> without any shared consensus about the bigger picture has also rendered our
> KPIs biased and unverified—in other words, they are pseudoscientific.
>
> Every second spent on fulfilling KPIs in an organization of such strategic
> and operational disparity is a huge waste of donor money in my opinion.
>
> And just because I saw "if you can't measure it, you can't manage it" in
> the article, I have to follow up with this article. :)
>
> http://www.druckerinstitute.com/2013/07/measurement-myopia/
>
> --
> Dan Duvall
> Automation Engineer
> Wikimedia Foundation <http://wikimediafoundation.org>
>
> _______________________________________________
> teampractices mailing list
> teampractices at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/teampractices
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/teampractices/attachments/20151214/2126bdd2/attachment.html>