[teampractices] interesting read on the problems with KPIs

[Dan Duvall]

Thanks for sharing these insights, Joel. I'm always eager to see what's
going on in those amazing TPG brains. :)

On Mon, Dec 14, 2015 at 2:12 PM, Joel Aufrecht <jaufrecht at wikimedia.org>
wrote:

> The "Compstat" effect, in which anyone who has a metric-based incentive is
> thus incented to influence the metric, not necessarily the underlying
> reality the metric is intended to measure.  I.e., hire and fire police by
> numbers and crime statistics will become unreliable; fire teachers based on
> student test performance and test-taking will become less reliable.
>

The "CompStat effect" is especially interesting to me but I also think it's
missing a relevant variable which is the authority/subordinate relationship
that may exist between the two parties involved. Just as an exercise, I
wonder how it would read if the subject was: 1) shifted to the people
involved and away from the metrics, and; 2) framed within an explicit
employer/employee relationship where it's assumed that continually
underperforming employee's are fired.

It might read:
An employee who's job performance is measured by an employer-mandated
metric may attempt to influence the metric so as not to lose his/her
livelihood. If we hire and fire police by numbers, either crime statistics
will become unreliable or we will no longer have police; if we fire
teachers based on student test performance, either test results will become
unreliable or we will no longer have teachers.

In other words, when the relationship is hierarchal and the metric is
mandated, I think the compelling force is mainly one of threat not of
incentive. An employee's livelihood is at stake in cases of
underperformance whereas the employer's livelihood is not, at least not to
a comparable degree.

This isn't to say that KPIs or other metrics have this dynamic baked
in—self assessment, as you mentioned, is a completely different thing. It's
only to say that, like any other tool, metrics can easily be abused by
people in authority to posture, manipulate, or evade questions of their own
incompetence. Metrics obfuscate these ulterior motives well because they're
often framed as precise and objective.

-- 
Dan Duvall
Automation Engineer
Wikimedia Foundation <http://wikimediafoundation.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/teampractices/attachments/20151214/b54d7a16/attachment.html>