[teampractices] interesting read on the problems with KPIs

[Antoine Musso]

Le 14/12/2015 21:43, Kevin Smith a écrit :
> Thanks for sharing that, Anne. I had seen it as well. I was hoping for a
> good critique, but at best it seemed to be a rant about not misusing
> KPI's. I would love to read an article covering the same ground, but
> with more concrete examples of bad KPI's.
<snip>

Hello,

I have some examples of a bad KPI.  This is going to be long, should
probably be a blog post but I have more confidence in mailman archives
over me maintaining a blog for the next century.


Let me give you the end user experience before and after some bad KPI
got introduced. Will then explain the transition and how the metrics
were badly taken in account shadowing the actual most important and
unmeasurable metric: end user satisfaction.


Lets take a small call centre.

A customer call enquiring about an issue it is facing. The respondent is
a technician, he ask a few questions to troubleshoot and find the root
cause. Having appropriate administrative access, the technician fix the
problem on the fly and ask for confirmation of resolution.  Time to fix:
15 minutes. Must be a great experience on both side.


Fast forward a few years, the company has filled an IPO went through a
merging-acquisition with a big corporation and customer service is
handled by an offshore call centre.

What is the experience for the customer?  It calls in, on the first ring
tone some one reply with a foreign accent:

 Welcome to Acme Inc call center.  For an issue related to your Product
press 1. <wait> For an issue with your Account press 2. <wait> For other
issues press 3.
<ait>

 Hello this is John Doe from Acme In call centre. How can I help you today?

 <explain problem>

Thank you Mr. Your ticket number is #130912091409 and a technical
representative from Acme Inc. will call you back.

15 minutes has elapsed.  You have a number and are not even sure it is
the right one.

Customer now has to wait.

After a random time ranging from half an hour to a couple days, you get
called back by a level one technician who barely understand the
technical matter at end.  So you ask for escalation, rinse repeat until
you get the person who can need (if you ever reach him).


What happened?  The original call centre had complains from customers,
it was barely reachable and slow to answer.  So metrics have been set to
better understand the problem, the first report was:

 Call durations:  15 minutes
 Wait time: 3 minutes
 Customers hanging up before talking to someone: 50%
 Time to resolution: 1 hour (not measured)

To reduce the wait time and reply to all customers, a front call centre
has been added and that quickly enhanced all metrics.  The line now
looks like:

  customer --> offshore centre  --> technicians --> customer

But the metrics were still not good enough.  Technicians took to long to
call back customers.  So the most experienced one are isolated in a back
office and have no interactions with customers while the junior are left
in a front office and have their administrative rights reduced so they
can focus on calling back.

 customer --> offshore centre -> front tech  <--> back tech
        <--------------------------/

The tech were not pleasant with the customers, and the offshore centre
had trouble verifying whether an issue was solved. So all issues had to
be channelled through the offshore centre. Make sense to better track
the life of an issue.

At the same time, some back tech were working mostly on very advanced
troubleshooting mostly dealing with other department.  Thus the whole
tech got reorganized in three levels and you end up with:

 customer -> offshore centre ->  tech L1 -> tech L2 -> tech L3

etc..

I told you it was going to be long.  The whole reorganization process
above sounds a complete bureaucratic nightmare and it is hard to see how
it helps fixing the customers issues faster.   But it is backed up by
metrics that shows how the call durations have dropped, wait time is
insignificant and no customer hangs up anymore. So KPI is a success.


What is the mistake there?  The call centre primary activity is to deal
with customer issues by actually fixing them.  If you knew that, after a
few call attempts if someone respond to you the problem get fixed, it is
probably a better experience than being answered a tracking number and
having to fight to move it forward.


By selecting a KPI based on a metric that is easy to measure but
disconnected from the problem to solve, Acme Inc. actually ended up with
a worth customer service.

Luckily, sales went with a new product: advanced support, which would
let you reach out directly to the tech L2 for an extra fee.


The bullshit in KPI is picking random metrics and focusing solely on
improving them. I am sure in all case it is going to lead to a failure
and dramatic side effects.



-- 
Antoine "hashar" Musso