[teampractices] Color schemes (was Re: Hazards of back-estimating backlog story points)

Tue Aug 4 23:36:38 UTC 2015

(sidenote, re: graph colors. If it's configurable in the software used to
generate those graphs, I'd recommend picking colors via
http://colorbrewer2.org/ in the future. It is designed for generating
easily distinguishable color schemes. Clicking the "colourblind safe"
checkbox is also recommended (I'm not sure if it was a problem in the
images below. Just always a good practice. :)  HTH.)

On Tue, Aug 4, 2015 at 4:11 PM, Joel Aufrecht <jaufrecht at wikimedia.org>
wrote:

> *tl;dr:* We assigned a default point value to all unestimated tasks in
> the VE backlog, but exploration revealed that this was wrong.  Moral: the
> more assumptions you make about your data, the more ways you need to
> inspect it for the consequences of bad assumptions.
>
> *Executive summary*
> Lead-time analysis of VE revealed that the VE team puts almost all of its
> effort into very old (2-3 + months) tasks.  By other measures, however, the
> VE team puts a third or more of its effort into "interrupts", unplanned
> work.  So either the interrupts are things that sit dormant and then erupt,
> or something's funny in the numbers.  Further investigation revealed that
> the assumption that all unpointed tasks are five points of effort conflicts
> with real-life observation, in which many unpointed tasks are closed every
> week with relatively trivial effort.  Not necessarily because they are
> actually very low effort, but more because they are record-keeping issues
> (duplicate, overtaken by events, fixed as a side effect, etc) than fresh
> work.
>
> *Details*
> Because VE has around 5000 open tasks and only 800 estimated tasks,
> backlog analysis is impossible without making assumptions about the open
> tasks.  Based on average point sizes for estimated tasks, we assigned a
> value of 5 to all unestimated tasks.  (VE doesn't use 5 in normal task
> estimation, so all 5s are former nulls.)
>
> I generated this lead-time chart, which shows the age of tasks that get
> resolved every week.  Dark blue means the task is less than a few weeks old
> on the day it gets resolved; light blue means the task is months old.  It's
> scaled by value of tasks, not raw count.
>
>
> This suggests that the VE team has been spending 80% of effort recently
> closing very old tasks.  This seems like a recent trend, but since almost
> all data was loaded into Phabricator in late January, near the beginning of
> the chart, tasks look artificially young on this scale for the first few
> months. This next chart shows what fraction of VE team effort goes to
> "interrupt" tasks, as opposed to planned work.  The line is total work
> completed; the black area is interrupt work:
>
> The implication of the two charts together is that the VE team is working
> on a third or more interrupts, but most of those interrupts are months
> old.  Which is an odd thing, since most interrupts are actually a week or
> two old.  So we dug into the task sizes for done tasks by week:
>
>
> The teal color in this chart is 5 point tasks, i.e., tasks that were null
> points and that we assumed are actually a default of 5.  This chart says
> that half of more of the effort each week has been on null-point tasks.  In
> practice, however, the VE team is confident that this has not actually been
> happening.  What is actually happening is that the VE team closes a number
> of null-point tasks each week because they are duplicates, or incidentally
> fixed by some other task, or otherwise more administrivia than substantive
> primary development.  So the assumption that null-point tasks are really 5
> points of work may hold true for the main backlog, but is not true for
> tasks that actually get marked done.
>
> So.  First, I identified a list of 470 resolved stories with zero points,
> and the VE team will retroactively assign points to these, the majority of
> which we expect to be 0 or 1 point.  Second, the VE team will have to start
> making sure to assign points to each task as it gets closed.  Third, we
> will have to reconsider our assumptions about how big the unpointed VE
> backlog is.
>
>
> *Joel Aufrecht*
> Team Practices Group
> Wikimedia Foundation
>
> _______________________________________________
> teampractices mailing list
> teampractices at lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/teampractices
>
>

-- 
Nick Wilson (Quiddity)
Community Liaison, Product
Wikimedia Foundation
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/teampractices/attachments/20150804/b9abc2e6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: VE-histopoints.png
Type: image/png
Size: 21313 bytes
Desc: not available
URL: <https://lists.wikimedia.org/pipermail/teampractices/attachments/20150804/b9abc2e6/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: VE-interrupt.png
Type: image/png
Size: 53026 bytes
Desc: not available
URL: <https://lists.wikimedia.org/pipermail/teampractices/attachments/20150804/b9abc2e6/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: VE-leadtime.png
Type: image/png
Size: 22966 bytes
Desc: not available
URL: <https://lists.wikimedia.org/pipermail/teampractices/attachments/20150804/b9abc2e6/attachment-0005.png>