We’ve been doing it all wrong!

I can hear myself saying it now: “If you count up the number of work items in each column, and keep a record of each item’s start and end date, then you can create a CFD [Cumulative Flow Diagram], control chart and histogram”. Apparently I was wrong. Apparently nearly all of us are doing it wrong.

That’s according to Dan Vacanti’s new book, Actionable Agile Metrics for Predictability, published in March 2015. Although it told me that some / much of what I have been doing is wrong or inadvisable, it is a great read and I would recommend it to anyone using Kanban metrics. In fact, I’d say it’s an essential read.

Here are a few of what I consider the headlines:

You cannot use Little’s Law to predict the future. Neither can you use a CFD.

“Little’s Law is a relationship of averages. … even if you could use Little’s Law as a forecasting tool (which you cannot), you would not want to”, says Vacanti. Although he concedes that you can use delivery rate and cycle time for a “quick gut check on project status”, he believes that you need to use Monte Carlo Simulators for projections. They are, he says, “the future of forecasting in knowledge work”.

You shouldn’t show your backlog on the CFD.

Vacanti’s first rule of CFDs is that the top line on the diagram always represents arrivals to the process (and the bottom line is always the departures). He welcomes you to store the backlog data, but if it’s in the chart, then that isn’t a CFD.

Don’t just count the items in a column to create a CFD

Counting the items in each column is not sufficient; he wants you to record the date that each item enters each column. If an item moves backwards, then the date can be removed and your data recalculated (although he states that, normally, the discovery of defects does not warrant a ticket moving backwards). This helps you filter out bad data at a later date.

A cancelled item is still counted as a ‘done’ work item

If an item is no longer required, then it is moved to ‘Done’ as if it was completed, and a note is placed against the CFD to acknowledge what has happened.

[I’m still struggling with this a bit. Although I see that we benefit from tracking discarded work, it means that we have to be a bit smarter in our calculations so that we don’t artificially reduce our average cycle times and delivery rates]

Control charts are all wrong!

We should be using scatterplots instead. And what we call control charts aren’t even proper control charts. Scatterplots are better because we replace mean and standard deviations with percentiles to show us where 50%, 85% and 95% of work items have fallen so far (you can choose whichever values suit you). These don’t fall foul of being heavily influenced by outliers, nor do they rely on our work having a normal distribution (which it never does).

Don’t expedite

We all know that expediting items causes cycle times to increase in other items. Vacanti delivers a very eloquent explanation about this, using FedEx to illustrate his points.

There are lots of other gems in the book, such as why we should commit to work, how we should communicate expectations for completing work to customers, how to ‘condition’ items so that they flow through your system, the violations of Little’s Law that invalidate it, how to spot flow debt, and much more.

Dan Vacanti’s Actionable Agile Metrics for Predictability is available from leanpub.com from $20.