Estimation meets Cynefin

Question: Which provides a better estimate: Planning Poker or an 'expert'?

Many teams would answer that question by saying Planning Poker is always better than an expert’s estimates. We’ve been told it’s the case so many times, that few question it.

Truth is, an expert estimate is sometimes more appropriate than an estimate created using Planning Poker. Sometimes it’s not. What is important is to use the most appropriate method for the situation. Here’s an introduction to what we’re calling ‘context-based estimation’:

1. Evaluate the situation

First you need to understand the context and environment that you are in. Dave Snowden’s Cynefin framework has four domains to describe the situation (if you’re new to this idea, see our blog post on Cynefin).

Cynefin framework (Dave Snowden, released under CC BY 3.0)

2. Understand the type of uncertainty

Most situations that fall in the simple or complicated domains can be described as having epistemic uncertainty: something you don’t know but is, at least in theory, knowable. I don’t know how to build a car engine but it is possible to estimate this given enough information. The cause and effect are often known before the event.

Aleatory uncertainty, on the other hand, is found in our left-hand domains: something you not only don’t know; it is unknowable.  However much information I have, even if they play as well next season, it is impossible for me to know whether Leicester City will win the Premier League again next year. This type of uncertainty is full of accidental causes, is often dependent on contingent events, and is sometimes down to plain old chance.

3. Choose my estimation/forecasting approach accordingly

Now that we know what type of environment we are in and what type of uncertainty we face, we can pick an appropriate estimation/forecasting approach. For example, when faced with a situation in the simple domain, an expert guess will suffice. If required to estimate a scenario in the complicated domain, a loosely mathematic forecast can be made – because a bell curve distribution is likely. When we are looking aleatory uncertainty in the eye, getting multiple data points and aggregating the data is probably going to serve us better.

So what about Planning Poker? I’ve often said that the biggest benefit of Planning Poker is the conversations that need to occur in order for sizing to take place, but if we judge this crowd-like estimation according to James Surowiecki’s 4 conditions (see recent post on The Wisdom of Crowds), it doesn’t fare well: although the first round of Planning Poker gives independent thought, that’s gone by round two; a team’s diversity of opinion, if ever present, will dwindle quickly as a small team works closely together with shared experiences and goals;  few companies truly have decentralisation; although Planning Poker works towards consensus, it isn’t really making much of aggregation; that leaves just decentralisation, which Surowiecki acknowledges is usually missing in most companies. So, as a method, Planning Poker is far from perfect. Getting a wider number of data points is often a good route when you’re in the complex domain: this could take the form of crowd-sourcing opinion or running thousands of scenarios, using ranges for as many variables as you have, in Monte Carlo simulations.

If you find yourself in the chaotic domain, you need to forget estimating and do something NOW! If your house is on fire, you don’t start calculating how much water you need to put the fire out; you just act.

Is that all?

Certainly not. We’ve not talked about revisiting estimates. We’ve not talked about whether you should estimate at all. We’ve not covered how to reduce uncertainty. In fact, we’ve only scratched the surface on this topic, but at least we’ve made a start in acknowledging that the method needs to suit the situation.