We all suck at estimating, regardless of how experienced we are. This is a fact that you should accept. Most of us are either ignorant to this or in denial. There are many ways we try to hide our inadequacies, mostly revolving around mathematical transformations of the form:
E’ = mE + c
I.E. make an arbitrary estimate (E), multiply it by some amount (m) and add a bit (c).
I’m not denying that there is some sense to this, you can spend considerable time and effort refining your favourite m value by tracking your velocity, having regular retrospectives and reflective analyses etc.
This method alone however is ignorant to significant mental “quirks” which affect the way you think and reason.
The effects I am talking about are:
- The “halo” effect
- Framing effects
- Attribute Substitution
- Base-rate neglect
The “halo” effect
The “halo” effect is defined as “the influence of a global evaluation on evaluations of individual attributes”. What this means in the realm of software development estimation is that you are likely to estimate the individual parts of a project with a bias towards how you feel about the overall project.
If you’ve formed an opinion that overall the project will be easy, all your estimates for the component parts are likely to be lower than if you viewed the project as difficult (known as the “devil” effect).
- Ignore prejudices
- Judge tasks independantly
- Don’t “do the easy ones first”
Framing effects refers to the way our mind perceives data differently depending on how it is presented. For example, food which is “90% fat free” sounds much better than food described as “10% fat”.
When estimating tasks, we are very likely to bias our judgement based on how the requirements are presented. For example, requirements which are positively worded/presented and which sound easy/appealing are much more likely to receive lower estimates.
- Has the way the requirements been worded affected your interpretation?
- Are your judgements of a specific problem being clouded by its surroundings?
Overconfidence and Substitution
Little weight is given to the quality or quantity of evidence when we form a subjective confidence in our opinions. Instead, our confidence depends largely on the quality of the story we can tell ourselves about the situation. What this means is that we are very likely to be confident in an estimate if we have convinced ourselves that we know what we’re talking about. This may sound obvious, however the devil lies in the detail. Do we really know what we are talking about? Our brains do not like doubt and uncertainty, we are much happier answering questions positively rather than negatively. When estimating a task, we are very likely to jump to a conclusion (underestimate) if the task is familiar to us. How many times have you said “oh yeah that’s simple, it will take X hours” without _really_ thinking it through? This is known as the mere-exposure effect.
This is where another problem creeps in, attribute substitution. When our brains are faced with a complex question, our sub-conscious often substitutes the problem for a more familiar, easier problem. This often happens without us realising. This leads to misunderstandings of the problem domain and therefore inaccurate estimates.
- Ask yourself why you are confident
- Are you biasing because of familiarity?
- Have you really understood the problem?
Base Rate Neglect
Base rate neglect or base rate bias is an error which occurs when assessing the probability of something and failing to take into account the prior probability. I use the term here partly in the strictest sense (as defined by Wikipedia above) and partly in a more general sense.
When we estimate tasks, we often fail to account for the “surrounding” or “prior” cost of the task, such as the complicated merge that will be required after the change, or the reliance on the third party delivering on time, the API documentation being adequate etc.
- Consider all the implications
- What assumptions have you made? – Are they sensible? Really?
- Refuse to estimate unknowns
Anchoring is an effect that causes you to bias your estimate based on estimates you’ve already seen or produced. If two developers are discussing an estimate and the first says “10 days”, the second developer is more likely to produce a number closer to “10 days” than if they hadn’t spoken previously. This is one of the main benefits of using planning poker. By avoiding the influence of others until you have produced your estimates, you are much more likely to get a broader range of estimate values.
You may think broader means worse, but this is not necessarily the case. If one dev thinks a task is “1 day” but another thinks its “10 days” – you’ve identified a problem. Either you have a huge skill disparity, or there has been a fundamental misunderstanding by one or both parties!
- Try to view each estimate in isolation, dont let previous numbers skew future ones
- Don’t confer with other estimators until you all have your own value, then justify why they are different
When producing estimates be aware of biases and make an conscious effort to spot when you might be making them. Being aware that you are likely to be biased is the first step in producing more accurate estimates, actually counteracting the biases in practice can be much harder 😉
- Estimate alone at first
- Get 2nd (or more) opinions on estimates, but be careful not to cause framing or anchoring biases
- Make a conscious effort to not misinterpret a requirement based on its wording
- Be sure you’ve not jumped to conclusions because of familiarity of the problem
- Review the estimates you produced last. Are they biased based on the estimates your produced first?
- Estimate each task in isolation. Dont let your opinions of other tasks or the whole project effect individual parts
If you are interested in learning more about the psychology of decision making and biases and how you can make personal improvements (not just in development estimation) then I highly recommend you get your hands on a copy of the following: