Short Overview of Estimation
Back in 2008, Bob Payne and I were working with a team learning to practice Agile Software Development. They were doing quite well at a lot of things, but their sprint velocity bounced up and down like a yo-yo. The sawtooth velocity chart indicated, since they were clearly delivering value at a pretty steady pace, that they were not very good at estimating stories. Bob suggested to me that they might do as well, with less effort, by counting the stories instead of estimating them.
We did a bit on number crunching on the data we had, and it seemed to support that hypothesis. We put out a call on some mailing lists seeking other datasets to examine.
Bob Payne and I have been considering the effectiveness of story estimates for planning via Yesterday’s Weather. We’ve looked at some historical data from a team we’ve worked with, comparing the power of story points and a simple count of the story cards. The results looked interesting to us, so we’d like to look at a wider set of data.
If you’d be willing to send us data from your team(s), we need historical data for a series of iterations of a team, containing
– the number of story points completed for the iteration
– the number of stories completed for the iteration
– the range of story points (low to high) for the stories of the iteration.
If this last value isn’t easily available, then the range of story points generally used by the team will do.
Thanks for your help.
The responses were encouraging. We submitted a session proposal to the “Agile Frontier” stage of Agile 2009. Unfortunately, it was not one of the 18 proposals accepted for that stage. We tried again, a few years later, and presented What’s the Point of Story Points at Agile 2012 to an over-stuffed room.
In this session, we said many of the things that people calling for “No Estimates” are saying.
- Estimation is not the goal, and can distract us
- Estimation wastes more time and effort than it’s worth
- Estimation encourages too-detailed planning, which then encourages “sticking to the plan” due to the sunk cost
- Estimation is fractal, and grows as we work in finer detail
- Velocity and estimates get treated as things they are not
- If we work in priority order and limit work-in-progress, estimating the amount of work that will fit a timebox is less important
- If we work on things that are an order of magnitude more valuable than the cost of producing them, then estimation of cost is less important
But a funny thing happened. While arguing against estimates, I found some residual benefit for estimation. From a business perspective, it’s quite valuable to project into the future to predict what might happen, and to make contingency plans, if needed. Making estimation less important does not make it unimportant.
Since that time, I’ve been asking different questions. In particular, what is the residual importance of estimation? I’ve come up with a number of ideas. In no particular order:
- Estimation allows us to bring past experience to bear on future work.
- Estimation is a valuable technique for making decisions, but it is not the only technique, nor is it always applicable.
- Estimates can be used to communicate to stakeholders, particularly those outside your organization, that you have the relevant experience to understand the work and have paid serious attention to their request.
- Estimates can be valuable for routine business activities such as planning release dates, managing risk, summarizing a situation for executive stakeholders, coordinating with parallel efforts, choosing between alternatives, and justifying decisions that spend someone else’s money.
- Estimates are based on assumptions, and using them as a hypothesis can provide early notice that those assumptions might be incorrect when actuals differ from the estimate.
When we have a clearer idea of what we want an estimate to accomplish, we can then ask how can we achieve the important goals simply?
I’ve previously suggested that “Continuous Estimation” might be a worthy tag, in the vein of continuous integration, continuous improvement, continuous design, and continuous planning. I think “Extreme Estimation” also has merit—pushing the limits of estimation to the extreme. The tagline “No Estimates” seems problematic, to me.
- Naming something for what it is not gives little information.
- Telling people not to do what they’ve been doing, or even appearing to tell them that, is needlessly antagonistic.
- I find that I do not want to eliminate all estimates, after all.
When I wrote the title of this article, before starting it, I estimated a “short” overview. As you can see, it’s not so terribly short. Such is the nature of estimates—they represent our best understanding, and sometimes our hopes, at the time we made them. We should always keep in mind that they are just estimates.