What Decision Are We Supporting?
In business, we’re often asked for estimates with too little context to understand the request. When that happens, we’re likely to expect the worst–that our estimate will be treated as a “guarantee not to exceed” and we’ll likely be in trouble at some time in the future. Of course we think that; we’ve been burned too many times in the past. Our fear of the consequences will encourage us to spend far too much time and effort trying to get the estimate “right” so we won’t be blamed.
If an estimate is really an estimate, then we know that it’s “wrong” in the sense that the subsequent actual reality is unlikely to equal it. The estimate is a guess, perhaps an educated guess, predicting the future. Predictions are hard, especially about the future.
Given these problems with estimates, why do we bother to make them at all?
We do so because we have a decision to make, and looking into the future helps us make that decision.
If we’re trying to decide which of two development efforts to pursue next, we’ll want to estimate the value and cost of each of them. Since both value and cost depend on the amount of time we spend doing the work, we’ll want to estimate the time and effort required.
If we’re trying to decide whether a project is worth doing, we want to estimate whether the value is greater than the cost. We may not care what are the actual value and cost, just that it will be profitable.
If project A is clearly more valuable and less cost than project B, it’s clear which to work on, assuming it’s also clearly more valuable than it will cost. But what if it’s not so clear? If project A and B are estimated as about the same value and about the same cost, which would you choose? Unless you have some guideline other than the estimates, it doesn’t matter. If project A will provide value that’s only slightly more than the cost, then we shouldn’t do it at all.
These are just two simple examples. There are many different reasons that a business will want to make forward-looking estimates. There are some caveats with estimating, of course.
Remember that these are JUST ESTIMATES.
We should not trust our estimates too much. Estimates are not data. Even if we’ve done well at predicting the future in the past, we might get this one totally wrong. We should expect our estimates to be off a bit, one way or the other, from what reality shows later. So if our decision is a close call, we should consider what we’d do if we had no estimate, or if the estimate came out differently. Only when the estimate clearly favors one choice over another should we trust it. Even then, we should validate our decision with real data as we go along.
Estimate to an appropriate level of precision
Since we know that our estimates are not data about future events, we should consider the amount of precision we need to make our decision. There’s rarely a need to estimate project completion to the day, or project cost to the dollar. If we think a project is worthwhile if it costs $100,000 but not if it costs $200,000, then we don’t need to estimate with any more precision than to the nearest $50,000. Using greater precision than we need increases the cost of the estimate without increasing its value for helping us decide.
Using an appropriate level of precision will also help us remember that these are estimates, not trustworthy data. There’s also some evidence that precision interferes with moving toward our goals.
Continually check and refine your estimates.
Estimation should not be a phase, that you do once and then hold onto it. Estimates give you a model to judge your progress against your expectations. It allows us to track our investment toward our goal. The actual progress acts as a check on our estimate, potentially giving us early warning of future disappointment–early enough to do something about it.
In the early 2000s, it was often said that “you don’t need to do design in Agile software development.” A lot of people were very dismissive of Agile software development because of comments like that. In truth, you do need to do design. You just don’t need to do it all up front. You need to do enough to see where you’re headed, and continue to question it and refine it as you learn more about what you need from the design. The same applies to estimation. We project into the future given what we know, track our progress, and make new projections as we learn more.
Don’t make this a big job, though. They’re still “just estimates.” Do a little at a time, but look at them frequently and regularly. Cross-check with other ways of estimating. Estimates are tentative; don’t put your full weight on them.
When they’re not agreeing with the subsequent reality, don’t blame the reality. Estimates give you a map; they’re not the territory. When these disagree, trust the reality. Does our decision still make sense? Would we decide differently, given what we know now? Are there possibilities that we didn’t consider when we made our first estimates? Should we change our decision, given where we are and what we know?
The estimates are not the goal, they’re just a tool. When we find they’re “wrong,” there’s no sense blaming the estimates or those who made them. They’re now a historical record of our previous thoughts and knowledge. Instead, use them as they’re meant to be used–as a tool for making decisions. And make new decisions.
I find those that are afraid/refuse to estimate are just not mature engineers. I think this article sums it up nicely -> http://www.kitchensoap.com/2012/10/25/on-being-a-senior-engineer/
I tend to think of “estimates” as time-based. When people request estimates, they typically want to know how long a chunk of work will take.
The problem they are trying to solve by learning how long the work will take is usually related to risk management or cost/benefit assessment to decide whether to proceed with a proposed initiative.
I disagree (with nearly everyone, apparently) that “estimates” are the best or sole way to inform such decisions. I’ve found that using the development organization’s prior delivery history as a guide, we can project the likely delivery performance into the near-term future with sufficient accuracy to inform a “go/no-go” decision.
When we use “estimates” we often end up with numbers that are padded, chopped, or otherwise mangled due to real or perceived management pressure to promise delivery on a predetermined date.
Derek’s comment suggests one of the problems that always arises when this topic of dicsussion comes up – his choice of words suggests he’s thinking about short-term, team-level planning. “Estimates” are most useful at a higher level of abstraction than that. Once the work has been decomposed into “playable stories” (or the equivalent thereof), there’s little or no value in “estimating” the individual work items. We’ve gone beyond the “go/no-go” decision point by then, and we can best serve stakeholders by delivering each work item in priority order without worrying about exactly how long we might guess each one will take.
Although Estimates may “seem” to be a way to track our progress against our goal, that is not what is happening in an estimation context. In an estimation context, estimates are used to *set* the goal. And that is wrong! The investment goal does not depend on the length of the work, it depends on the business assessment of the market. Actually the goal does not have anything to do with software or it’s implementation cost. Software is merely a tool.
Vasco, there are a number of things I don’t understand about your comment.
I don’t think estimates are a way to track our progress. I think they’re a way to give meaning to tracking our progress by identifying when and where our progress differs from our expectations. I suggest measuring progress with something more closely aligned with what we really want. In software development, we typically estimate costs and costs are a lousy metric for tracking progress. Estimating value is less common. In either case, I would suggest measuring progress by some sort of actual value measurement, whether by count of new features, or by actual usage of new features and the benefits derived from that usage.
I also don’t think estimates are used to set goals, though sometimes they are used as proxy measurements for the goals. At it’s simplest, the goal of a software development project is functioning software. Estimates are often used to help us tune that goal to fit our schedule and budget. (They’re also used to coordinate with other efforts related to the software development. Many times developers neglect to notice this usage.)
As you say, software is merely the tool. Delivery of software functionality is only a sub-goal of the organization. If the larger goal is increasing revenue, then measuring that increased revenue is an appropriate measure of progress at the higher level. In some domains, this measurement does not produce any information for extended periods of time, so we often have to use sub-goals to measure near-term progress. In most domains, this measurement has a lot of noise unrelated to our efforts, so it’s prudent to cross-check it with other measures.
Most recently, I’ve only estimated elapsed time, and this is because the biggest constraint on a team’s performance is the organisation itself – it bureaucracy, its facilities and how you may consume them, and the likelihood of drastic changes to the project by executive fiat.
So the history of past projects is often a good predictor of the messiness of an actual project implementation and thus the degree to which you ideal case estimate may need to be adjusted.
When a company with a lot of shareholders is trying to embark on a new, sizable project, it may need to conduct an independent cost estimate in order to make the whole process more open and accountable to the shareholders. For instance, if a public company with lots of shareholders decides to build a new factory in Japan as part of its globalization process, it will have to obtain an independent cost estimate in order to assure the shareholders that everything is aboveboard. These independent estimators will give their assessment of the project based on a number of predetermined criteria that include factors like logistics, manpower, site costs, design and others. Such an independent cost estimate will ultimately have more value than any one conducted by in-house estimators.
Hi, Dorothy,
An independent assessment makes a lot of sense if you don’t trust those doing the work. It seems that you’ve still got a number of problems, though.
What gives you confidence in the estimate made by an independent estimator? Do they have more expertise in the domain than the people doing the work? If so, do you have the wrong people doing the work?
What happens when the work is different from work done in the past? This is a common occurrence in software development.
When you predetermine factors “like logistics, manpower, site costs, design and others,” aren’t you constraining the solution to one envisioned before any work starts? How can you use the learning you get from doing?