Framing the Question
“I need this project done by date D and within cost budget C. Now calculate an estimate on the project.”
A friend of mine used this example to illustrate anchoring bias in estimation. Note, however, that he doesn’t make the question explicit. Further conversation revealed that he had in mind that the date and cost should be the output of the estimation. With that assumption, that statement preceding the request will definitely anchor the answer, and realizing that this bias is likely will call into question whatever estimate is given.
Given the stated need, however, I would reframe the call for an estimate from “When will this project be done and how much will it cost” to “What is the likelihood that the project can be done within these constraints?” When the time and money constraints are already known, it’s somewhat disingenuous to ask that these values be estimated. If the estimate exceeds the budget, then it’s likely that a negotiation will follow. Such a negotiation helps bring the estimate in line with the budget, but generally does little to bring the actual costs in line.
Reframing the question is much more helpful. It should provoke a response along the lines of “We feel 80% confident that we can meet the time and money budget based on these assumptions”¦. We have identified “¦ as risks that would endanger meeting the budget and which should be monitored closely. Another possibility is that the response is of the form “There is a 10% chance that this project can be accomplished within the time and money constraints. If you’d like, we can explore finding a subset of the project that has a higher likelihood of completion but still have value.”
Either of these answers is more useful for the person with the budget who wants to decide whether to invest in the project or not. Sometimes the best outcome for a project is to kill it before starting when it’s unlikely to result in a happy conclusion. Other times we know to proceed cautiously, and can keep an eye on variables that will indicate whether we’re on a path to success or failure given the stated constraints. We should have alternate plans that we can turn to in the event that one or more of the risks materialize.
Such an estimate doesn’t guarantee success, but it does give us a feel for the merits of pursuing the endeavor. The associated assumptions and risks provide information to help us understand whether or not we’re still on track. This probabilistic estimate is much more helpful than is an estimate of the values.
Not all situations benefit from an estimate of the probabilities. We should think about what sort of answer would really help us, and ask the question that will produce an answer of that sort. Asking for “an estimate” without knowing and communicating the form of answer that will be useful is unlikely to give us appropriate information.
George, does the suggestion ““We feel 80% confident that we can meet the time and money budget based on these assumptions”¦” contain multiple types of estimates? All rolled into one generalization? Would we be better to break these types of estimates out explicitly. For example the estimate of 80% “confident”; I don’t believe that estimate unless one can show their work. Then there is the two estimates of time & budget; these are related by some fashion; but are both estimates.
I agree with you. A single figure estimate is rarely a good idea. We are actually working with probability distributions.
Humans are very irrational. Computer scientists such as myself often like to think we are more rational, but when it comes to estimation it is often a charged atmosphere full of irrational and emotive people.
The more money is at stake, the more pressurised the environment tends to be. When this is the case, it always leads to less accurate estimation.
I feel that setting a large budget for a fixed amount of work may often be the first major mistake on a project.
According to Boehm and Jones, a project typically experiences a 25% change in requirements throughout its development, which adds more than 25% effort to the project. Some projects will have much bigger changes, sometimes halfway through development the customers will realise the product is the wrong product for the market.
Rather than thinking that is a disaster we should recognise how much better that is than if that was not realised until after the project is complete.
So I believe a better to think of the project is how can we maximise the value we receive on a daily budget. Every day we know the staff costs to a much better accuracy than we could ever know the real project budget. If we take this approach, then we have a better chance of also giving adequate consideration to quality.
If the customer leads in with schedule and budget requirements, the development team is more likely to cut quality in an attempt to achieve this. This usually ends up having the opposite effect: the project takes longer because there are more bugs to fix.
There is also the costs of the future projects to consider. Is the development team creating legacy code in this project in an attempt to meet an aggressive schedule? If so, the budget for version 2 of the product may be a lot higher.
So there is a lot to consider when planning a project. It important not to lose sight of the big picture (where does the company want to be in 5 years time and how will this project help with that) but it is also important pay attention to the small details (what can we achieve today? What visible progress do we have? Etc.)
David, I don’t know what you’re suggesting should be broken out. Yes, the one sentence representation for this blog article is insufficient for an actual estimation. It represents a way of framing the answer. Certainly that answer should be the beginning of a longer conversation unless the project sponsor decides to abandon the attempt.
Kevin, certainly there are many more aspects to estimation than this blog post mentions. You can find other aspects discussed under the tag “Estimation.” [http://blog.gdinwiddie.com/tag/estimation/]
Your “better way” is a rational approach, but doesn’t take into account the common needs of many stakeholders who might finance a project. As you say, humans are irrational. Much of their decision making is based on emotions, and rationality is used to support those decisions after the fact. Choosing to invest money and opportunity cost on a project is a bigger question for those providing the money than for those consuming the money. It’s therefore worthwhile to spend a little to help them make that decision.
As you suggest, you could just start building and see how that goes. How much would you have to build before you could answer the question, “Can we reach our stated goal within our stated constraints?” How much would that cost? How happy would the stakeholder be if you reach the limits of the constraints and don’t have the desired functionality? Or, alternatively, when would you first make a projection about that probability if you don’t make it at the beginning?
You mention that a lot can change over the course of development. To me, that calls for frequent checks that our predictions are coming true, and perhaps new estimates performed in a different manner. Estimation is not something to be done once and then trusted indefinitely.
Working in a fashion to maximize the daily value created is an excellent development strategy. It does not replace estimation. It does, however, give us a measure to compare against estimates to perform a mutual check. Either, or both, the measured progress and estimated progress could be wrong. It’s like double-entry bookkeeping in that regard.
You mention code quality. I would suggest thinking of code qualities, instead. There are many different qualities you may notice about the code, and different people tend to assume different qualities are the one important one. It seems that you are talking about qualities such as maintainability, flexibility, and readability which help keep development costs lower and development speed higher. I agree that these are important. They’re also hard for non-technical managers and business people to assess. And, even programmer disagree on how to achieve or measure these qualities. Even without schedule pressure, it’s possible to write “instant legacy code.” That, however, is a discussion for another day.