Iteration, Increments, and Timeboxes
Like many words, we often use “iterative” and “incremental” quite loosely. I’m OK with that, until that lack of precision causes misunderstandings, disagreements, or misdirection of effort.
It’s quite common for Agile teams to speak of an iteration as a synonym for the Scrum term sprint. Both of these usages really mean timebox.
A timebox is a fixed amount of time given to accomplish some goal. Using a timebox guards against the risk that the work toward that goal will grow unchecked, perhaps without achieving the goal at all. We might reach the end of the timebox without achieving the goal, in which case it would be prudent to reassess the situation. Perhaps we should try a different tactic to reach our goal? Perhaps the goal is not valuable enough for us to spend more effort?
Sometimes the approaching end of a timebox helps us a focus on getting something done, rather than doing the best and most perfect thing. The approaching deadline may stimulate choosing the current best choice identified and avoid the analysis paralysis that leads us to continue looking for better possibilities. This effect is often called the student syndrome and described with the observation that, “if it wasn’t for the last moment, nothing would ever get done.”
When we line up equal-sized timeboxes end-to-end, we gain further advantages in planning and observing progress. We can simply compare the results of one timebox with another, without any fiddly mathematical calculations. We can presume that, all things being equal, future timeboxes will be about as productive as past ones, making estimates into the future reasonably accurate without undue precision.
Human beings seem to do best for the long term when they get into a rhythm in their work. Work songs and sea chanteys were ways to synchronize the rhythm of manual labor. We can get similar benefit using timeboxes for software development or other complex knowledge work.
Where do iterative and incremental fit into this?
Incremental development means that we’re building the end result bit by bit. We can look at this from many angles. Each timebox of work produces one increment. Or each feature we build can be seen as one increment. Or each module.
Project managers have used incremental development for a long time, sometimes effectively and sometimes not. Working in timeboxes is one thing that helps incremental development work better. Another aid to effective incremental development is delineate the work as functional pieces, rather than arbitrary modules. This helps us to know whether an increment is really done or not. And we get a better sense of rhythm if we make these functional increments small and frequent.
But what if the functionality is too big to fit into the timebox? That’s why (well, one of the reasons) we also want to iterate.
Iterative development means going over the same thing repeatedly. Think of smoothing a board with sandpaper. First we sand with coarse grit to knock of the high spots quickly. Then we come back over the same board with medium grit to level the surface more precisely, removing the scratches left from the coarse grit. Finally we switch to a fine grit, removing the smaller scratches of the medium grit without removing large amounts of wood and leaving a more polished surface.
Working in an iterative fashion helps us move toward our goal in smaller steps. It lets us interleave working on the various functions we want to incorporate, effectively doing all of the coarse sanding before proceeding to medium grit on any of them.
And when we look at the product/project as a whole, each timebox is one iteration of the whole. Hence the popular usage of “iteration” to mean “timebox.”
George, great post as usual!
We wrote about this today as well!
As you correctly state, iterations are ‘going over the same thing repeatedly’.
Why would we consciously choose a technique of rework? It’s when we have no other recourse than to try and successively approximate by asking repeatedly “is this right yet?” and responding until we have a successful outcome. Scrum introduced iterations as a way to guess more quickly at requirements, time boxing is just a way to make us guess more frequently.
We used to use the same technique to test our code: developers would write the code, then testers would test, then developers would go rework the code. We iterated on the same code, approximating at each cycle until we reached sufficient quality and functionality.
TDD changed that dynamic. Now when we write a test, some code, and do necessary refactorings we are done with that increment. We move on to the next test. The practice of TDD does not use guessing as the primary means of accomplishment.
Yes, we still use iteration as a safety net to validate that the assumptions of our tests are correct. But the process of creating code from tests is not iterative in nature.
All of this why I have been trying to create a set of Feature-level TDD practices that can reduce and focus the amount of assumptions and move the bulk of feature development into an incremental, not iterative, process.
Marty, maybe it would help understanding if you didn’t use the word “rework.” To me, “rework” has the connotation of doing a poor job the first time and having to do it over. I prefer “refinement” with connotes getting better and better.
The test is not your increment, any more than the sheet of sandpaper is.
Perhaps I TDD at a finer level than you do. I might write a test that a new book has zero reviews (adding the notion, but not the mechanism, for book reviews). Then I might write a test for adding & viewing one review (adding rudimentary mechanism). Finally I might write a test for multiple reviews (refining the mechanism to handle a collection). These tests iterate over the book review functionality, incrementally adding behavior.
Why would you want to eliminate iteration? That would only increase the step size, requiring all the expected behavior to be produced at once. This gives fewer points to reflect and make decisions. It puts more in motion at once, increasing the likelihood of mistakes and ironically increasing the amount of rework.
The process of creating code doesn’t have to be iterative, I suppose. I’ve just found that faster and more effective.
Very interesting post! Enjoyed reading it.
I’m going to pick on “Sometimes the approaching end of a timebox helps us a focus on getting something done, rather than doing the best and most perfect thing.” So here’s a situation: It is the end of the “timebox”, you have to knock out a one-line change feature, which was estimated to take 4 hours. You look at it and you find out that one line is in the midst of deep spaghetti legacy code, which has no unit tests around it. You figure if you have to do a true TDD, it’d take much longer than the time you have until the end of the timebox, yet you are “fairly” certain that one line of change would not break anything else. Do you just make that change without tests around it so you can provide a visible value to the customer, or do you reestimate and carry over to the next “timebox” so you can do a true TDD?
Boyko, it’s great to hear from you!
Dealing with surprises in legacy code is always a challenge. It doesn’t have much to do with the timebox, but with how much time and effort is appropriate to spend on this change at this time.
Except with the most perniciously dependent code, I can generally find some way to get some *characterization* tests around the functionality to be changed. Characterization tests (see Michael Feathers’ book, Working Effectively With Legacy Code) are tests that exercise the code and expect the current behavior, whatever that is. I generally write them with a bogus expectation (such as a comparison with an empty string) and then copy the actual value into the test as the expectation. These tests may not tell you that the code is right, but will alert you to changes (well, some of them) in the legacy code you don’t understand.
Characterization tests are a big help when you can’t isolate the code well enough for true TDD. They don’t provide the same level of reassurance about the functionality, though.
Great post – a common language is so important in a project, but you also clearly and concisely differentiate between incremental and iterative. From a DSDM perspective these two terms are embedded as 2 of the 8 fundamental Principles.
I also like your mention of rhythm – I often refer to the term borrowed from Lean (and German) namely Takt (time).
Keep up the good work!!
Hello and thank you for your great information. I am quite new to Agile. Please may I ask – is it ok to have more than one time box happening at the same time?
I’m not sure what you mean by your question. It’s certainly possible to have different timeboxes for different things. In Kanban, that’s called “cadence.” While timeboxed processes tend to use the same cadence for all things (e.g., planning, implementing, testing, releasing, reflecting), you can have different cadences for these things. For example, most Scrum teams release functionality only at the end of a Sprint (though I’ve seen exceptions to this), while a Kanban team is likely to release whenever a new feature is ready.