This course of action is obviously prudent when the requirements are handed down to the development organization in the form of a document. It does not, of course, detect errors in the requirements document itself. Nor does it detect errors of interpretation if both the programmers and the testers read the requirements in the same, incorrect way. When there is significant distance between writer and readers, this is distressingly common.

It’s difficult work to write clearly in prose. Publishing industries have developed the roles of editor and proofreader to check documents so that erroneous or unclear segments may be rewritten before they’re seen by potential readers. People who write requirements documents are frequently better versed in the desired functionality than in the process of writing them. And, they frequently don’t have so much help.

It’s also difficult to precisely interpret the writings of others, particularly of people you don’t know. A word may have an obvious and precise meaning in the context of the business, but an obvious and different meaning in the context of software development. In the fields of literature and law, expert interpreters may spend years or centuries honing a consensus interpretation of the written word. In software development, we rarely have time for lengthy discussion of complicated passages. If the writing is ambiguous or contradictory, faithful interpretation of the intent is even less certain.

We often seek clarification of the requirements to back up our reading of the document. Sometimes it’s difficult to get sufficient access to the document authors, and the programmers and testers may turn to each other for what clarification they each offer. I’ve had testers from a different company ask me how my software was supposed to work because they had insufficient access to the requirements document author.

Cross-checking the interpretation of programmers and testers punches holes in the wall of independence, of course. This sad state of affairs is a major source of defects in which the software “works as designed” but not as desired.

We can do better than this!

Instead of the business representative writing the requirements and handing them off as a document, imagine the business representative telling them to a programmer and tester (everyone taking notes as needed to keep things from being forgotten). Imagine the programmer and tester testing their understanding in real time with questions made directly to the business representative while things are still fresh on everyone’s mind. In doing so, they can not only challenge each others interpretation of the requirements, but also the assumptions of the business representative. Together they can hone the requirements to a sharper edge than possible when working separately.

I don’t typically call myself an Agile Coach, though I’ll use that term informally if it’s the term used by those with whom I’m having a conversation. Instead, I call myself a Software Development Coach. To me, the goal is developing software more effectively, not becoming Agile. Agile processes and practices happen to be excellent tools for effective software development, but lousy goals in themselves. Or so it seems to me.

This morning, I got a call from a recruiter looking for an Agile Coach for a client. They were a bit unhappy when I gave them my daily rate. “The client has a budget and will never pay that much.” When I asked what rate they were expecting, they said $50/hour, all inclusive.

I made more than that a decade ago as a contract programmer. I cannot imagine finding a competent experienced coach for that rate. I’m sure that you can find a body to sit at a desk, though. Is there value in that?

This low rate, and the fact that cost is a primary factor, but value isn’t even mentioned, makes me wonder about what this role of “Agile Coach” has come to mean to organizations looking to hire them.

If the value received and the cost paid are nearly equal, then cost is of critical importance to avoid spending more than the value received. If the value received is an order of magnitude more than the cost paid, then variation in the cost has much less affect on the Return On Investment.  This is very similar to the point that Tom DeMarco made in his article “Software Engineering: An Idea Whose Time Has Come and Gone?” [IEEE Software, July/August 2009] “This leads us to the odd conclusion that strict control is something that matters a lot on relatively useless projects and much less on useful projects.”

I fear that for many large companies the generic “Agile Coach,” and the Scrum specific “Scrum Master,” has become a term for a person who neither programs nor tests software, who is added to a development team to make it Agile. It’s as if you could sprinkle some pixie dust on your development teams to make them more productive, or whatever advantage they can see in the adoption of Agile.

Is that what “Coach” now means?

I suppose I shouldn’t be so surprised. The meaning of “’instructor/trainer’ is circa 1830 Oxford University slang for a tutor who ‘carries’ a student through an exam” based on the Hungarian word for a carriage. It’s clear that some people have always wanted to hire someone rather than learn to do for themselves.

Certainly there’s value in getting one project done a little more effectively than you might otherwise. If you can hire someone to work full-time on a project and guide the actions the team to improve the effectiveness for the duration, then I expect you’ll get enough marginal value that you might get some ROI. I’m skeptical about accomplishing that with the most cut-rate of coaches, though.

The true value of coaching, however, is to build the capability of the existing team. Rather than making choices for the team, the coach provides guidance about the choices available, perhaps making recommendations, and encouraging them to consider the options and choose their actions. The coach teaches the team about techniques or tools that increase their available choices. The coach offers observations about the team’s activities, and helps the team make it’s own observations and reflect on them. The coach helps the team articulate the results it wants, and generate courses of action to achieve those results. The coach partners with the team on the coaching process, but allows the team to exercise its own judgement about the software development practice. The coach does not become a member of the team, but endeavors to wean the team off of the need to consult with the coach on a regular basis.

There are consultants whose business model includes making the client more dependent on the consultant. That, to me, is not coaching. And that’s not the model of consulting that I choose.

Some months back, I noticed that their website had become bandwidth greedy with javascript doodads. This made it painfully slow on many hotel internet connections. They seem to have backed off from that a bit, but have now broken it in a way that prevents me from buying the ticket I want. (In addition, when I call the “Southwest Airlines Customer Representative for assistance” as prompted on the website, I find they no longer have enough to handle the call volume. I wonder if they’ve reduced the support, or if the call volume has increased, or both.)  There’s a lesson here about considering all of the important user stories when implementing new functionality, so it bears describing.

I went to the Southwest web site to purchase my ticket to Salt Lake City for the Agile2011 Conference. I took a look at the available flights on the appropriate travel dates. Then I remembered that I have an unused travel award, so I went to check if those dates were available, even at this late date. Hooray, they were! Boo hiss, the flights I want were not. Oh, well, it serves me right for putting this off so long.

So, I went back to buy the tickets with money. But when I search for the flights, instead of showing me the flights for purchase, it shows me the dates available for using my travel awards. I sign out of my account and try again. Now it gives me an error: “Oops! We are currently unable to complete your request due to an undefined error.” I try deleting cookies. I try closing the browser and re-opening it. No dice. Still, if I’m logged into the account it gives me the travel award availability dates, and if I’m logged out it gives me an “undefined error.”

I also notice that even after deleting cookies and restarting the browser, when I go to search for a flight it remembers my airport and date selections. This means that they’re keeping state on the server side, probably associated with my IP address (since I’ve logged out and deleted client side session data), and there’s no way I can find to reset this state.

In their attempt to be helpful, they’ve neglected to consider many common user scenarios. The Southwest website has long maintained enough state that you couldn’t do multiple searches in parallel to compare them and then select the one you wanted. But you could, in my experience, restart the search once you figured out which you wanted. Now there’s no mechanism I can find (and I’ve been struggling with this for over an hour) to take a purchase path that involves changing your mind and backtracking.

Am I the only indecisive customer? Or has Southwest forgotten the variability in the ways that humans approach things, and coded their website for a robotic “customer unit” that follows the one expected scenario?

More importantly, what do you and your organization do?

These are exciting times.  The tools of acceptance testing and behavior-driven development are progressing beyond the domain of the techies.  They are entering the realm where they can help the Whole Team.

Using these tools, or even better ones to come, the Three Amigos, Developer, Tester, and Business (and User Experience expert, and Manual Writer, and …), can invent their own shared language, specific to the system under development, to talk about what the system is supposed to do.  The acid test: that the Business representative feels comfortable reading the examples written in these terms and definitively say, “Yes, that’s what I mean” or not.

After the presentation, Uncle Bob Martin showed me the Given When Then capabilities he’s added to Fitnesse since I last looked at it last fall.  Other tools mentioned included Cucumber, SpecFlow, JBehave, easyb, GivWenZen, Morelia Viridis, StoryQ, and StorEvil.  I’m sure there’s more to come.  And, as we use these tools more and more, I’m sure that we’ll develop patterns beyond Given When Then and language specification tools beyond regular expressions.

Just remember, it’s not the technology that’s important.  It’s the communication between the Three (plus) Amigos.

AGILE methods emphasize realizing short-term OBJECTIVES rather than creating long-term objectives and trying to satisfy them.

I completely disagree with this statement.  I don’t agree that short terms objectives are the emphasis in Agile methods and I don’t agree that short term objectives are preferred over long-term objectives.  I think that Steve’s misunderstanding is rooted in the fact that he has only read about Agile methods, and not practiced them.  I think that it is impossible to get a deep understanding of Agile methods by reading without experiencing.  Therefore, I’d like to encourage Steve and others talking about Agile methods to try to get some experience before making such statements.  I’d also like to offer an explanation that attempts to clear up this particular misunderstanding.

Steve described (in a telephone conversation) “the good old days” when he would spend weeks or months documenting the objectives of a project in a detailed way that could be tested.  He noted that Agile projects don’t do this requirements definition phase.  So far, so good.  He then, however, made a leap to the conclusion that this was an indication that the long term objectives are deemphasized.  Bzzztt! Wrong answer.

Agile methods emphasize the long term objectives every iteration–or even every day.  I’ve previously blogged about What is Agile? In that posting, I noted that the fundamental definition of Agile (in my mind) was “the use of feedback that is

• frequent
• timely
• aligned with our desired goals
• pervasive”

That “aligned with our desired goals” includes the long term goals.  Indeed, short term goals are mostly subservient to the long term goals.

The short term objective of delivering working software at the end of every sprint or iteration is a tool toward meeting the long term objectives of the Customer or Product Owner.  As the Agile Manifesto Principles state

Working software is the primary measure of progress.

They state this because working software can be compared against the long term objectives to see if those objectives are being realized, how much of those objectives have been realized, and if those objectives still represent what the Customer wants in light of what has been learned.

So, what’s happened to the expression of those long term objectives that used to be entombed in detailed requirements documents in “the good old days?”  Those objectives are now expressed in conversations with the Customer/Product Owner, who doesn’t “send the developers away to build two weeks worth of software,” but remains engaged in the process, clarifying, learning, and steering the project toward long term success.

At least, that’s the way it happens when things are done well.  There are often people playing the Customer/Product Owner role who came from the same “good old days” that Steve remembers, or whose organizations have a culture bound strongly to that phasist vision of software development.  In some cases, development does proceed without a vision of the long term objectives.  And yes, Steve, things don’t work so well when that happens.  This post is written to help those people understand a more effective approach that doesn’t throw out the baby of good things they know with the bathwater of phasist, predictive software development approaches.

Steve, I’ve tried to represent your position as clearly and fairly as I can.  If I’ve misinterpreted anything, please let me know or leave a comment here.

Jim, I think you’ve left out, or under-emphasized, an aspect here.  It’s perhaps not explicitly called out in the XP literature, and isn’t mentioned by Scrum (neither of which mention the QA department as an entity), but it’s an important aspect in successful Agile adoptions in corporate IT environments, in my experience.  That aspect is Defect Prevention.

If the QA people take the normal approach of waiting to the end and then looking for bugs, they’re going to be outside of the tight feedback loops of Agile development.  It’s so much more effective, and more Agile, for QA to be intimately involved throughout the process.

You mention helping the business provide clear examples of what they want.  A good QA person can help the business elaborate those examples to cover situations that might not have been considered.  It’s generally good to do at least some of this example elaboration with both the business and the developers.  Having these three perspectives at the same time is hugely helpful in covering the ground thoroughly and uncovering the odd cases that will show up later as defects.

You mention writing automated non-functional tests.  I’d also add automated functional tests from the business point of view of the full system.  These are often not included in the tests produced by the developers.  Fundamentally, these are executable elaborations of the business examples mentioned above.  Fortunately, there are tools that allow this expression in language that the business people can read to verify, “Yes, that’s what I mean.”  For many testers, connecting this business-accessible language to the system under development will require the assistance of developers.  This is an example of the partnership between various members of the team working to prevent defects, as the resulting regression test is an important tool to protect against losing or changing old functionality when new is added.

You mention doing exploratory testing, but I would elaborate that to include doing exploratory testing of unfinished features. Long before you can “find bugs,” you can notice spots where the developer’s understanding of the business’ desires are incomplete or slightly off-kilter.  Or, you can notice situations that were not foreseen when drawing up the business examples. Or, you can notice things that seemed reasonable when discussing them, but seem less so when seeing them on the screen. These should be brought to the attention of the business experts providing the wants of the business.

I know all of this is second nature to you, but it’s not to many of your readers.  The role of QA people in Agile development is much richer, more varied, and important than might be obvious from your description.  At least in corporate IT environments, I’ve found that this deeper involvement is important for success even when (or perhaps especially when) the development is following Scrum without the XP technical practices.

Serial lifecycles provide a (false) prediction. And, boy oh boy, is that prediction comforting to your senior managers. “When will the project be done?” might be their most-asked question. Of course, a serial lifecycle provides a prediction that’s almost guaranteed to be wrong, especially if you use a project scheduling tool. The tool provides you a single-point estimate, which is the first date you can’t guarantee the project won’t be done by–the first possible, optimistic date.

I like that characterization of the predicted date.  Another characterization, usually true of serial lifeycles, is that the predicted delivery date is the first day of schedule slip.  I’ve seen many serial projects get almost to that date before they first admit that they’re in trouble.

There was a comment from Pawel Brodzinski to Johanna’s post saying,

About predictability I’d say it doesn’t have to be false. Waterfall project which was properly planned and is properly managed should end around deadline and should deliver functionalities specified in requirements.

He’s right.  It doesn’t have to be false.  It usually is, however, because a serial lifecycle doesn’t give you any tools to manage schedule risk.  It only gives a prediction.  Experienced managers will guess the amount of padding to add to the schedule to give themselves a chance of meeting it.  Cautious but less-experienced managers will add a lot of padding, to increase their chances.  Of course, a development team working to meet a highly padded schedule may be so over-confident that they slow down to match, and then still miss the schedule if something unexpected happens.

In Pawel’s own blog post, he quotes Steve McConnell who describes an XP software development project that failed to produce a salable product because Customer (a.k.a. Product Owner, in Scrum terms) didn’t have a vision for a salable product.  For some reason, probably ignorance about Agile Software Development methods, Steve lays this failure on the development team creating one iteration at a time.  He apparently missed the practice of release planning.  When he says they “did it by the book,” they obviously didn’t use the book Planning Extreme Programming, which was published only a year after the first edition of Extreme Programming Explained: Embrace Change. Surely in two years the team could have read a second book.

Back to the point about Agility and Predictability, Steve says,

True agility–which means adopting a posture that allows you to respond rapidly to changing market conditions and customer demands–conflicts with predictability. Some businesses value agility, but many businesses value predictability more than they value the ability to change direction quickly. For those businesses, becoming more Agile is a second level consideration; the first level consideration is how to become more predictable.

Here, Steve seems to make the common mistake of using the dictionary definition of “agility” instead of checking the values and principles of the Agile Manifesto.  The “Agile” of the manifesto is a brand name, of sorts, to replace the wimpy “light-weight methodology” term that was starting to be commonly used at the time.  It’s a good brand because Agile does allow you to change directions more easily.  But Agile doesn’t require you to change directions and it certainly does expect you to have a direction.

How does Agile give you a predictable schedule? By giving you early, frequent, and fine-grained feedback and allowing you to steer the project to that schedule.

Let’s take a look at the release plan created at the start of the project.  First, write story cards for all of the features you want in your product.  These cards will generally be big stories, a.k.a. epics, and you’ll want to break them into smaller stories for development.  The big stories will work just fine for release planning, though.  Estimate the size of these cards.  I prefer estimating in story points, a fictional, unitless measure.  Some people like to start with ideal engineering days.  Either way, you have to guess how many of these your devlopment team can accomplish in a short iteration.  (I recommend one or two week iteration lengths.)  Count up the story points, divide by this guess, and you’ve got the number of iterations it will take to create the software.  This plan is exactly the same as the plan created by a serial lifecycle.  It’s just a guess–one made as carefully as our experience will allow–but still a guess.

Now develop software for one iteration.  How many story points did you accomplish?  Was this different than your guess?  If so, you may want to adjust your prediction to use this value instead of the guess.  Sometimes people may want to wait for three iterations before doing so, to let the team get up to speed, but they’re put on notice at the end of the very first iteration.  That makes a pretty good prediction, don’t you think?  And you can refine your prediction at the end of each iteration.  With a serial lifecycle, you’re either stuck with your original guess, or you get nervous enough that you make a new guess.

If wishes were horses, even beggars would ride. My mother used to tell me that when I wanted something that was clearly out of reach.  Sometimes a software development team just can’t accomplish all of our wishes by the desired date.  We’ll see this as we replan our schedule at the end of each iteration.  And, unless the desires and capabilities are completely out of whack, Agile Software Development gives us the tools to meet the desired date anyway.

How can we meet the date when there’s too much to do?  We do the most important things first.  And we do things at a fine-grained level.  Instead of leaving whole features to the end, we leave “nice to have” refinements of features to the end.  And we then trim scope to fit the date.

We can do this if we’re getting each little story done-done as we go.  This includes testing, so we know it works the way we want.  And it includes automated regression tests, so we know it continues to work without getting buried by re-testing.  A serial lifecycle leaves this important feedback to the very end, when there’s little time to remedy problems that are discovered.  That late discovery leads to the lack of predictability produced by a serial lifecycle.  Agile Software Development gives you more predictability than those carefully planned-up-front project plans can provide.

Oh, and yes, it can provide the dictionary definition of agility, too, should your goals change.

It sounds very odd, doesn’t it?

Now I happen to know that this particular bank has worked to embrace Agile software development. I know people who have worked with them to do so. And I’m sure that, considering the size of the organization, they’ve made great strides in improving their software development practice. Yet the events that transpired today tell me that they’re missing an important feedback loop–arguably THE important feedback loop–the one that involves the customer.

Here’s what happened: I got an advertisement in the mail from them, offering bonus rewards when purchasing from certain of their business partners. This advertisement was printed in full color on both sides of a cardstock trifold that folded down to about #10 envelope size for mailing. It was mailed at presorted first class rates, not bulk mail. This advertising campaign obviously cost some time and money and effort.

And it was effective. I booted up a PC and headed to their web site to sign up for this program. Huh? I get an error message when I try to login. I tried again. No joy. I called the 800 number. And as I was punching buttons to get through the voicemail system, I recalled having a problem when I last tried to use their web site, about six months ago.

Well, the customer service people were apologetic, and they forwarded me to the tech support people. The tech support people were apologetic, but explained that when online accounts weren’t used for a time, they expired the accounts.

“But I don’t use the online account on a regular basis!”

“You don’t have to login every month, but you should do so every two months. You’ll just have to register a new login.”

“You mean I can’t use the same login I had?”

“No, you have to use a login that no one’s used before.”

This scheme is a non-starter for me. I’d effectively be creating a one-time-use login every time I used the system. And since I couldn’t use the login that comes naturally to me, I’d have to keep track of my current login ID in case I came back to the site in a short time. But since I was assured there was no way to reuse the existing login ID, I asked to be transferred to someone who could change this policy.

Well, they said they were transferring me back to Customer Service. But whether there was some error or they knew that there really wasn’t anyone who could change the policy, the call dropped and I had to call back.

The Customer Service person offered to connect me with a Supervisor, but warned that they had no power, either. She confided that there was a “general correspondence” fax number that, if there were enough complaints, might get someone’s attention. I took down the number.

But I don’t send many faxes, and it’s not convenient for me to do so. I probably will send them a copy of this post, but I’m dealing with them as a customer, not as a consultant. Is it worth my time to help them straighten out their business processes? Hence the application to their competitor. Maybe they’ll be better.

An Agile development team works hard to deliver exactly what satisfies their Customer (in XP terms) or Product Owner (in Scrum terms). There are numerous feedback mechanisms to measure how well that satisfaction is produced, and this can be used to steer the project. It’s customary in larger organizations (and sometimes in smaller ones) for the person (or people) playing the role of Customer to be a proxy for other parties who have an interest in the success of the venture. These other parties may be User Experience people, Marketing people, high-level Executives…. These people, in turn, are often proxies for others. It can be a long chain of proxies before it reaches the end-user customer-who-buys-the-service.

What this company has done, is to effectively prevent any feedback from those end-user customers from reaching people who make the decisions. The problem is not that they have written a policy into the software that does not work for their customers. Everybody makes mistakes. The real problem is that there’s no feedback mechanism for discovering their mistakes until enough customers leave that the Accounting office notices. And the Accounting office may never notice, given that this is business as usual for them.

So, how is it in your company? Do you work hard to create a web site to be the center of your customers’ lives? Do you let the viewpoint of that website overrule the viewpoint of the marketeers trying to increase revenues? Do you have a mechanism or process to measure the effectiveness with real customers? And, perhaps most important of all, do you have a communications channel to allow receiving information from real customers who take the trouble to let you know when you’ve made a mistake?

It’s all about feedback.

Our teams sometimes have multiple projects. I am wondering what is the best way and what is the SCRUM way of handling this. My feeling is that the best way is to have a single backlog per team (even if this means that in a sprint the team is working on backlog items belonging to multiple projects). I think the purists will recommend splitting the team and having multiple backlogs.

That’s what Gilad Gruber asked on the Scrumdevelopment list. This question reminded me of a client I once had.This client did split the team and used multiple backlogs. It was very convenient for the Product Owners. The could each concentrate on their own piece of the pie and ignore the needs of the business as a whole. Had it worked well, I might never have worked with them. What went wrong?

First, it took a lot of pre-planning by the development manager to allocate developers to the various areas of the business. It’s not easy to guess, a year in advance, how much development capacity each would need. And since the development capacity was split by assigning developers to the business line, the allocation was more-or-less static for the year (unless you wanted to re-do this guessing more than once). But each Product Owner got his or her own piece of the pie–something they could count on having and not worry about sharing. Except when it didn’t work out that way.

Given the number of developers and number of product owners, it was not practical to split the teams into integer numbers. As it was, the developers assigned to any product owner was a small team, from two to five to a team. Some of the developers had skills in short supply, so they might be assigned on a percentage basis to two or three Product Owners. While this made the teams bigger (though still a little small), it also split the attention of the developers. Estimates being estimates, and the work to satisfy a Product Owner being lumpy from time to time, the hours and percentages of developers didn’t always work out easily. It was left up to the developers to make things work. Usually they’d negotiate with the Product Owner whose work was being delayed for another, but not always. In any event, it was the developers who were making the decisions about which work had priority.

I’ve described a situation here where development management is making the large-scale priority decisions balancing between business groups, and developers are making the short-term priority decisions of what gets done first. Wouldn’t it be so much better if the business people could get together and decide business priority instead?

Gilad, I highly commend your suggestion of a single “product” backlog per team, even though it may contain stories from multiple projects. It may even be that stories for multiple projects are picked for a single sprint, just because that’s how the sprint boundaries and priorities fall out. Yes, it’s true that it’s better (more efficient) if the team concentrates on one thing at a time, but that’s a small price and the team will probably work it out OK. It’s much more important to have clear priorities based on business value. And if the business can’t decide what’s most important, they should solve that problem rather than abdicate to development.

If you could get the (client) boss(es) to understand JUST ONE THING about computer consulting and contracting, what would it be?

It would be that software development is not a commodity. The cost-value equation is not just enhanced by cutting costs, but more effectively, by enhancing value. It’s not just a matter that software is produced, but what software is produced. Does that software clearly express the nature of the business? Is it an asset that can be extended and adapted as the business grows and changes? All too often, the only measure applied seems to be the hourly cost of the software production (neglecting even the number of hours required). Too often I get the feeling that I, the consultant, pay more attention to the long-term value to the company than do the employees and executives of the company.

I can sympathize that the value of software can be very hard to measure. This value has two components. The external component is the costs saved, or the income received, by having the software rather than not having it. These sorts of measures are dependent on estimates and assumptions, and therefore imprecise. They are the sorts of estimates and assumptions that the marketing and accounting departments are accustomed to make. The CIO can simply ask for these, assuming there is sufficient trust in the organization. Asking for an explanation of the rationale behind the estimates can strengthen your sense of trust if it’s weak.

The internal component is the foundation which this software provides for building the next software you need. This component falls naturally into the IT realm, but how do you measure it? From an executive office, the internal quality of the code is invisible. You could ask the opinion of a trusted technical person in your organization–but you may be worried that they, having a hand in guiding the creation of the software, will take a less-than-critical look. You could ask an external consultant for an assessment–but you may be worried that they will merely tell you what they think you want to hear. Or that they may tell you what they think will fuel your fears and provide them with future work.

If asking others for an opinion is an unreliable means of ascertaining the internal quality of your software, what might you observe to infer it yourself? As I mentioned above, one characteristic that “good” software has is that it’s more easily extended and adapted. You might look at how your organization reacts when asked to extend or adapt an existing program. Do they groan and suggest a ground-up rewrite? Or do they smoothly provide the modifications as if they were expecting them to come? Do such modifications seem harder over time (a sign of increasing “technical debt”) or do things seem more stable? You’ll find it easier to monitor the ease and success of such modifications if you have lots of smaller projects rather than a few big ones.

Now that you’ve started monitoring (albeit indirectly) the quality of your code, how do you go about improving it? Outsourcing may reduce your costs, but is unlikely to increase the value. A contracting company, particularly one in the outsourcing business, is most likely to do the minimum work necessary in order to cut costs and ensure winning the bid. The sort of quality that enables long-term success requires just a bit more than the rock-bottom minimum effort. Even if the outsourcing company does an excellent job, the knowledge of the software internals and how it maps to the business of your company will not be known by your own employees. When next you want modifications made, you’ll find you’re even more likely to go with that outsourcing company, as they know the business better than your own IT department.

Long-term success is more likely to result from investing in your own capabilities. Train your own employees to produce the kind of quality that can support the company through the inevitable coming storms. Build a culture of excellence and knowledge. That culture will remain, even as employees come and go. Make it possible, no, make it easy for your employees to share their knowledge and learn from each other. By all means, bring in outside contractors and consultants to help where needed, but bring them into your organization in a fashion such that your employees learn from them–increasing your internal capabilities. Those you bring in should not only increase your capacity while they’re there, but leave behind an increased capacity after they’ve gone.

There is an old saying in organic gardening,

Feed the soil, and the soil will feed the plant.

I’ve seen this attributed variously to Dr. William A. Albrecht, a former chairman of the University of Missouri’s Department of Soils, and to Robert Rodale, son of J. I. Rodale who founded Rodale Press and Organic Gardening magazine. Whoever coined the phrase, it illustrates the greater value of creating an environment for success, rather than focusing solely on the immediately desired success. Similarly, as Franklin D. Roosevelt noted that

The Nation that destroys its Soil, destroys itself,

so does the Company that destroys its IT Capability, destroys itself. The dependence of business on IT isn’t going away any time soon. Any business that must depend on others for core development functions will be left begging when times are hard.

So what is the heart of Agile practice? In the ensuing discussion, Dale Emery posts a message the turns attention to feedback.

The whole team focuses intensely on producing accurate, relevant, timely feedback about product, project, and process.

I’ve written about the importance of feedback, before. Using feedback is not the defining aspect of Agile, of course. Using feedback is the basic mechanism for any control system. As Kelly Anderson puts it,

The core of any successful human endeavor is timely and accurate feedback. Human beings learn from feedback. Some may argue that it is the ONLY way humans learn. If you believe the neural networks folks, it’s ALL feedback.

I have some minor quibbles with that statement, though I fundamentally agree. Sometimes we get lucky and hit the target without feedback. As Kelly later says, “Give a trained marksman a good rifle, and he can hit the target with the first shot.” This is an example of a good first guess.

Also, I don’t think the importance of feedback is restricted to human endeavors. Control systems work by comparing the actual with the desired, and using some portion of the resulting error measurement as an input to the system to reduce that particular error in the future. Amplifier design controls the gain by the same method, feeding some of the output back to the input. In both of these cases, there is an inversion of the signal being fed back, hence the technical term “negative feedback.” That’s the mathematical “negative,” not a value judgment. In Jerry Weinberg’s Quality Software Management: Systems Thinking and Peter Senge’s The Fifth Discipline: The Art & Practice of The Learning Organization, the authors show how this feedback is essential in managing and learning. This is true even with self-organizing teams. Feedback is the basic means of steering, so that we correct as we go rather than just close our eyes and head in the direction we initially chose.

So, if feedback is important to the practice of Agile, but is used in all controlled processes and isn’t solely the domain of Agile, what makes Agile, Agile? As Jeff McKenna stated years ago,

Why, it is the taking of small steps.

If feedback is applied frequently (or continuously), a control system will track the desired output much more closely, with smaller errors and, therefore, smaller corrections. If you’re driving a car and adjust the steering wheel only every 10 feet, your path will be rather jerky compared to that if you adjust the steering wheel every foot. Of course, it’ll still be much better than if you would adjust every 100 feet. Working in smaller increments allows you to adjust to minor perturbations before they become major ones.

Feedback also needs to be timely. Delays in feedback cause control systems to become unstable. They cause amplifiers to become oscillators–howling out of control like Jimi Hendrix’ guitar. If you adjusted the steering wheel every foot, but based on the input you received 10 feet ago, you’ll weave down the road like a drunk driver. (In fact, delayed reaction time is exactly why drunk drivers weave down the road.) Ten feet ago, you might have been on the right side of the lane, so you decide to adjust the wheel to the left. If you’re now on the left side of the lane, that “correction” is exactly the wrong thing to do, increasing the error rather than reducing it. It’s inevitable that your adjustments become larger and larger until you run off the road.

Kelly also points out that “Inaccurate feedback is also destructive.” Destructive might not always be the right word. In an amplifier, for example, if the amount of feedback is different for different frequencies, the amplifier becomes a filter. That may be what’s desired, or it may not. You could, for example, ignore the end-user feedback about the aesthetics of your software, but pay attention to feedback about the functionality. That would arguably produce a better functioning product, if not a pretty one. As function is, perhaps, a more objectively measured attribute than aesthetics, this might be a good trade-off. Or, it might not.

Agile chooses, as a desired goal, working software that meets the Customer’s needs. Note that, for practical reasons, this Customer role (Product Owner, in Scrum), is a proxy for all the stakeholders who desire the creation of the software. If the person(s) filling this role make “bad” choices that don’t represent those stakeholders, that would illustrate the destructive “bad feedback” that Kelly mentions. Suppose that Customer desires elaborate design documents rather than software? Or measures lines of code produced rather than useful functionality? If you think about this, you can see where the XP testing practices promote working software that meets customer needs rather than the predictable outputs of such misguided measures. The feedback must be aligned with our real goals.

Feedback that is only produced by the completion of the desired software, though, would be neither timely nor frequent. Don Wells has an excellent graphic of feedback loops that operate on different time scales. Good pair programming gives us feedback from our pair about many aspects of our programming. Unit tests give us feedback that our code does what we think it does. There is feedback at all levels of software development. Then, also, retrospectives give us feedback on our software development process. The feedback must be pervasive.

What about the other XP values? Ultimately, I think they all resolve to feedback, too. Communication is how feedback happens. Simplicity makes feedback more frequent, and also enhances the communication. Courage allows communication, and hence feedback, to happen even in difficult situations. Respect allow communications to happen between people; without respect there is little communication even if there are lots of words.

So, what is Agile? I think it is the use of feedback that is

• frequent
• timely
• aligned with our desired goals
• pervasive

All the rest either supports this feedback, or is window-dressing.