And it’s very different from the role I’ve normally seen them take.  Most software architects got the title because they were very good software developers.  I’ve seen far to many who assume that they should now do the “highest level” of the work they’ve been doing, that of the system level design, and leave the more mundane work to those who are still software developers.  In other words, they’re considering their promotion from the point of responsibilities they’re shedding rather than those that they’re taking on.Alan, instead, speaks of the enormous responsibility that he has with the role of architect.  He says, “my job is to make sure that the project doesn’t fail for technical reasons.”  There’s no refuge in saying, “I did my part right.”  It all has to work.

This is an important point for all of us to consider, whether architect or not.  It’s never very satisfying to hide behind the statement, “I did my work well, but those people over there screwed everything up.”  We can all do better than that.  We can all look around and see what else is going on.  We can do things to help that larger picture.  We can do things to adjust to the reality of that larger picture.

Of course, we can still fail.  But when we fail, it’s a pathetic refuge to blame the failure on others if we made no attempt to succeed at more than the tasks right in front of us.  And if we did try hard to help the project succeed as a whole, but it still failed, then must accept some responsibility for that failure.

“In spite of my best efforts, the project failed.”  There is, in that statement, both a sense of failure and a sense of pride.  “I gave my best efforts.”  Perhaps they were not enough for these circumstances.  Perhaps they were the wrong thing in these circumstances.  We do well to reflect on what we did, why things happened the way they did, what we might have done differently, and what we might do in the future.  Retrospecting as a team would be wonderful.  Retrospecting by ourselves is something we can always do.

This topic is a great illustration of Jerry Weinberg’s statement, “The ultimate limit to software is courage. Imagination is a distant second.”

Now, go read Alan Keefer’s post, Taking Responsibility.

OK, I know that “acceptance tests” are somewhat a misnomer.  While they may provide a go/no go indicator for the functionality of a user story, we all know that it’s possible that the application passes the test and still isn’t what the Product Owner or Customer wants.  You still need to show it to the Product Owner/Customer to get their acceptance.  Bear with me, though, and let’s use this common term.

So, a Product Owner, a Developer, and a Tester walk into a bar sit down to talk about something that the system under development should do.  The Product Owner describes the user story.  The Developer and Tester ask questions (and make suggestions) until they think they can answer the basic question, “How will I know that this story has been accomplished?”

No matter how or when it’s done, these three amigos (to borrow a term from my friends at Nationwide) must agree on this basic criteria or things will go wrong.  Turning this agreement into an automated acceptance test (or three) gives it a precision that often tests the agreement and uncovers fuzziness or conflicting definitions in the words we use.  Automated acceptance tests help us express our expectations.

If you don’t use automated acceptance tests, how do you clearly communicate desires (”requirements”) between the business, the developers, and the testers?

If your testing is manually executed, or is automated using record-and-playback, then you’ll have the problem where the testers have to wait until the developers think they’re done before they can start verifying the functionality.  This puts the testers behind from the very beginning.  It also delays the feedback to the developers when the functionality doesn’t behave as expected and results in bug-fix cycles on code thought to be complete.  These things combine to slow down the pace of development.

It’s more valuable to automate those tests while the code is still written.  As development proceeds, you can see those tests start to pass, providing a clear indication of the progress.  If a developer writes code expected to make a particular test scenario work, but the test fails, then you can delve into the issue right away.  Is there a mistake in the code, in the test, or just a lingering disagreement about what we intended to do?  Automated acceptance tests express the growth of functionality in our application.

If you don’t use automated acceptance tests, how do you monitor the progress of development?

Once the functionality works, we want it to continue working, unless we expressly decide it should work a different way.  If we want to know that it continues to work, we need to verify that.  That means that we need to continue to check a growing amount of functionality.  If that checking requires significant human effort, we’ll soon be overwhelmed by it and our progress will get slower and slower.

Computers are great for doing our repetitive grunt work.  Yes, it’s a continually increasing job for them, too, but they’re usually faster than people, they can work longer hours, and they can easily scale to handle more work by adding hardware.  If computers are checking that all of our tests are still working on a daily or more frequent basis, then we rapid notification when we’ve accidentally broken an old feature.  Automated acceptance tests express our confidence that the system continues to work.

If you don’t use automated acceptance tests, how do you maintain confidence that the system still works?

If you don’t use automated acceptance test, please let me know the answers to these questions–especially the last one.

Bob Payne and I recently had a conversation with Sanjiv Augustine about the issues, and some ways of dealing with them.

They should never, in a million years, have been doing Agile.  They were not ready for it…  They could not be helped.

Ouch! Are there really organizations that must be written off as hopeless?

I think not. I think a more accurate description is “We could not help them.” This is not such a bad admission. There may be many good reasons that we are not the person that can help the organization.

Maybe the solutions we know how to provide are not appropriate for the current state of the organization. Josh says this organization was not ready for Agile, and I believe him. But maybe they were ready for some improvements that would help them be ready for Agile in the future. In other words, perhaps Agilization was appropriate for the current context, but not yet an Agile Transition. Could we help them with that?

Maybe the current situation has impediments that we don’t know how to address. Josh says that they had “a newbie executive who didn’t have any connections within the organization.” Building those connections, and getting support and collaboration from the larger organization around the one you’re helping, is beyond the scope of an Agile Transition. Yet almost any transition needs some help in this area. Even an executive with good connections and considerable political skill is likely to need some help noticing things that are significant in terms of the transition, and help devising strategies to deal with these issues. Could we help the newbie executive make connections and learn how to work with the politics?

Maybe we could. Or maybe not, and it’s better for them to work with someone else. I’m happy to recommend someone else when a job is outside my range.

Of course, Josh knows the sweet spot of Industrial Logic. He’s learned to turn down work where he can’t help. I highly recommend listening to this talk. He’s got some great things to say.

If you’ve got the time, give it a listen and give us some comments.

In my experience, most organizations have much room to improve both their project management and their technical engineering practice.  Those that start with Scrum seem to improve their project management practice enough that the deficiencies in their technical engineering practice become more painfully obvious.

The answer is simple and obvious–improve the technical engineering practice.  The way to do that is not so easy, however.

For example, Tim Walker suggests “Attention to technical excellence and hiring motivated people are required for it to be successful.”  As true as this is, it’s no way for an organization to get from where it is to a desired state of being able to reliably create desired functionality in their software.

People cannot suddenly be attentive to technical excellence.  They must slowly acquire the awareness and deeper understanding that they lack, or they would already be attentive.  Hiring motivated people suggests that those doing the hiring will suddenly be able to understand what sort of motivation is required, how to discern it in candidates, and follow through with this criteria in preference to the criteria they’ve been using.

And what would you do with the people you have now?  Those that are presumably not motivated toward technical excellence?  Fire them all and replace them?  This is the Human Resources equivalent to the total rewrite of a legacy application, and it has some of the same problems.  There is a lot of domain knowledge hidden in there–knowledge that would be difficult to find written down anywhere.  The new must achieve the competency of the old before it can begin to surpass it–until then, no benefit is seen.  The new will be starting in the same context as the old, and therefore is likely to produce something with a similar organization (or lack thereof).

The bottom line is that the developers have to learn to recognize what is “better,” they have to learn how to do it, and the cultural context in which they work has to change such that it really is important enough to have happen.  Depending on where you start, that’s a lot to change all at once.  It can take a lot of time and energy.  And leadership.  And an understanding of people.  And technical knowledge.  And…

Serial lifecycles provide a (false) prediction. And, boy oh boy, is that prediction comforting to your senior managers. “When will the project be done?” might be their most-asked question. Of course, a serial lifecycle provides a prediction that’s almost guaranteed to be wrong, especially if you use a project scheduling tool. The tool provides you a single-point estimate, which is the first date you can’t guarantee the project won’t be done by–the first possible, optimistic date.

I like that characterization of the predicted date.  Another characterization, usually true of serial lifeycles, is that the predicted delivery date is the first day of schedule slip.  I’ve seen many serial projects get almost to that date before they first admit that they’re in trouble.

There was a comment from Pawel Brodzinski to Johanna’s post saying,

About predictability I’d say it doesn’t have to be false. Waterfall project which was properly planned and is properly managed should end around deadline and should deliver functionalities specified in requirements.

He’s right.  It doesn’t have to be false.  It usually is, however, because a serial lifecycle doesn’t give you any tools to manage schedule risk.  It only gives a prediction.  Experienced managers will guess the amount of padding to add to the schedule to give themselves a chance of meeting it.  Cautious but less-experienced managers will add a lot of padding, to increase their chances.  Of course, a development team working to meet a highly padded schedule may be so over-confident that they slow down to match, and then still miss the schedule if something unexpected happens.

In Pawel’s own blog post, he quotes Steve McConnell who describes an XP software development project that failed to produce a salable product because Customer (a.k.a. Product Owner, in Scrum terms) didn’t have a vision for a salable product.  For some reason, probably ignorance about Agile Software Development methods, Steve lays this failure on the development team creating one iteration at a time.  He apparently missed the practice of release planning.  When he says they “did it by the book,” they obviously didn’t use the book Planning Extreme Programming, which was published only a year after the first edition of Extreme Programming Explained: Embrace Change. Surely in two years the team could have read a second book.

Back to the point about Agility and Predictability, Steve says,

True agility–which means adopting a posture that allows you to respond rapidly to changing market conditions and customer demands–conflicts with predictability. Some businesses value agility, but many businesses value predictability more than they value the ability to change direction quickly. For those businesses, becoming more Agile is a second level consideration; the first level consideration is how to become more predictable.

Here, Steve seems to make the common mistake of using the dictionary definition of “agility” instead of checking the values and principles of the Agile Manifesto.  The “Agile” of the manifesto is a brand name, of sorts, to replace the wimpy “light-weight methodology” term that was starting to be commonly used at the time.  It’s a good brand because Agile does allow you to change directions more easily.  But Agile doesn’t require you to change directions and it certainly does expect you to have a direction.

How does Agile give you a predictable schedule? By giving you early, frequent, and fine-grained feedback and allowing you to steer the project to that schedule.

Let’s take a look at the release plan created at the start of the project.  First, write story cards for all of the features you want in your product.  These cards will generally be big stories, a.k.a. epics, and you’ll want to break them into smaller stories for development.  The big stories will work just fine for release planning, though.  Estimate the size of these cards.  I prefer estimating in story points, a fictional, unitless measure.  Some people like to start with ideal engineering days.  Either way, you have to guess how many of these your devlopment team can accomplish in a short iteration.  (I recommend one or two week iteration lengths.)  Count up the story points, divide by this guess, and you’ve got the number of iterations it will take to create the software.  This plan is exactly the same as the plan created by a serial lifecycle.  It’s just a guess–one made as carefully as our experience will allow–but still a guess.

Now develop software for one iteration.  How many story points did you accomplish?  Was this different than your guess?  If so, you may want to adjust your prediction to use this value instead of the guess.  Sometimes people may want to wait for three iterations before doing so, to let the team get up to speed, but they’re put on notice at the end of the very first iteration.  That makes a pretty good prediction, don’t you think?  And you can refine your prediction at the end of each iteration.  With a serial lifecycle, you’re either stuck with your original guess, or you get nervous enough that you make a new guess.

If wishes were horses, even beggars would ride. My mother used to tell me that when I wanted something that was clearly out of reach.  Sometimes a software development team just can’t accomplish all of our wishes by the desired date.  We’ll see this as we replan our schedule at the end of each iteration.  And, unless the desires and capabilities are completely out of whack, Agile Software Development gives us the tools to meet the desired date anyway.

How can we meet the date when there’s too much to do?  We do the most important things first.  And we do things at a fine-grained level.  Instead of leaving whole features to the end, we leave “nice to have” refinements of features to the end.  And we then trim scope to fit the date.

We can do this if we’re getting each little story done-done as we go.  This includes testing, so we know it works the way we want.  And it includes automated regression tests, so we know it continues to work without getting buried by re-testing.  A serial lifecycle leaves this important feedback to the very end, when there’s little time to remedy problems that are discovered.  That late discovery leads to the lack of predictability produced by a serial lifecycle.  Agile Software Development gives you more predictability than those carefully planned-up-front project plans can provide.

Oh, and yes, it can provide the dictionary definition of agility, too, should your goals change.