Testing in depth

In the late 1970s, in the Co-Evolution Quarterly, the magazine successor to The Whole Earth Catalog, Peter Warshall stated that geodesic dome houses always leak.  This was a bold and surprising statement at the time, coming from a man who was considered one of the finest builders of dome houses–ones that didn’t leak.

Why did he make this statement? (Continued)

More on Automated Acceptance Testing

Jim Shore has posted a response to the reactions about his previous post on Acceptance Testing in which he defends the way he and the teams he coaches are working.  About the same time, Lisa Crispin posted her thoughts on the topic.

As Lisa says,

I can’t tell you the one right way to test and develop software….  The one right way for your team to code and test will continually evolve over the years. In fact, I’m sorry to tell you that you’ll never find the one right way; it’s a moving target, but you’ll get closer and closer to it.

This is an incredibly important point!  There may be many “wrong” ways—wrong in that they fail to achieve your objectives—but there is no “right way.”  So I’m happy that Jim and his teams are able to achieve the results they want.  I’m not saying they’re doing it wrong. (Continued)

The Reality of Automated Acceptance Testing

Recently, Jim Shore wrote about The Problems With Acceptance Testing.  I like Jim, and respect him a lot.  Because of my respect for his opinions, I found it quite discouraging that he said, “I no longer use [automated acceptance testing] or recommend it.”  Gojko Adzic has posted his response to Jim.  This is mine.

Certainly when something’s not giving you the results you want, it’s time to make a change.  That change can be to drop the practice that’s not working for you.  It can also be changing the way you go about the practice, or changing what you want to accomplish.  Or, instead of changing, maybe the word “refining” is a better fit. (Continued)

3 Legs to Standing Up an Agile Project

Maybe you’re starting your first Agile project.  You’ve read books and blogs.  You’ve had training.  You think you’re ready, but it’s still a daunting prospect.  There’s just so much to remember—so many details.

Here’s a little cheat sheet.  Forget all the details and the various ways you can implement Agile for the moment.  Let’s simplify the picture.  There are just three essential legs that your Agile project needs to stand.  Get those in place, and you’ll do OK.  Keep improving all three, and you’ll do fantastically! (Continued)

The testers get behind at the end

It’s a very common complaint, such as this one left on the Scrumdevelopment yahoogroup:

Usually in different phases, workload for tester and dev is different. E.g. when a project is coming to the end, most of the tasks will be test.

There are a couple of big red flags waving at me in those two sentences.  One is “different phases.” Why should a software development project, especially one only a couple weeks to a couple months long, have phases?  The other is, at “the end, most of the tasks will be test.” Postponing testing to a phase at the end has never worked very well.  It always results in the testers being behind at the end.

Does this situation sound somewhat familiar to you?  If so, what can we do about it?

(Continued)

Testability & Good Design

Much of the time, the test-driven development yahoogroup is pretty quiet, but it has recently awakened from winter hibernation.  The question “Is it OK to add code to a class only to improve its testability?” stirred up a wide-ranging discussion that brought in the topic of what constitutes good design.  “Uncle Bob” Martin drew a bold line in the sand with his comment,

One reasonable definition of good design is testability.  It is hard to imagine a software system that is both testable and poorly designed.  It is also hard to imagine a software system that is well designed but also untestable.

I greatly sympathize with this statement, though I wouldn’t go quite that far.  I don’t think it is so hard to imagine code that is testable, but poorly designed.  For a trivial counter-case, there could be rampant duplication of testable code.  I would call that poorly designed, but it doesn’t affect it’s testability.  Therefore I would soften Uncle Bob’s definition to “One reasonable component of the definition of good design is testability.”

To me, the notion of “testable code” is the same thing that “testable circuit” was back when I worked on a custom integrated circuit.  Mostly, that depends on the ability to put the circuit or code into a known state, exercise it, and see the interactions with its collaborators and its resulting state. (Continued)

It’s not the script, it’s how you do it.

I’ve had numerous discussions with Michael Bolton where he makes the claim that scripted testing (whether via automation or a person following written directions) is not testing but checking.  He quotes Cem Kaner’s definition of testing: “testing is an empirical, technical investigation of a product, done on behalf of stakeholders, with the intention of revealing quality-related information of the kind that they seek.”  Running a script that validates certain desired behavior certainly fits this definition. (Continued)

Long term focus

My good friend, Steven M Smith, tweeted

AGILE methods emphasize realizing short-term OBJECTIVES rather than creating long-term objectives and trying to satisfy them.

I completely disagree with this statement.  I don’t agree that short terms objectives are the emphasis in Agile methods and I don’t agree that short term objectives are preferred over long-term objectives.  I think that Steve’s misunderstanding is rooted in the fact that he has only read about Agile methods, and not practiced them.  I think that it is impossible to get a deep understanding of Agile methods by reading without experiencing.  Therefore, I’d like to encourage Steve and others talking about Agile methods to try to get some experience before making such statements.  I’d also like to offer an explanation that attempts to clear up this particular misunderstanding. (Continued)

Questionable security

Adding security questions to back up password-protected web sites has become all the rage. I’ve had to deal with this on many web sites I use, and have encountered it on the back end helping teams implement sites. I’ve known security departments of financial organizations that say these are required by industry rules and not to have them would be a breach of responsibility.

The fact is, having “security questions” weakens security rather than strengthens it.  Recent news of the infiltration of Twitter’s proprietary documents is a good case in point.  The New York Times reports

Instead of circumventing security measures, it appears that the Twitter hacker managed to correctly answer the personal questions that Gmail asks of users to reset the password.

If you don’t believe me that security questions weaken security, just ask Bruce Schneier.

If you don’t automate acceptance tests?

Amr Elssamadisy reports on InfoQ that automated acceptance tests are “only used by a small minority of the community.”  Is this true?  If you and your team don’t use automated acceptance tests, please let me know how you handle regression tests as the application grows larger.  You can leave a comment here or, if you’d rather not say it in public, email me directly. (Continued)

Better Tag Cloud