Testing in depth

In the late 1970s, in the Co-Evolution Quarterly, the magazine successor to The Whole Earth Catalog, Peter Warshall stated that geodesic dome houses always leak.  This was a bold and surprising statement at the time, coming from a man who was considered one of the finest builders of dome houses–ones that didn’t leak.

Why did he make this statement?

He went on to explain, that the design of a dome house depended on a single skin being perfect waterproofing technology.  Traditional houses, by comparison, have multiple imperfect layers.  There are overlapping shingles, which keep most of the water out.  Below that there’s a layer of tar paper, which keeps out most of what reaches it.  Then there’s the plywood sheathing, which blocks or absorbs most of what penetrates the tar paper.  Then the attic insulation….

No single layer of this system has to be perfect.

Software testing works the same way.  If you depend on one method of testing, you’re going to leak bugs into production.

If you do unit testing (or micro-testing, as Mike “GeePaw” Hill calls it) of small chunks of code at a low level, you’ll catch most of the coding mistakes.  Then, if you do small scale integration tests of the code that talks to other systems (such as the database), you’ll catch most of the remaining ones.  Then higher level integration or acceptance tests that check that the whole system is wired together right will catch most of those that have escaped so far.  Then exploratory testing….

Any competent network security professional will tell you that you can’t protect your network from malicious people by relying on perimeter security.  You have to bake security into other levels of access, too.

Again, the same is true of testing to protect your application from bugs.

Post to Twitter Post to Plurk Post to Yahoo Buzz Post to Delicious Post to Digg Post to Facebook Post to MySpace Post to Ping.fm Post to Reddit Post to StumbleUpon

Comments (3) to “Testing in depth”

  1. Terrific post, George!

    It sings to me.

  2. It sings to me too, but…

    …the preamble is that some folks never automate testing, and some only write developer tests. So the point is we need TDD, then test-last tests, THEN storytests, all to provide that “layering”.

  3. […] piece of code, but can’t assure us that the pieces are connected together properly. We need multiple levels of testing to get the assurance we […]

Post a Comment
*Required
*Required (Never published)