With any kind of automatic testing you get the overall results where all these hints disappear in loads of data.

Another thing is automatic testing always check exactly the same path while human will most likely do things a bit differently because of curiosity, distraction or fatigue.

On one hand this gives you broader range of tests but on the other doesn’t assure you have gone exactly through the predefined list of scenarios.

That’s why both exploratory and automatic testing fit each other well.

And one more comment on preparing test cases – the biggest value of test cases is not withing executing but within preparing them. That’s the place when all the creative thinking happens and you catch a number of issues.

This is done with cognitive engagement. It’s just not done in “real time.” The cognitive engagement happens when the decision rule is written.

Yeah, well, there’s another part that either you didn’t read or that you’ve deliberately ignored: the whole paragraph that precedes it. Here THAT is again:

“It takes sapience to recognize the need for a check—a risk, or a potential vulnerability. It takes sapience—testing skill—to express that need in terms of a test idea. It takes sapience—more test design skill—to express that test idea in terms of a question that we could ask about the program. Sapience—in terms of testing skill, and probably some programming skill—is needed to frame that question as a yes-or-no, true-or-false, pass-or-fail question. Sapience, in the form of programming skill, is required to turn that question into executable code that can implement the check (or, far more expensively and with less value, into a test script for execution by a non-sapient human). We need sapience—testing skill again—to identify an event or condition that would trigger some agency to perform the check. We need sapience—programming skill again—to encode that trigger into executable code so that the process can be automated.”

That is of the essence, George, of saying that the cognitive engagement happens when the rule is written. It’s at the moment that the check is performed, by a machine, that sapience is absent, by definition.

I understand you’re really, really enthusiastic about checking, so here’s more: the issue about “which we are going to value: the checks (‘we have 50,000 automated tests’) or the checking. Mere checks aren’t important; but checking—the activity required to build, maintain, and analyze the checks—is. To paraphrase Eisenhower, with respect to checking, the checks are nothing; the checking is everything. Yet the checking isn’t everything; neither is the testing. They’re both important, and to me, neither can be appropriately preceded with ‘mere’, or ‘merely’.”

Beware: the next step is for me to visit your home and read it to you.

Can we agree, please, that manual scripted checking is an expensive waste of time, and that (yet another quote from that same blog post) automated “checks can be hugely important?”

—Michael B.

WRT the charge of not reading all of your post: I noted that you didn’t mention these as activities of the tester. I don’t think the tester’s job properly ends with reporting and analysis.

WRT the decision rule: This is done with cognitive engagement. It’s just not done in “real time.” The cognitive engagement happens when the decision rule is written.

My point is that exploratory testing and scripted testing are both doing the same sorts of things, but in different orders and with different timings. When you say scripted testing is just “checking,” you’re leaving out part of the picture. Part of exploratory testing is just checking, also.

I understand that you’re trying to combat some practices that are chronically ineffective and low-value, but this distinction you’re making is, in my opinion, the wrong one. And I think it makes some people dismiss other things that you say–things I’d rather they didn’t dismiss.

I’ve been explicit that checks themselves do not require sapience, but that the preparation and analysis of them do. That’s not an “admission”.

What he doesn’t mention is the action that might be taken in response to the results.

How far did you read in the very blog post that you cite? In that post, I said (and I’m quoting): “Sapient activity by someone—a tester, a programmer, or a product owner—is needed to determine the response. Upon deciding on significance, more sapient action is required—fixing the application being checked (by the production programmer); fixing or updating the check (by the person who designed or programmed the check); adding a new check (by whomever might want to do so) or getting rid of the check (by one or more people who matter, and who have decided that the check is no longer relevant).” I therefore object to the suggestion that I didn’t mention that action might be taken in response to the results. I mentioned it, clearly and explicitly.

Then you go on:

It’s that ‘exercise and observe’ step (and making a decision as to whether the observation is cause for concern or not) that Michael calls a ‘check.’

The main sentence is correct, but the parenthetical is something that you just made up. Again, read the blog post that you cite. It says exactly this:

—Michael B.