Comments on: The Reality of Automated Acceptance Testing http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/ Effective software development Wed, 08 Feb 2012 21:01:40 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Money quotes from the Shore acceptance-testing discussion « Matt Philip's blog http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-117885 Money quotes from the Shore acceptance-testing discussion « Matt Philip's blog Thu, 20 Jan 2011 21:42:52 +0000 http://blog.gdinwiddie.com/?p=353#comment-117885 [...] – George Dinwiddie [...] [...] – George Dinwiddie [...]

]]> By: George Dinwiddie http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-104701 George Dinwiddie Sun, 11 Apr 2010 00:39:12 +0000 http://blog.gdinwiddie.com/?p=353#comment-104701 Pete, It may be important to consider that most orgs that Jim Shore works with are still in the throes of Agile adoption. Automated end-to-end regression testing is not generally the first thing I try to instill, either. Pete, It may be important to consider that most orgs that Jim Shore works with are still in the throes of Agile adoption. Automated end-to-end regression testing is not generally the first thing I try to instill, either.

]]> By: Pete Tansey http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-104699 Pete Tansey Sun, 11 Apr 2010 00:27:07 +0000 http://blog.gdinwiddie.com/?p=353#comment-104699 I gather from the links that James Shore is simply reflecting on the value of automating acceptance tests, not automated testing. He values the business collaboration associated with identifying the concrete examples - not the automation beyond a unit level. But surely there is still business value in being able to provide some sort of automated end to end regression acceptance testing? I gather from the links that James Shore is simply reflecting on the value of automating acceptance tests, not automated testing.
He values the business collaboration associated with identifying the concrete examples – not the automation beyond a unit level.
But surely there is still business value in being able to provide some sort of automated end to end regression acceptance testing?

]]> By: Questioning Automated Acceptance Testing « The Art of Software Development http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-102885 Questioning Automated Acceptance Testing « The Art of Software Development Wed, 03 Mar 2010 10:13:29 +0000 http://blog.gdinwiddie.com/?p=353#comment-102885 [...] George Dinwiddie thinks Shore is throwing out the baby with the bathwater and that the problems are fixable. [...] [...] George Dinwiddie thinks Shore is throwing out the baby with the bathwater and that the problems are fixable. [...]

]]> By: Есть ли смысл в приемочном тестировании? « XP Injection http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-102882 Есть ли смысл в приемочном тестировании? « XP Injection Wed, 03 Mar 2010 08:16:20 +0000 http://blog.gdinwiddie.com/?p=353#comment-102882 [...] споров по поводу проблем приемочного тестирования и нужно ли приемочное тестирование вообще, стоит ли его автоматизировать или есть [...] [...] споров по поводу проблем приемочного тестирования и нужно ли приемочное тестирование вообще, стоит ли его автоматизировать или есть [...]

]]> By: Adam Knight http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-102813 Adam Knight Mon, 01 Mar 2010 23:10:59 +0000 http://blog.gdinwiddie.com/?p=353#comment-102813 I cannot see anything in either of the two objections stated above that cannot be overcome with sufficiently skilled testers and the appropriate level of communication. "Typically, responsibility for the tests gets handed off to testers, which defeats the whole point" - I would question why this defeats the whole point. In more traditional development processes an analyst would be present who would be skilled in converting the requirements in the language of the business into the appropriate models and requirements to drive the development of the system. If a tester possesses the necessary analytical skills then why not rely on them to create the appropriate tests based on the requirements communicated from the cusomer. To put it another way - if your analyst could create a specification that tested itself you would be very happy, so why not furnish the tester with analytical skills to allow them to achieve the same? In terms of the maintenance costs, I have been running with an in-house developed automated test harness for 3 years now. There is an overhead on maintenance but with careful consideration towards maintaining the tests separately from the executable code we have managed to overcome some significant rework of application functionality without having to rework large numbers of tests. Again this does require the necessary skills within the test team to treat the regression harnesses as applications in their own right, and design them with maintenance in mind. I would reiterate Lisa's point that it’s an evolution, and there are obstacles to overcome, many of which we are stil tackling. I do not see, however, any of the problems as sufficient reason to give up on automated acceptance testing, which are the one of the cornerstones of our current development process. --------------------------------------------------------- http://www.linkedin.com/in/adampknight http://a-sisyphean-task.blogspot.com http://www.twitter.com/adampknight I cannot see anything in either of the two objections stated above that cannot be overcome with sufficiently skilled testers and the appropriate level of communication.

“Typically, responsibility for the tests gets handed off to testers, which defeats the whole point” – I would question why this defeats the whole point. In more traditional development processes an analyst would be present who would be skilled in converting the requirements in the language of the business into the appropriate models and requirements to drive the development of the system. If a tester possesses the necessary analytical skills then why not rely on them to create the appropriate tests based on the requirements communicated from the cusomer. To put it another way – if your analyst could create a specification that tested itself you would be very happy, so why not furnish the tester with analytical skills to allow them to achieve the same?

In terms of the maintenance costs, I have been running with an in-house developed automated test harness for 3 years now. There is an overhead on maintenance but with careful consideration towards maintaining the tests separately from the executable code we have managed to overcome some significant rework of application functionality without having to rework large numbers of tests. Again this does require the necessary skills within the test team to treat the regression harnesses as applications in their own right, and design them with maintenance in mind.

I would reiterate Lisa’s point that it’s an evolution, and there are obstacles to overcome, many of which we are stil tackling. I do not see, however, any of the problems as sufficient reason to give up on automated acceptance testing, which are the one of the cornerstones of our current development process.

———————————————————
http://www.linkedin.com/in/adampknight
http://a-sisyphean-task.blogspot.com
http://www.twitter.com/adampknight

]]> By: Patrick Wilson-Welsh http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-102805 Patrick Wilson-Welsh Mon, 01 Mar 2010 21:51:55 +0000 http://blog.gdinwiddie.com/?p=353#comment-102805 Thank, you George! All the more pressing now, really, that those of us who wish to rescue this practice get together and attempt to harvest best practices for launching it for a team. I hope that in Nashville we can come closer to speaking about it with one voice. I have seen it work wonders. How sad that powerful thoughtleaders are publicly abandoning it. -Patrick Thank, you George! All the more pressing now, really, that those of us who wish to rescue this practice get together and attempt to harvest best practices for launching it for a team. I hope that in Nashville we can come closer to speaking about it with one voice. I have seen it work wonders. How sad that powerful thoughtleaders are publicly abandoning it.

-Patrick

]]> By: George Dinwiddie http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-102804 George Dinwiddie Mon, 01 Mar 2010 21:43:44 +0000 http://blog.gdinwiddie.com/?p=353#comment-102804 Thanks to Markus Gärtner (http://blog.shino.de/) for finding the Jerry Weinberg quote. Thanks to Markus Gärtner (http://blog.shino.de/) for finding the Jerry Weinberg quote.

]]> By: Paul Boos http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-102803 Paul Boos Mon, 01 Mar 2010 20:51:50 +0000 http://blog.gdinwiddie.com/?p=353#comment-102803 My thoughts after reading yours, Ron's and Gojko's responses as well as Jim Shore's original post is could the experiment perhaps for him to try would be to use pair test creation (much like an XP pair programming)? Have the customer and the knowledgeable tester work through the test as a pair. The customer has teh trust in what was written, but didn't actually do the "hard" part in their mind. As someone who is trying to get a team to strive towards automating their acceptance tests, seems lik increasing the conversation around concrete examples is the way to go. My thoughts after reading yours, Ron’s and Gojko’s responses as well as Jim Shore’s original post is could the experiment perhaps for him to try would be to use pair test creation (much like an XP pair programming)? Have the customer and the knowledgeable tester work through the test as a pair. The customer has teh trust in what was written, but didn’t actually do the “hard” part in their mind.

As someone who is trying to get a team to strive towards automating their acceptance tests, seems lik increasing the conversation around concrete examples is the way to go.

]]> By: Money quotes from the Shore acceptance-testing discussion « Excellentist http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-102802 Money quotes from the Shore acceptance-testing discussion « Excellentist Mon, 01 Mar 2010 20:50:51 +0000 http://blog.gdinwiddie.com/?p=353#comment-102802 [...] – George Dinwiddie [...] [...] – George Dinwiddie [...]

]]> By: Lisa Crispin http://blog.gdinwiddie.com/2010/03/01/the-reality-of-automated-acceptance-testing/comment-page-1/#comment-102799 Lisa Crispin Mon, 01 Mar 2010 20:26:55 +0000 http://blog.gdinwiddie.com/?p=353#comment-102799 My sentiments exactly! I wonder if people just get impatient. It takes months and even years of experimenting with different approaches, learning how to help customers illustrate desired behavior with examples and turning those into tests, knowing how much detail to get into with tests before coding starts... it's an evolution. No magic happens overnight. My team's been doing this 6+ years and we still are refining, every sprint. My sentiments exactly!

I wonder if people just get impatient. It takes months and even years of experimenting with different approaches, learning how to help customers illustrate desired behavior with examples and turning those into tests, knowing how much detail to get into with tests before coding starts… it’s an evolution. No magic happens overnight.

My team’s been doing this 6+ years and we still are refining, every sprint.

]]>