When something persists, some reward exists
Jason Gorman has just written a piece in defense of Software Craftsmanship that highlights how very dependent our world has become on software. He offers Gorman’s Law Of Software-Dependent Business Evolution:
Software-dependent businesses can only evolve as fast as their ability to write and evolve their software allows them to.
I think this is not only true, but an incredible opportunity for businesses that understand that. Let’s face it: most businesses spend an awful lot of time for a very meager increase in systems capability. Companies that do better than average can shoot to the top. Look at the spectacular successes of some of the relatively young internet companies, for examples.
As Corey Haines points out, we, as an industry, know how to produce relatively bug-free software in a steady stream, delivering needed functionality early and often to businesses starved for capability. But, for the most part, the IT industry doesn’t do so.
Jason’s proposals for addressing this gap fall short, in my opinion. The first, that getting more kids to choose software development as a career, seems odd to me. I don’t think we’ve got a lack of people. I think we’ve got a lack of people who know and practice the techniques that we, as an industry, have found to work. The second, getting more practioners to want to do better, addresses this lack. I’m all in favor of this, but I don’t think it, alone, will make much of a dent in the overall situation.
Even if we’ve got excellent software craftsmen, if the quality isn’t demonstrably valued by their organization, they’re most likely to either get demoralized and slack off their efforts, or leave that organization for greener pastures. So, while I do think that most software developers need to improve their skills (and continue to do so over their entire career), I don’t think that will drive improvement in those organizations that need it most.
Organizations, like most systems composed of people, tend to develop amazingly stable patterns of operation. These patterns persist through reorganizations and changes in personnel. The reason they persist, is that there is an informal network of effects that regulate the system, putting and keeping it in its current state. This stability doesn’t happen without forces reinforcing the behavioral patterns and inhibiting other behavioral patterns.
The study of these patterns and the forces that reinforce them is called Systems Thinking. You can read more about it in Peter Senge’s The Fifth Discipline, Jerry Weinberg’s Quality Software Management: Systems Thinking, or An Introduction to General Systems Thinking, among others.
A fundamental lesson I’ve learned from Systems Thinking is that when some behavior persists over time, there’s something that’s rewarding it in some fashion (perhaps unintentionally). If the behavior doesn’t make sense to me, or seems counterproductive, then I often find it more productive to look for what that reward might be rather than plead harder for a different behavior.
In this case, I think that large organizations operate in ways that support doing a slapdash job of software development. I’ve seen many organizations that primarily reward behavior that’s irrelevant to helping them evolve more rapidly. That and simple neglect of the behavior that would aid being more nimble in the marketplace are enough to produce the stability of the systems I see.
Many of these instances of rewarding irrelevant behavior come down to focusing on costs (which are easy to quantify) at the expense of value (which is much harder to quantify). To promote the development of higher quality, more valuable software, I think we need to make that value more visible to the decision-makers. They’re not stupid, after all. They’re just working with the information they have.
What other drivers do you see behind the vast sea of mediocre software production?
A profoundly important topic, rarely considered – let alone addressed – by the majority of software practitioners (nor by managers and executives either, for that matter).
Gorman’s Law Of Software-Dependent Business Evolution seems to me at least three “Why’s” away from the root cause of the issue. I agree about the incredible opportunities for software-dependent businesses that can increase their effectiveness significantly beyond the norm. “Rightshifting” (my work) illustrates just how much scope for improvement there is “to the right” of the median.
And The Marshall Model of Organisational Evolution explains the root cause of just why so few organisations manage (sic) to achieve high levels of effectiveness: It’s down to mindset.
I agree with your observation that there’s something rewarding organisations (specifically, their core groups cf Art Kleiner) such that they stick with their ineffective, dysfunctional, analytic mindsets a.k.a. their status quo. I suggest that the reward in question is preservation of self-image (membership of the Analytic management/executive/MBA peer-group) along with a reduction in both individual and collective cognitive dissonance.
The well-researched “The Knowing-Doing Gap” by Pfeffer and Sutton (amongst other sources) illustrates this at length.
HTH
– Bob (@FlowchainSensei)
I think you make excellent, excellent points.
If you take “when something persists, some reward exists” up a level from the organization, what is the answer?
As you say, the decision-makers at these large organizations aren’t stupid.
What reward are those decision-makers, and the organization as whole, receiving for doing a slapdash job of software development? Who are they receiving it from, and why?
How can we change it so that we stop rewarding organizations for producing low quality software?
It’s easy to agree with George and Bob on this subject. What’s missing from the post, perhaps, is a starting point for learning how to discover just what the unintended rewards (or stabilizing forces in the organization) actually are. I’ve found William Dettmer’s “Logical Thinking Tools” to be useful for this, as well as the Diagram of Effects.
George briefly touches on the matter of measuring cost vs. value. Cost Accounting is the norm in most businesses. Theory of Constraints offers an alternative called Throughput Accounting that can expose process wastes that are concealed (and maybe even reinforced) by Cost Accounting. This could be another useful tool for discovering the root causes of apparent problems.
When Bob writes Gorman’s Law is at least three Whys away from the root cause, it reminds me of the fact that we in the “agile” community tend to focus on software development activities, since that’s what most of us do for a living, but software development plays a secondary role in most end-to-end value delivery processes in business. I would caution against starting an improvement program by streamlining the software development area, since that is unlikely to be the “weakest link” in the chain, even if it appears less than optimal to an “agile” development practitioner.
Improving any link besides the weakest one will have no effect on the overall process. I would observe that in many of the cases when people say they “tried agile and it didn’t work,” it actually /did/ work at the level of the software development teams that adopted it, but there were no corresponding organizational changes, and ultimately the “agile” teams merely operated against the organizational grain and were crushed.
If an organization encourages “slap-dash” software development practices, as George puts it, it /may/ be because variances in software quality are swallowed up in the general background noise of a horrible overall delivery process.
I think this is tied to the continued practice of trying to create systems by focusing on functional requirements rather than how the system will prove value and make the customer happy.
It’s hard to care about quality of production when you have know idea whether your work is actually going to make someone happy. Or worse when you do deliver high-quality and still your product crash upon the rocks of requirement.
Have you looked for a correlation between having a good grasp on customer happiness and willingness to produce high-quality code?
My personal experience is (just as Dave Nicolette describes) that software development teams (in companies where SD is not at the “core” of the business) usually get “lost” in the noise of a horrible overall delivery process. In this situation agile is used as a means for the team to isolate itself from the “irrelevant behavior” that’s all around them by focusing on delivering value. Unfortunately this usually only works for a while and in the end the team is washed away/crushed.