Software Value: Impact on Software Process Improvement

Business value has not always been the primary driver of software process improvement, but that is changing. This is the main point of an excellent article by Richard Turner in the March/April edition of CrossTalk, “The Impact of Agile and Lean on Process Improvement.”

Turner’s article is a concise and refreshingly frank walk through the history of software process improvement from the perspective of an expert who has been intimately involved. With a hint of frustration that I certainly share, Turner captures perfectly the thinking that has led to a move away from process improvement initiatives like CMMi in commercial software development organizations:

“One of the drawbacks of earlier process improvement approaches was the concept and distribution of value. The overall value of the process improvement was often situational at best and nebulous at worst. Where it was seen as a necessity for competitive credibility [as was the case for my development group at Sanchez Computer Associates back in 2001], the value was in passing the audit rather than in any value to the organization and the customer. In other cases, the value was essentially associated with the success of one or two champions and disappeared if they failed, changed positions or left the company [as I did]. On those occasions where PI was primarily instituted for the actual improvement of the organization, the internal focus on practices was often valued as a way of cutting costs, standardizing work [We certainly needed to make our processes repeatable] or deploying better predictive management capabilities rather than improving the product or raising customer satisfaction.”

While I agree with 95% of Turner’s analysis here, in my experience both passing the audit and standardizing our processes raised customer satisfaction. We went from having one customer ready to give us a reference to most of our customers being referenceable on the basis of solid evidence that we had fixed the reliability of our software development.

Turner contrasts historic process improvement initiatives, mostly targeted at waterfall operations, where business value was not a prime driver to today’s initiatives where, “With the emergence of Agile and Lean, the concept of value became more aligned with outcomes. The focus on value stream and value-based decision making and scheduling brought additional considerations to what were considered best practices.”

Turner recognizes that in today’s Agile and Lean software development teams, the teams themselves are responsible for their own processes. Mostly, this is a strength because creative people are likely to optimize processes under their control out of simple self-interest (which benefits the organization). Where this falls down in my experience is where, “These organizations rely on cross-fertilization of personnel across multiple projects to improve the organization as a whole.” To put it bluntly, this rarely happens. Teams can be self-organizing but groups of teams don’t typically self-organize. Hence, there is still a place for organizational process improvement – with a lean, software value driven emphasis – in the most modern software development organization. By way of evidence, scrum teams that are working together on the same program struggle to develop ways to coordinate and synchronize their efforts unless a framework such as SAFe is introduced through a process improvement initiative.

That said, I will leave the last word to Turner, “Process improvement that does not improve the ability to adapt has little value.”