One of the things that has been extremely popular in software organizations for a long time is to count the number of defects that have been found via testing. I propose that teams no longer do this for two reasons.

First, one of the lines of thinking behind counting defects is that it is somehow a measure of quality. If we were running an assembly line in a manufacturing plant, and the assembly line didn’t change from one week to the next, but the number of defects increased, we would know there was a quality problem somewhere – something on the assembly line was likely broken. Despite some similarities, software development is not an assembly line – unlike an assembly line, every line of code that is written and tested has never been written or tested before, so expecting defect counts to be an indicator of quality doesn’t correlate. If a team found 25 defects last week, and 50 this week, what does that tell us? Is the code quality really getting worse, or did the team just do a lot more testing? If so, did the tests executed reflect real-world usage, or were the tests all edge-cases? Did the team deploy the code into a new environment that had never been used before? I’m sure you can come up with many more scenarios… Anyway, the possible reasons for the higher number of defects this week are almost boundless, and the time spent trying to determine why the variation occurred would be better used to: work more with customers to better understand their usage patterns and particular needs; create more needed functionality; improve their processes further; cross-train team members; adopt a new tool; etc., etc., etc. All of which leads to my second point…

Agile software development has its foundations in Lean Thinking, and one of the Lean principles is to eliminate waste. Does counting defects, and spending time trying to figure out what causes variations in the numbers of defects from one time period to the next, contribute directly to the success of the project? If not, then the time spent doing so should be viewed as a waste – time that could be better spent doing more important work.

In conclusion I’d like to leave you with two thoughts regarding defects: first, when defects are found, they should be fixed immediately – period. Don’t allow a backlog of defects to accrue. If it does make sense to do some root-cause analysis to determine why a particular defect occurred, then that’s fine – do so when it makes sense to do so, but it likely does NOT make sense to do so for the vast majority of the defects found. And the second point is that a mature, Agile team should be able to have “difficult” conversations when needed: “Hey Scott, I’ve noticed that I’m finding a lot of defects in your code this week – is there anything that’s distracting you, or anything I can help you with?” Instead of taking umbrage at such comments, I should be thankful that my team is willing to raise the issue and have the discussion.

So, the next time you’re asked to track the number of defects found, ask “Why…?” The answer will likely shed light on ways to eliminate waste and help overcome the “That’s the way we’ve always done it!” mentality, as well as foster a relentless, continuous improvement mindset.

As always, thoughts, comments, and questions are most welcome! Thank you!

When teams complain that they “do not have enough people”, that may be a sign the team composition has not resulted in an agile whole team. Agile’s mandate for working software requires that teams be composed of the right set of people to deliver on this goal. That generally means developers, testers, writers, user experience experts and so forth. That also means that there are team members from each of the component technology disciplines so that, as a team, they can deliver end-to-end functionality.

Large teams moving to agile often have to form several smaller agile teams with each team delivering completed user stories. Teams that have traditionally been organized by technology or by discipline, are reluctant to move to a cross functional structure because there may not be enough technical and domain knowledge in each of the resulting teams. What they discover quickly is the majority of the knowledge and skills in the team resides with very few people.

The solution to this problem is to cross train the team members so that they are better able to deliver product capability with the right set of skills and domain knowledge. Many teams resist this solution because it requires upfront costs and will slow the team down for a while. In fact many teams are so convinced this will not work that they do not make any effort to cross train and end up with teams that cannot deliver working software. This is what leads them to believe that they “don’t have enough people”.

It is a very hard sell but cross training your teams so that they can deliver a greater range of product functionality is well worth the upfront costs. Software engineers usually have a product area in which they prefer to work or a technology that they favor. But good software engineers have the skills to learn new technologies as required. And it benefits a team when engineers do expand their skills because not only are they able to help complete additional work, but their expertise and perspectives are beneficial when building new software and solving problems.

It is important to let management know there will be a short-term slow down (maybe as much as several months) when cross training team members. But when teams are willing to cross learn in a results oriented way, they become more productive and tend to move faster.

There are ways to cross train and continue to move forward at the same time. Pair programming is a great way to introduce a developer to a new technology or a new area of the code. Developers can also try to fix defects in an area outside of their domain, and then request a code review of the completed fix before it gets checked in. Figuring out how to fix a problem really forces one to learn about how the code works. Engineers can also test and write test code outside their current domains so that they learn additional areas of the product. There are no doubt many other ways you can come up with and we would love to hear your ideas. But teams that are not willing to take this step will struggle to make agile stick, and will continue to complain that they “do not have enough people”.