Design

Do You Inspect?

By Capers Jones and Olivier Bonsignour, November 30, 2011

Two software quality experts argue that inspections are the defect-prevention technique of choice

The Efficiency Factor

Our research shows that most forms of testing are less than 30% efficient in finding errors and bugs. Formal inspections, on the other hand, have a defect-removal efficiency rate that ranges from just less than 50% to more than 85% and an average efficiency level of roughly 65%. Static analysis tools have about the same level of defect-removal efficiency, but they tackle only a narrow band of code-related defects.

Gilb says that some inspection efficiency rates have been as high as 90%. Authors Lew Priven and Roger Stewart report inspection defect-removal rates topping 95%. With the possible exception of high-volume beta testing involving more than 10,000 simultaneous test sites, testing never approaches this level of efficiency.

A combination of formal inspections, static analysis, formal testing by test specialists, and a formal (and active) quality-assurance group are the methods most often associated with projects achieving a cumulative defect-removal efficiency higher than 99%.

How Inspections Are Conducted

Formal inspections are manual activities in which four to six colleagues go over requirement and design specifications page by page using a specific protocol. Code inspections follow the same approach but go over listings or screens line by line. To qualify as an inspection, certain criteria must be met, including:

Appointing a moderator to keep sessions moving;

Having a recorder take notes;

Having participants adequately pre­pare for each session;

Keeping records of discovered defects;

Not using defect data for ap­prais­als or punitive purposes.

The original concept of inspections was based on participants actually getting together. Online communications and collaborative Web tools now mean that inspections can be performed electronically, saving travel costs for teams that are geographically dispersed.

Thanks to the flexibility of the human mind and its ability to handle inductive logic as well as deductive logic, inspections are the most versatile form of defect removal and can be applied to essentially any software artifact. Indeed, inspections have even been applied recursively to themselves to fine-tune the inspection process and eliminate bottlenecks and obstacles.
The Development Process
One of the best ways of showing the effectiveness of inspections is to produce graphs that connect the point in time when software defects are discovered with the point where the defects originate (see Figure 1). An acute angle in the line connecting defect discovery and origin points indicates a serious software quality-control problem, because the gap between making an error and discovering it can be many months. The goal of defect removal is to have the angle connecting defect origins and discoveries approach 90 degrees, indicating the defect happens and is discovered at the same time. A 90-degree angle is unlikely, but formal inspections can at least bring the angle up from perhaps 30 degrees to more than 60 degrees (see Figure 1).

Figure 1.

Software projects that don't use formal inspections enter a "zone of chaos" during the test cycle (see Figure 2). Problems with requirements and specifications that are deep in the process suddenly emerge, and require extensive and expensive repair and rework.

Figure 2.

When you use formal inspections, the lines connecting the discovery points of defects with their origins have angles approaching 90 degrees. Defects that originate within a phase tend to be eliminated during that phase and don't pass to downstream phases.

These graphs provide a quick visual gauge of the delays in software defect discovery, and resolution. Wider use of this approach and of manual inspections would result in earlier discovery and correction of defects, leading to higher-quality software that's produced faster and more efficiently. That's a goal we should all support.

Capers Jones is an independent consultant and has written several books on software productivity management and software estimation. Olivier Bonsignour is VP of product development at CAST, which specializes in software testing. This article is adapted from their book, The Economics Of Software Quality, published by Addison-Wesley.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!