Be Prepared: Software Forensics Gaining Steam at FDA

July 1, 2008

FROM THE EDITORS

The software sleuths at FDA are no longer operating under cover. A recent article in the Baltimore Sun examined the issue. The reporter talked with FDA's Larry Kessler and Brian Fitzgerald about the agency's efforts to build up its forensic capabilities with the use of static analysis tools.

I normally avoid addressing a topic twice. I just talked about FDA's move in this direction in the February issue of MD&DI. But the forensic software lab at FDA seems to be getting some legs. “This approach is gaining a lot of momentum, and it is probably not going to be turned back,” says David Vogel, PhD, founder and president of ­Intertech Engineering Associates in Westwood, MA.

The problem for medical device companies­—especially small firms—is that implementing static analysis programs can be costly. However, it is clear that the expectation for quality assurance in software engineering is here to stay, so it's time to step up your game. Whether you should opt to implement your own static analysis tools or seek another method is open to debate.

“Good software engineering, which includes design controls, risk analysis, a quality system, and proper tools and techniques, is critical in reducing the likelihood of errors,” says Andrew Dallas, president of Full Spectrum Software (Southborough, MA).

Device companies must be increasingly vigilant about preventing software-coding errors. The question, however, is whether static analysis tools are the answer to eliminating the software-­coding errors.

“Static analysis tools have a place in software development engineering. But, they are not very productive, and I don't think FDA should be forcing their use on industry,” says Vogel.

Dallas strongly believes in the value of static and dynamic analysis tools, but he adds, “I would stress that devices are not designed to make decisions but rather they are designed to require input from trained professionals. For instance, before treatment can be delivered, confirmation is required by the operator that the dosage is correct.”

He points out that software development is often looked at as a necessary evil to get hardware products to market. But software development takes talented, disciplined engineers with the right tools and training to produce the quality of software demanded for sophisticated medical devices.

Dallas notes that finding software defects is very much like detective work. “Perhaps it's straightforward, but there are processes, protocols, and tools that we use to find defects. With multithreading, multiuser access, and real-time acquisition and control, defects can hide without discovery unless static and dynamic analysis tools are used.” Vogel disagrees. He says that static tools are hyped to do more than they can actually deliver. “Static analysis looks for simple coding errors and does not apply heuristics to understand how it will perform dynamically because it is a static analysis tool,” he says.

Industry is slow to adopt these tools because they are not great, explains Vogel, noting that they create more false-positive findings than true positives. “Industry's limited resources are spent on more tried-and-tested methods that don't waste resources documenting why false positives are false.” Unfortunately, the headline of the Sun article—no less than sensational journalism­—may amplify the problem. Its headline, “Flaws in Medical Coding Can Kill,” could create some unnecessary panic among consumers, and it seems the article was written to fulfill the headline's premise.

Referring to a dialysis machine being investigated, Fitzgerald told the Sun, “We declared the software innocent.” Such a statement implies that FDA is taking its forensics role very seriously.

“Regulation in this area is much more prescriptive than it was 20 years ago, and the results haven't improved,” says Vogel. “I don't think more regulation and forced use of static analysis tools will make things better.”

Perhaps these tools—and FDA's lab—are a solution in search of a problem. Nevertheless, the lab appears to be here to stay, and you need to be prepared for the fallout.