Coverity Study Ranks LAMP Code Quality

The pillar of most open source applications is the ubiquitous LAMP -- Linux, Apache, MySQL, and Perl/PHP/Python stack. According to results of a Coverity study, the LAMP stack has fewer code defects than a baseline of 32 open source software projects.
So far, the government-sponsored study has found the open source stack to have a better defect density than the baseline.

The effort may well prove to help make open source more secure.

The Department of Homeland Security (DHS) announced the project, called "Vulnerability Discovery and Remediation Open Source Hardening Project," at the beginning of the year. It includes a grant from DHS of $1.24 million to be dispersed over three years to Coverity, Stanford University, and Symantec.

The first results of the Coverity scan report that the average defect density in LAMP code is currently 0.290 defects per thousand lines of code. A baseline scan run by Coverity of 32 popular open source projects, which also includes the five LAMP applications, found an average defect density of 0.434.

Previous Coverity studies (not backed by DHS) have found low defect densities in the Linux kernel and in the MySQL database.

Ben Chelf, Coverity's CTO, commented that he wasn't surprised with the defect distribution shown by the new analysis.

"The average defect density was probably about where we thought it would be," Chelf told internetnews.com.

Chelf was surprised, however, by the performance of the Perl language.

"Of the LAMP stack, Perl had the best defect density well passed standard deviation and better than the average, Chelf said.

Perl had a defect density of only 0.186. In comparison Python had a defect density of 0.372 and PHP was actually above both the baseline and LAMP averages at 0.474.

Other popular projects included in the 32 projects that made up the baseline are: Firefox (0.355), Gaim (0.352), Gnome (0.458), GCC (0.202) and Samba (0.695), among others.

Apparently the results also show that larger projects don't necessarily have more or fewer defects than small ones.

According to Chelf, "there wasn't a correlation between size of code base and defect density."

The goal of the DHS-backed effort isn't just to identify bug defects, but also is intended to help make open source applications more secure, as well.

"The thing about security vulnerabilities is that any defect in code can be a security vulnerability," Chelf explained. "It's all about how that defect is triggered."

A crash condition, for example, could be considered a vulnerability if a user has the ability to poke at it and trigger the crash.

Chelf also added that the scan detects specific defect classes that are tagged as security vulnerabilities. Those defects have to do with the way an application handles tainted data coming in from the outside, essentially making sure that an application does the "right thing" with data that is input via a keyboard or over a packet.

Identifying issues also isn't the only goal of the effort; according to Chelf the identified defects are actionable.

"Measuring code is good. Fixing code is better," Chelf said. "I think that's the big next step, working with the maintainers of these projects and saying 'hey this data is available and let's talk about how to fix the problems.'"