Buggy security software can make you less secure. Indeed, a growing number of intruders are exploiting bugs in security software to gain access to systems. Smart system administrators have known for a long time to be careful about deploying new “security” products.

A company called Audible Magic is trying to sell “content filtering” systems to universities and companies. The company’s CopySense product is a computer that sits at the boundary between an organization’s internal network and the Internet. CopySense watches the network traffic going by, and tries to detect P2P transfers that involve infringing content, in order to log them or block them. It’s not clear how accurate the system’s classifiers are, as Audible Magic does not allow independent evaluation. The company claims that CopySense improves security, by blocking dangerous P2P traffic.

It seems just as likely that CopySense makes enterprise networks less secure. CopySense boxes run general-purpose operating systems, so they are prone to security bugs that could allow an outsider to seize control of them. And a compromised CopySense system would be very bad news, an ideal listening post for the intruder, positioned to watch all incoming and outgoing network traffic.

How vulnerable is CopySense? We have no way of knowing, since Audible Magic doesn’t allow independent evaluation of the product. You have to sign an NDA to get access to a CopySense box.

This in itself should be cause for suspicion. Hard experience shows that companies that are secretive about the design of their security technology tend to have weaker systems than companies that are more open. If I were an enterprise network administrator, I wouldn’t trust a secret design like CopySense.

Audible Magic could remedy this problem and show confidence in their design by lifting their restrictive NDA requirements, allowing independent evaluation of their product and open discussion of its level of security. They could do this tomorrow. Until they do, their product should be considered risky.

The always interesting James Grimmelmann has a new paper, Regulation by Software (.pdf), on how software relates to law. He starts by dissecting Lessig’s “code is law” argument. Lessig argues that code is a form of “architecture” – part of the environment in which we live. And we know that the shape of our living environment regulates behavior, in the sense that we would behave differently if our environment were different.

Orin Kerr at Volokh wrote about Grimmelmann’s paper, leading to a vigorous discussion. Commenters, including Dan Simon, argued that if all designed objects regulate, then the observation that software regulates in the same way isn’t very useful. If toothpicks regulate, and squeaky tennis shoes regulate, what makes software so special?

Which brings us to the point of Grimmelmann’s paper. He argues that software is very different from ordinary physical objects, so that software-based regulation is not the same animal as object-based regulation. It’s best, he says, to think of software as a different medium of regulation.

Software-based regulation has four characteristics, according to Grimmelmann. It is extremely formal and rule-bound. It can impose rules without disclosing what the rules are. Its rules are always applied and cannot be ignored by mutual agreement. It is fragile since software tends to be insecure and buggy.

Regulation by software will work best, Grimmelmann argues, where these four characteristics are consistent with the regulator’s goals. He looks at two case studies, and finds that software is ill-suited for controlling access to copyrighted works, but software does work well for managing online marketplaces. Both findings are consistent with reality.

This is a useful contribution to the discussion, and it couldn’t have come at a better time for Freedom to Tinker book club members.

Freedom to Tinker is hosted by Princeton's Center for Information Technology Policy, a research center that studies digital technologies in public life. Here you'll find comment and analysis from the digital frontier, written by the Center's faculty, students, and friends.