The reemergence of Conficker, which has infected millions of computers worldwide since first surfacing in November 2008, is a reminder of just how difficult it is to eradicate self-replicating worm programs once they penetrate a network.

A team of researchers at The Pennsylvania State University in University Park is developing an approach to finding and confining computer worms that relies on a computer's ability to detect suspicious network activity before it becomes a serious problem. Although there's already a market for this type of software—known as anomaly detection system (ADS)—the researchers think their new algorithm will improve on existing ADS, specifically protecting local networks (those run inside an organization's firewall) from the spread of worms.

The algorithm checks all of the devices (including computers, network routers and printers) connected to a local network to determine which of these are susceptible to a worm infection. (A printer, for example, would not be a target because it does not engage in two-way communications as a computer would). Because worms move from computer to computer by scanning victims for vulnerabilities, the algorithm informs the network's intrusion detection system to monitor for scanning within the network (some of which is legitimate) and initiate a lockdown if it sees more than the usual scanning activity or other suspicious behavior from a particular computer.

The algorithm is novel for its ability to accurately estimate how much scanning to permit before locking a device out of the network, says Peng Liu, a Penn State professor of information sciences and technology and one of the researchers. The goal is to have an optimal trade-off between, on the one hand, ensuring that a potential worm threat does not cause an epidemic within a local network and, on the other, that network activity is not disrupted due to false alarms. "The worm-containment software that is already on the market does not know what the best threshold is," says Liu, who along with his colleagues published a paper describing their research in the February issue of Computers and Security.

The key to a good anomaly-detection approach is being able to distinguish between a self-propagating worm and normal network scanning, says George Kesidis, a Penn State professor of electrical engineering and computer science and engineering who worked with Liu on the research. "Network traffic is so complicated that you can't just fire off an alert each time you see something that's abnormal," he says.

"In computer virology it's all about slowing the infection rate and buying time," and this is often done by cutting some machines off from the network, says Herbert Thompson, an adjunct professor of computer science at Columbia University and program committee chair, RSA Conferences (an annual gathering of security professionals). The danger in doing this is that if you mistakenly target machines that are not infected you could paralyze your network. If you are too lax in containing infected machines, however, the worm will quickly spread beyond your ability to manage it. "We definitely need better answers about where to set such a threshold," Thompson says, "and this research seems like a step forward."

Worms can infect local networks from a variety of sources. One of the most common is when someone connected to the network is surfing the Web and their computer is attacked by malicious software written into a Web site they visit. (This is sometimes referred to as a "drive-by download".) Another common entry point for worms is via e-mail, when a computer user opens up an attachment that has been infected by a virus carrying the worm. "The vast majority of the attacks we see are silent attacks looking to steal things from a person's computers," Carey Nachenberg, vice president and fellow of security technology and response for Symantec Corp., a technology security company based in Mountain View, Calif.

Once a computer has been infected, a worm can spread quickly and infect other computers on the local network, Nachenberg says, adding, "The problem that [the Penn State researchers] are trying to solve is a real one."