Researchers and hackers are developing tools to execute a new data-leak threat: sneaking proprietary information out of networks by hiding it within VoIP traffic.

Techniques that fall under the category of VoIP steganography have been discussed in academic circles for a few years, but now more chatter is coming from the hacker community about creating easy-to-use tools, says Chet Hosmer, co-founder and Chief Scientist at WetStone Technologies, which researches cybercrime technology and trains security professionals investigating cybercrimes.

“There are no mass-market programs yet, but it’s on our radar, and we are concerned about it given the ubiquitous nature of VoIP,” he says. VoIP steganography conceals secret messages within VoIP streams without severely degrading the quality of calls.

Steganography in general is hiding messages so no one even suspects they are there, and when done digitally, it calls for hiding messages within apparently legitimate traffic. For example, secret data can be transferred within .jpg files by using the least significant bits to carry it. Because only the least significant bits are used, the hidden messages have little impact on the appearance of the images the files contain.

There are more than 1,000 steganographic programs available for download online that can place secret data within image, sound and text files, Hosmer says, and then extract it. There are none for VoIP steganography yet, but in the labs, researchers have come up with three basic ways to carry it out.

The first calls for using unused bits within UDP or RTP protocols – both used for VoIP – for carrying the secret message.

The second is hiding data inside each voice payload packet but not so much that it degrades the quality of the sound.

The third method calls for inserting extra and deliberately malformed packets within the VoIP flow. They will be dropped by the receiving phone, but can be picked up by other devices on the network that have access to the entire VoIP stream. A variation calls for dropping in packets that are so out of sequence that the receiving device drops them.

These techniques require compromised devices or conspirators on both ends of calls or a man-in-the-middle to inject extra packets. “It’s much more difficult to do and much more difficult to detect,” than hiding data within other files, Hosmer says. The medium used to carry secret messages is called the carrier, and just about anything can be a carrier. For example, x86 executables can carry secret messages, according to Christian Collberg, an associate professor of computer science at the University of Arizona and co-author of the book Surreptitious Software.

By manipulating the compiler, it can be made to choose one addition operation over another, and that choice can represent a bit in the secret message, Collberg says. “There are lots of choices a compiler makes, and whenever you have a choice, that could represent a bit of information,” he says.

Even something as broadly used as TCP/IP can be host to steganographic messages. One of the newest methods takes advantage of TCP retransmission – known as retransmission steganograpny (RSTEG) – in which sending machines resend packets for which they fail to receive acknowledgements.

The sending and receiving machines must both be in on the steganography, according to a paper written by a group of Polish researchers headed up by Wojciech Mazurczynk at the Warsaw University of Technology. At some point during the transmission of a file, the receiving machine fails to send an acknowledgement for a packet and it is resent. The resent packet is actually different from the initial packet and contains a steganographic message as the payload. The receiving machine can distinguish such resent packets and opens up the message, the researchers say. In his blog Crypto-Gram Newsletter, security expert Bruce Schneier dismisses the threat from RSTEG. “I don’t think these sorts of things have any large-scale applications,” he says, “but they are clever.”

Mazurczynk and his colleagues have spent a lot of time figuring out new carriers for secret messages, publishing research on embedding them in VoIP and wireless LAN traffic.

In general, defending against steganography is tough to do because traditional security devices such as firewalls and application firewalls don’t detect this type of illicit transfer; a file containing a secret message looks just like a legitimate file.

The best way to combat suspected use of steganography to leak corporate data is to look for the telltale signs – known steganography programs on company computers, says Hosmer. On systems where it is found, forensic analysis may reveal files that contained messages and an indication of what data might have been leaked.

When the steganography program is known, it can be applied to the carrier to reveal the secret message. That message may be in code and have to be decrypted, he says.

In many cases, just knowing that steganography is going on and who is responsible is enough for a business. They can confront the person and take steps to prevent further leaks, Collberg says.

But businesses can take more active steps such as destroying the secret messages by altering the carrier file. For instance, if the carrier is an image file, setting all the least significant bits to zero would destroy any messages contained there without significantly changing the appearance of the image, he says.

Free programs such as Stirmark for scrambling files enough to destroy steganographic messages are available online. Keith Bertolino, founder of digital forensics start-up E.R. Forensics, based in West Nyack, N.Y., has developed double stegging – inserting stenographic messages within files with the intent of disrupting other stenographic messages that might also be in the files. He is waiting to find out if he gets a Small Business Innovation and research (SBIR) grant from the government to pursue turning his steganography jamming technology into a commercial product.

According to Hosmer, a look at evidence in closed cases of electronic crime found that in 3% of those cases, criminals had steganographic programs installed on their computers. “The fact that these criminals were even aware [of steganography] was a startling surprise to law enforcement agencies,” he says.

Interest in steganography is growing, according to Wetstone Technology’s monitoring of six popular steganography applications. In 2008, the six combined logged 30,000 downloads per month, up from 8,000 to 10,000 per month about three years ago, Hosmer says. That’s not a dramatic increase given that the use of Internet-connected computes has gone up in the meantime, but it is still noteworthy, he says.

Steganography is not always bad. Technically, steganography is just the same as digital watermarking, but with different intent, Collberg says.The watermark is a secret message embedded, for instance, in an image file so if the image is use online, a Web crawler can find it. Then the creator of the image can check whether the site displaying the image has paid for it or is violating copyright, he says.