Five Must-Have Capabilities for Controlling Modern Malware

Sophisticated Malware is Crafted to Ensure it Remains Undetected by Antivirus Products. Organizations Must Prepare to Deal with Unknown Files.

2011 has been an active year in IT security to say the least. Modern malware and advanced attacks are top of mind for security managers. Over the course of this series we’ve reviewed some of the key capabilities and best practices needed to build into security infrastructures in order to be prepared for the challenge of these threats going forward.

Network-Based Malware Controls

Malware has become a network-borne and network-enabled threat, and as such we need to bring network controls to the fight against malware. Today the network is integral to all parts of the malware lifecycle, from infections via drive-by-downloads, digging deeper into a compromised network, and the command-and-control traffic used to coordinate the malware and ultimately exfiltrate data. If you can take away the ability for malware to communicate, you can effectively take away much of its power.

The problem is that most enterprises are not prepared for dealing with malware at the network level. For years, anti-malware was seen as an end-point technology, while intrusion prevention was seen as a network technology, and the two areas rarely overlapped. Additionally, malware analysis was often too slow to be performed inline, because the security solution needed to download the entire file before it could be analyzed. This problem is increasingly being tackled through the use of stream-based anti-malware technologies, which allows the security solution to begin analyzing a file as soon as it is received and to continue analysis as it arrives. This greatly accelerates the analysis and allows security to scan for malware at roughly the same speeds the industry has come to expect from IPS and other security technologies.

Full Inspection of All Traffic

Modern malware and attacks use a web of subterfuge and evasion in order to avoid detection. This includes blending in to approved types of traffic, tunneling within approved applications, traveling over strange or non-standard ports or hiding within encryption, proxies or other circumventors. In short, attackers take advantage of the places they know we don’t bother to look, or where we can’t look for technical reasons.

The simple truth is that the only way to be sure that we actually analyze all malware-related traffic is to perform full inspection of all traffic on all ports. This is a big shift, because the past two decades of security infrastructure has been built on the assumption of classifying traffic based on ports, fast-pathing as much as possible and selectively going deeper only on predetermined types of traffic. If we continue to accept the presence of systemic blind spots in our security infrastructure, we can’t be surprised when malware shows up in those channels. The good news is that advancements in hardware and software have made this goal of full analysis of traffic a practical option, and opens the door for reclaiming the visibility that we have slowly lost over the past decade.

A Reasonable Return to Default Deny

The concept of positive control (meaning only approved or “white-listed” traffic is allowed, and everything else is denied) is one of the foundational building blocks of what a firewall was historically supposed to do. Over time, this control eroded as applications began to share common ports or intelligent enough to find any open port. This often meant that deny policies were not effective and also carried the possibility of unintentionally blocking valid, approved apps.

However, as classification technologies have improved and threats become more sophisticated, we have begun to see the concept of white-lists and positive control making a strong comeback. This is important for controlling malware, because as we limit the applications and traffic allowed on the network, we drastically reduce the amount of room that malware has to maneuver and avoid detection. For example, an organization may choose to default to deny all applications that tunnel other applications, and then create exceptions for the applications that employees specifically need. Or by policy, IT may allow the application, but disallow tunneling in order to reduce the risk. By leveraging positive control, we can immediately remove a key safe haven for malware.

Context in the Enforcement Phase

In modern security, context is king. We have lots of users, devices, applications and lots of threats. Very few IT teams have the man-power to comb through every log at all hours of the day, and we have turned to context in order to make more informed decisions about what constitutes a real and active threat.

The problem with context today is that it is largely applied in a forensic capacity as opposed to real-time enforcement. Having context in a post-mortem investigation is critically important, but it is far more beneficial if we can use that context to catch and prevent threats in the first place. This requires inline security technologies to share information in real-time. This is a key reason why integrated approaches to network security are becoming popular again – not because of convenience but because it’s far more powerful and effective at stopping complex attacks. If we can integrate knowledge of the application, user, exploits, malware, file transfers and URLs into a single real-time security context, then we can play at the same level of sophistication that we see from modern malware and threats.

Analysis of the Unknown

One key to any type of security is to recognize when something does not belong, and this is especially true of modern malware. Once we have inspected all the traffic and done our best to identify it, we then must be prepared for how we will deal with the unknowns. This can include unknown (or unclassified) network traffic, or it may be new files we haven’t encountered before. Malware traffic often presents as unknown UDP or TCP since they often use their own custom protocols, which are then even wrapped in unknown encryption. By inspecting all traffic we can reestablish a baseline of what we expect to see in our networks, so that we can easily recognize and block suspicious traffic when it shows up.

Some of the most sophisticated and dangerous malware is custom crafted in order to ensure it is not caught by antivirus products, so it is also important to prepare for dealing with unknown files. IT needs to expect this type of traffic on the network and build in the procedures and systems to deal with it, such as sandboxing solutions, that can reveal if a file is malicious or not.

Wade Williamson is Director of Product Marketing at Vectra Networks. Prior to joining Vectra, he was a Senior Threat Researcher at Shape Security. He has extensive industry experience in intrusion prevention, malware analysis, and secure mobility. He has extensive speaking experience having delivered the keynote for the EICAR malware conference and led the Malware Researcher Peer Discussion at RSA. Prior to joining Shape, he was Sr. Security Analyst at Palo Alto Networks where he led the monthly Threat Review Series and authored the Modern Malware Review. He has also led the product management team at AirMagnet where he helped to develop a variety of security and network analysis tools targeted to WiFi networks. He has been a steady and active researcher of new threats and techniques used to compromise enterprise networks and end-users.