Webroot Bulletin Regarding AV-Comparatives Results

If there is one thing that can be observed about the AV industry, it is that no solution is ever 100% effective at blocking malware. With this in mind, Webroot SecureAnywhere (WSA) was designed to protect users even in cases where undetected malicious software has made it onto the system.

AV-Comparatives recently published results for June’s “Real World” Protection Test. This test aims to replicate a real world experience for how malware would infect a PC. The scores indicate how many threats were detected vs. missed.

In June’s test, Webroot SecureAnywhere detected 93.4% of samples while 6.4% were able to “compromise” the test PC. I put compromise in quotes because WSA functions very differently from the other products in this test. The key difference is in how WSA protects its users when a malicious piece of software slips by. Unfortunately this test doesn’t demonstrate how well a user is protected in the event of an infection, or how long it takes before an unidentified infection is detected and removed. These are two very important factors which should be considered when judging WSA’s efficacy. Because of these layers and because of how our threat research process works, we’re constantly aware of how effective our solution is for our users. Since the release of SecureAnywhere, we’ve seen a massive improvement in user security based on the dramatic decrease in the number of threats we’ve had to manually remove from user systems. In addition, our support cases have decreased while our satisfaction scores have climbed well above others in the industry.

To better understand how WSA provides protection in these areas it is first necessary to take a step back and understand how WSA is fundamentally different from a traditional definition-based antivirus solution. One primary difference is WSA uses a cloud-based architecture which is populated in real time with files seen by all WSA endpoints around the globe. A major benefit of this approach is there are no definition updates to burden the system, rather WSA checks in with the cloud whenever a new file is discovered. Another major benefit is that malware researchers are able to focus on and classify new files exclusively seen by our user base, also in real time. This ability is especially important because it drastically reduces the turnaround time from discovery of a new malicious application to the classification and protection of all WSA users.

After looking at the 68 misses from June’s AV-Comparatives test, we found 65 of the samples had been classified within a few hours of their test, with the remaining three being classified minutes after receiving the samples. Of the 68 misses, 34 of the files were seen for the very first time during the test; none of our users were ever affected by them and we had never encountered those components across our entire user base. The other 932 samples were blocked automatically during the test.

So this begs the question, how did WSA protect these infected endpoints while the infections were still unknown to the cloud user base? There are two pieces to this puzzle. The first piece focuses on ensuring WSA is able to reverse all system changes made by a new unknown file and to prevent any irreversible changes from taking place. For example, if a newly discovered program makes file system, disk, registry, or memory changes, these are recorded and analyzed in real time. WSA then checks frequently with the cloud while the program runs to see if an updated classification is available for the unknown files on a system. During this period, the program is able to change the system, but it is under a transparent sandbox where all of the changes taking place are not only being analyzed for behavior correlation, but are also being recorded to see the before-and-after view of every modification to the system. If at any point the cloud comes back and indicates a file is malicious, WSA will automatically remove the infection and restore the system perfectly to a pre-infection state. WSA effectively creates its own system imaging feature that works on a per-application basis, allowing it to generically and completely revert out any threat without needing humans to write signatures. Pretty neat, eh?

The other piece to protecting an infected system is to prevent sensitive data leakage. Most often, malware is after credentials to various websites, whether that is personal email, banking sites or social networking sites. To prevent this from happening, WSA has an innovative combination of security components that are used to create a safe browsing environment which works without any requirement of user interaction. This is done by blocking the various methods used to lift keystrokes from secured browser sessions as well as the numerous new methods that threats are using today: information stealing attacks running in the browser, screen-grabber threats, man-in-the-middle attacks, and various other forms of covert information gathering.

With this layer of protection, WSA will block threats even if it doesn’t know that a file is malicious. While it is definitely best to remove any threats from your system for performance reasons, with WSA enabled, you could install an undetected, zero-day Zeus infection and continue safely banking online even with it active on your system. This layered defense allows WSA to close the gap from what it finds with pure signatures to what it is able to actually protect the user against. While detection is certainly very important, actually blocking the vectors of attack used by malware is what the goal of security software should really be.

Currently most “Real World” tests rely on automation and AV scanners are only given a single chance at detection before the test system is reverted for the next round of testing. Unfortunately this testing model doesn’t give WSA a chance to leverage our unique cloud approach as it has a very static view of the files being tested. If another scan had taken place a short amount of time later, nearly all samples would have been detected from background rules running in our cloud and all system changes would have been reversed automatically.

Webroot continues to drive innovation in our products though this isn’t always met with equal innovation in the testing process. We are actively working with various 3rd party testers to build better real world testing environments which gauge how well security software is able to mitigate the risks of infected systems along with how rapidly vendors are able to update protection to their user base after the discovery of a new malicious threat.