On Survivorship Bias and Bulletproof Umbrellas

According to Gartner’s recent report: “Strategies for Dealing With Advanced Targeted Attacks”, we’re in the eye of a five years’ storm; a pwnado (or would you prefer malwarricane? vulncano?). However, the strategies being adopted by many enterprise InfoSec /OpSec teams to combat these threats often suffer from survivorship bias, packaged by security vendors under the guise of “Defense in Depth”.

The Misconception: You should focus on the successful if you wish to become successful.

The Truth: When failure becomes invisible, the difference between failure and success may also become invisible.

In her blog, Deana writes about an incident of survivorship bias that nearly cost many World War II bomber crews their lives. In the early years of World War II, the chances of a member of a bomber crew making it through a tour of duty was about the same as calling heads in a coin toss and winning (hardly favorable odds). When the military looked at the bombers that had returned from enemy territory, they recorded where those planes had sustained the most damage. Over and over again, they saw that the bullet holes tended to accumulate along the wings, around the tail gunner, and down the center of the body. Wings. Body. Tail gunner.

Considering this information, where would you put the extra armor? Naturally, the commanders wanted to put the thicker protection where they could clearly see the most damage, where the holes clustered.

In the context of cyber defense, when I ask CISOs why they pay for Windows anti-virus, I’m often pointed to sites with tests that show some incremental improvement over included/free tools. These tests measure what percentage of malware was detected by the anti-virus software, and while some commercial products score slightly better, it’s my contention that Security Essentials (Windows out of the box) does a good enough job. An investment in anti-virus is an investment driven by survivorship bias.

Back to World War II: The commanders were wrong. The fact was the bullet holes showed where the planes were strongest. The holes showed where a bomber could be shot and still survive the flight home. After all, here the planes were, holes and all. It was the planes that weren’t there that needed extra protection, and they needed it in places that these planes had not. The holes in the surviving planes actually revealed the locations that needed the least additional armor. Look at where the survivors are unharmed, and that’s where the bombers were most vulnerable; that’s where the planes that didn’t make it back were hit.

What is broken in today’s information security culture is the belief that incremental innovations and investments in existing technologies can stave off modern attacks.

We should look to the RSA hack of 2011 as the canary in the coalmine. RSA did everything right according to the playbook: they implemented defense in depth – and they still got hacked. Since then we have seen a continuous rise in sophisticated attacks that architecturally circumvent historically effective technologies.

When it rains, invest in an umbrella to stay dry. Should it start raining bullets, invest in Kevlar gear so you don’t die. Bromium vSentry applies new innovations in virtualization to secure your company from threats that bypass all other protections, including legacy technologies like anti-virus and newer ones like sandboxes.