What's in your test, training, and exercise program?

Kacy Zurkus |
May 17, 2016

From tabletop to functional test exercises, is running tests in anything other than an actual network really effective?

May marks the month of graduation for many college students across the country. For the past few years, these learners have been testing and training in a classroom environment, and now they are presumably ready to enter into the 'real world.' Are they prepared though?

If you question whether college grads have acquired enough hands-on experience to aid in putting out the security fires blazing in your enterprise, then perhaps you may also want to question how well your test, training, and exercise program prepares your security practitioners to respond to a crisis.

The National Institute of Standards and Technology (NIST) developed standards and guidelines not only to aid corporations in developing a preparedness plan but also to ensure that the plan is maintained in a state of readiness. The NIST guide said, "This includes having IT personnel trained to fulfill their roles and responsibilities; having plans exercised to validate their policies and procedures; and having systems tested to ensure their operability."

But are these plans ensuring that exercises are not overly focused on narrow types of sensational threats, to the point of yielding little value for the types of threats more likely to affect businesses?

With the current tactic of posing red teams against blue teams, said Ben Cianciaruso, co-founder and COO at Verodin, "There is way too much emphasis on prevention and not a lot around detection and response. Exposing holes doesn't really facilitate improvement and learning."

Verodin has noticed in many organizations that a lot of people were heavily reliant on the tools that they had. "What we are trying to do is enable organizations on the defender side with something that is measurable to mature those capabilities," Cianciaruso said.

In examining the testing exercises of different enterprises, Cianciaruso said, "One of the things we found is that red teams are set up as a gotcha exercise. They are intended to show the blue team 'look we got you', but it doesn't really provide a mechanism to allow the blue team to improve. That is where you are going to get your value."

Rather than identifying holes and playing the 'gotcha' game, Verodin wants to see organizations rethinking their training and testing exercises so that the red team is instead able to hand over their findings to the blue team and re-run attack simulations so that now they have the means and the access to learn and improve.

"Understanding where the failings are is critical in preparing them for when the attack happens for real," said Cianciaruso.

The goal in these testing, training, and exercise programs is to get more value out of the people you have, to "Measure stock abilities and provide opportunities to address where they failed on a quarterly basis. To see whether you are improving through these exercises," Cianciaruso said.