Events

Press

Archive: Planning a Proof of Concept: Thinking Red, Thinking Blue

January 15, 2016 /
Paul Morville

(Editor’s Note: On July 19, 2016 Carbon Black announced its acquisition of Confer. The enclosed blog was originally posted on Confer’s website on January 15, 2016.)

This is the first in a 2-part series on how to ensure a successful proof of concept (POC) when evaluating endpoint detection and response (EDR) solutions. In this installment, Confer co-founder Paul Morville provides CSOs with guidance on how to plan an effective POC, in part by taking into account the mindset of both the Red Team and the Blue Team.

Few things are more disappointing or costly than deploying a product that fails to live up to the vendor’s claims or meet your team’s expectations. Too often, an EDR solution that performs smoothly in a POC proves to be cumbersome or intrusive in a real-world setting. Savvy users become frustrated and turn off security controls, increasing the attack surface. The security team may actually uninstall the product and go back to the drawing board, a setback that increases costs while undermining users’ trust.

These pitfalls are common, but they can be avoided by using best practices during your EDR proof of concept. As discussed in an earlier blog on security testing (“It’s a Circus Out There”), vendors can skew POC results to favor their own products. However, you can avoid being “gamed” by following some proven guidelines for planning and implementing an effective POC.

Below, we offer four recommendations for the planning stage. In Part 2 of this series, we will offer practical tips for implementing the POC in a way that helps ensure valid and actionable results.

Don’t delegate the scoping and planning process.

Because your senior team members are so busy, it’s tempting to delegate the task of planning a POC to someone in a junior role. However, doing so misses an opportunity to test the various product claims made by vendors and to evaluate which solution will be most effective in real-world attack scenarios.

The POC is your chance to define what your organization wants to get out of an EDR solution in terms of technical, operational, and business requirements. In forward-thinking organizations, it is common for the CSO or a direct report to be engaged in the upfront planning. Leaving the job to a single tech tester means you’ll miss the broader perspective a decision-maker brings to the table.

Ask “Will it flatten the stack?”

When testing a product, ask whether it will help you flatten the endpoint security stack—and thereby reduce management cost and complexity—by delivering multiple capabilities. How many items on your requirements list can you check off with the product? To what extent will it help you detect, prevent and respond to attacks?

Your POC should evaluate the key functionality the product claims to offer. For example, if the product supports incident response, does it give you full visibility into the details of a malware attack? Can you see the C2, secondary payloads, process execution, and/or injection?

Challenge the Red Team.

Adopt the attack mindset of the Red Team. Your test should serve as a proxy for the determined adversaries you face in the real world: It should emulate the types of attacker behavior you are likely to face, unleashed on representative endpoint targets. And it should not focus solely on breach prevention. Skilled attackers can easily penetrate most networks. The POC should evaluate what kind of damage they can do once they are inside and how readily their behavior can be detected and thwarted.

Get the Blue Team involved.

In a typical POC, the Red Team launches an attack and then, a month later, writes a static report that says, “We got in, and here are the vulnerabilities we found.” Your POC will be far more useful if one or two key members of your Blue Team are involved, sitting alongside the Red Team and interacting with them.

Your people can watch how an attack unfolds, analyze how your defenses react, and evaluate what kind of information is generated by the product being tested. In turn, this gives them a better sense of how the product can actually be used in your existing environment.

Following these planning guidelines, you can test how well a vendor product addresses your unique EDR requirements. In our next post, we’ll offer tips on implementing your POC to yield the most useful results and insights.