3
3 The Mantra: “Infosec is Terrible at Metrics” The metrics we can measure has little to do with security –Ex: Success of Antivirus System The stuff we really need to convey is the hardest to collect/quantify –“What is the sound of one hand clapping?” When we quantify numbers, they question our calculations They really don’t care about security…only compliance –“What needs fixing in security, and when will it be fixed?” Kim L. Jones CISM, CISSP, CRISC, MSIA

7
7 Contraxiom #1 -- Work Kim L. Jones CISM, CISSP, CRISC, MSIA For Geeks, Work is about solving problems Problems organize our thinking and provide a specific structure and approach Problem solving starts in the present. GeeksNon-Geeks For Non-Geeks, Work is about achieving a vision Visions are an imagined experience that get us out of bed in the morning. Vision realization starts in the future.

8
8 Contraxiom #1 -- Work Impact on Metrics –Do we truly understand the vision? And what the business must do/is trying to do to achieve that vision? –Are we relating our metrics TO the vision? This gives our metrics appropriate business context (the “So What?” factor) Kim L. Jones CISM, CISSP, CRISC, MSIA

11
11 Asking the Right Question Is The Road Open? Kim L. Jones CISM, CISSP, CRISC, MSIA How close is the nearest rebel encampment? Are there mines on the road? What is the current state of rebel supplies? Is the destination still neutral

12
12 Asking the Right Question Are We Secure? Are We Compliant? What Is The Current Level of Risk? Are Our Controls Sufficient? Is The Risk Balanced Sufficiently To Achieve Our Vision? Kim L. Jones CISM, CISSP, CRISC, MSIA

14
14 Compliance Isn’t Always Bad Executives latch on to compliance because it meets the requirements of a good metric. The problem (as we all know) is that compliance doesn’t equal security –Worse, compliance does not equal appropriately balanced risk Even if you win the metrics battle, compliance will remain an issue if you are a regulated entity Possible (useful) workaround: measuring compliance with your policy framework –Meets compliance standards –Sets the risk floor! –Is in line with the vision! Kim L. Jones CISM, CISSP, CRISC, MSIA

16
16 Testing the Hypothesis… Corporate Mission: “Enable a Better Way for Trusted Commerce Infosec Mission: “We ensure the Trust in Trusted Commerce” –Trust defined as: your transactions will process as expected, when expected, how expected (i.e., without alteration). Hypothesis: Our Transactions Can be Trusted –Sub-Hypotheses: There are limited points of entry through which an outsider can get into our information systems Once inside, attackers cannot obtain access to internal systems because of strong passwords An intruder finding a hole somewhere cannot jump to core transactional systems Administrative credentials are difficult to obtain Kim L. Jones CISM, CISSP, CRISC, MSIA

17
17 Testing the Hypothesis: Disproving the Negative Kim L. Jones CISM, CISSP, CRISC, MSIA There are limited points of entry through which an outsider can get into our information systems Attackers cannot obtain access to internal systems because of strong passwords An intruder finding a hole somewhere cannot jump to core transactional systems Administrative credentials are difficult for attackers to obtain The network is porous, permitting easy access to any outsider Attackers can obtain access to internal systems because of password policies are weak An intruder finding a hole somewhere can easily jump straight to core transactional systems Once on the network, attackers can easily obtain administrative credentials

18
18 Testing the Hypothesis: Diagnostic Questions Kim L. Jones CISM, CISSP, CRISC, MSIA How many sites are connected directly to the core network without intermediate firewalls? How many sites have deployed unsecured wireless networks? Starting with zero knowledge, how many minutes are required to gain full access to network domain controllers? What percentage of accounts could be compromised in <15 minutes? How many internal zones/subnets exist to compartmentalize the environment? How many administrative-level passwords could be compromized in the same time frame? How many universal administrator accounts exist in the environment? The network is porous, permitting easy access to any outsider Attackers can easily obtain access to internal systems because of password policies are weak An intruder finding a hole somewhere can easily jump straight to core transactional systems Once on the network, attackers can easily obtain administrative credentials.

19
19 Making the Subjective Objective… One of the complaints re: security metrics is an inconsistency in measurement –This undermines even the strongest/most significant metric as being opinion versus fact. Semi-qualitative metrics are a good starting point…but consider going a step further and implementing a standard evaluation checklist with relative values. Plotting the results of multiple assessments over a specific population may create a contextually relevant metric Kim L. Jones CISM, CISSP, CRISC, MSIA

23
23 Wrapping it Up… Security is, at a fundamental level, a state of mind –Ditto for balanced risk It stands to reason, then, that measuring security and/or risk can be like catching a moonbeam –“What is the sound of one hand clapping?” Metrics and measurement are both art and science…you need to study both Make your metrics contextually relevant –What’s the vision? Be sure you’re answering the right question!! Kim L. Jones CISM, CISSP, CRISC, MSIA