4 The Need for Security MetricSome simple questions difficult to answer:Are we more secure than that company?Are we secure enough?How much additional security will be provided by that firewall?“You cannot improve what you cannot measure”A security metric will allow for a direct measurement of security before, and after deploying the solutionSuch a capability will make network hardening a science rather than an art

6 An Example Metric for Known Vulnerabilitiesftp_rhosts(0,1)root(2)rsh(0,1)trust(0,1)sshd_bof(0,1)user(1)ftp_rhosts(1,2)trust(1,2)rsh(1,2)rsh(0,2)trust(0,2)ftp_rhosts(0,2)user(2)local_bof(2,2)user(0)0.80.90.10.0870.720.60.54Attack probability (DBSec’08)E.g., probability of exploiting ftp_rhosts is 0.8E.g., probability of reaching root(2) is 0.087Here is one example of existing metrics for known vulnerabilities (based on our DBSec’08 paper). No need to explain in details.Just say the metric assigns probability to individual vulnerability based on CVSS (e.g., 0.8 for ftp_rhosts(0,1)), and calculating probability for reaching the goal based on probability theory (e.g., for eaching root(2)).

7 An Example Metric for Zero-Day Attacksk-zero day safety (ESORICS’10)k: the minimum number of distinct zero-day vulnerabilities required for attackLarger k means safer networksE.g., assuming no known vulnerability here, then k=1, if ssh has no known vulnerability; k=0, otherwiseHere is one example of existing metrics for zero day attacks (based on our ESORICS’10 paper). No need to explain in details.Suppose host 2 is the target. If no service has any known vulnerability, then attacker will require at least one zero day vulnerability (in ssh) to attack host 2.

8 How to Measure Both?A natural next step is to develop metrics that are capable of handling the threats of both known vulnerabilities and zero day attacks

9 Outline Introduction Motivating Example The MTTC metric modelsSimulationConclusionI’ll first motivate our study, and summarize the limitations of related work.I’ll then introduce some basic concepts such as attack graph.I’ll describe our model in three stages: how to assign individual values using CVSS, how to compose them in a static case, how to compose them in dynamic caseI’ll discuss two case studies.

10 How to Measure Both?A viable approach is to combine those two types of metrics,Known vulnerabilitiesZero day vulnerabilitiesthrough, for example, a weighted sumE.g., we assign a score s (0 <= s < 1) to known vulnerability, and 1 to zero day vulnerabilityHowever, such a naïve approach may lead to misleading resultsOne way to measure both is to combine existing metrics for known, and zero day vulnerabilitiesFor example, a naïve solution is to use weighted sum, to assign a smaller than 1 score to known vulnerability and 1 to zero day vulnerability (since known vulnerability is considered easier than zero day ones)

11 Issues with Such a Naïve SolutionConsider this sequenceInitially, sssh+sssh+sbofIf we patch one of the ssh services, sssh+1+sbofIf we path both ssh, 1+sbofPatching both is less secure than patching only one – difficult to explainAdding the two metrics together makes little sense, when they have different semanticsThe problem with this naïve solution (of adding the two together) is that it makes little sense, because the two metrics measure different things (one for difficulty of exploiting a known vulnerability, one for the likelihood of having a zero day vulnerability).Explaining this example is optional.

12 Our Solution: Using Time to Combine Different MetricsDefine the MTTC t of a vulnerability xInitially, t1 = f(ssh)+f’(ssh)+f(bot)Patch one ssh, t2 = k+min(f(ssh),k’)+f(bot)Patch both ssh, k+k’+f(bot)Which case more secure will depend on how you define f and k. What is important is the model still applies.No need to explain the formulas in details (say explanations are in paper).The key point is, we combine the two types of metrics using time, so there is a clear and coherent semantics, and no matter how you define f(x) and k, the model still applies.

13 ContributionAmong the first security metrics capable of handling both known vulnerabilities and zero day attacks under the same model with coherent semanticsThe proposed metric provides more intuitive and easy-to-understand score (time) than previous work based on abstract value-based metricsWe take a layered approach such that the high level metric model remains valid regardless of specific low level inputs

15 Mean Time-to-Compromise (MTTC)Given an attack graph and goal, the MTTC of a condition c in an attack graph is defined as the average time spent by an attacker in reaching the goalMTTC(e) is the average time required for the exploit ePr(ec) represents the conditional probability that a successful attacker actually chooses to exploit eP(c) represents the probability of an attacker being successful (i.e., s/he can reach the goal condition c)(Note that ‘chooses to exploit’ and ‘can exploit’ are two different things)Intuitively speaking, the dividend is the total time used by all successful attackers in reaching the goal condition c. the divisor is the number of successful attackers. So by diving the two, we have the average time used by each attacker, e.g., if among 100 attackers, 50 can reach the goal, and those 50 attacker use 36 hours in total, then 36/50 = 0.72hour is the MTTC.

16 An Example To determine MTTC(goal)We need to find the probabilities P(goal) and Pr(egoal) for each e (we will do this in three steps)We need to estimate MTTC(e) for each e

17 Step 1: Probability of Being Able to Exploit e When Its Pre-Conditions Are SatisfiedFor known vulnerabilities, we assign the probability based on CVSS scoresFor zero day vulnerabilities, we assign a fixed nominal 0.08 based on following assumptions:This is the probability of an attacker being able to exploit e.For known vulnerability, just use CVSS scoreFor zero day, we make some assumptions about their CVSS base metrics, as stated here on the slides, and then calculate a nominal score as 0.08.

18 An Example Apply this to our example:Here we just assign probabilities to each exploit, either based on CVSS (first two cases), or as a nominal value (last one)

19 Step 2: Probability of Being Able to Exploit eConstruct a Bayesian network based on the attack graphCalculate the probability that an attacker can reach the goalThis is to follow the previous probability assignment.No need to explain. Details are in paper.

20 Step 3: Probability of Attacker Choosing Exploit eHere we can make different assumptions, e.g.,An attacker may always choose the easiest exploit s/he is able toAn attacker may still choose harder exploits, the likelihood of which are proportional to their relative difficultiesEven though an attacker can exploit e, s/he may or may not choose to do so, because s/he usually have more than one choices.Don’t explain the algorithm. It calculates the probability that an attacker chooses e, based on the two assumptions.The procedure calculates pr(e) based on those two assumptions

21 An Example Apply this to our example:Here we just assign probabilities to each exploit, either based on CVSS (first two cases), or as a nominal value (last one)

22 Estimating MTTC(e) – Known VulnerabilitiesTo estimate MTTC(e), we average the two complementary cases:Exploit code already exists, e.g.,Exploit code does not exist, e.g.,Note those only represent one (rough) way of estimating MTTC(e)No need to explain. You can say the results are based on search theory and the previous MTTC work (Leversage et al., SP’08). Especially, the ‘5.8 days’ comes from Leversage et al., SP’08.

23 An Example Apply this to our example:Here we just assign probabilities to each exploit, either based on CVSS (first two cases), or as a nominal value (last one)

24 An Example The final result of our example:Here we just assign probabilities to each exploit, either based on CVSS (first two cases), or as a nominal value (last one)

26 SimulationThe algorithms are implemented using Python and libraries including the Networkx, OpenBayes, Pygraphviz[33] and Matplotlib. To render the graphs, we use GraphVizThe experiments were performed inside an Intel Core I7 computer with 8Gb of RAM. The computer is running Ubuntu LTSIf they ask why not do experiments, say there do not exist any publicly available datasets of attack graphs.We generate random attack graphs by growing from a seed graph, which is based on real world networks.

27 Simulation: MTTC vs Network SizeThose results show how the MTTC grows in the size of attack graph (# of nodes) or that of networks (# of hosts). In the figure, ind represents the maximum number of pre-conditions of each exploit.

28 Simulation: Running Time vs Network SizeThose results show how the running time grows in the size of networks. We can see that the running time is dominated by the generation of attack graphs and Bayesian networks (so that our algorithms do not cost that much time).

30 ConclusionWe have proposed a MTTC framework for developing metrics in order to measure both known and zero day vulnerabilitiesWe have defined our MTTC model, and provided examples of concrete methods for estimating inputs to the modelFuture work will be directed to developing more refined estimation methods, applying the metrics to network hardening, and conducting more realistic experiments