Science of Security Quarterly Lablet Meeting (UMD - Oct 2014)

College Park, MD October 30, 2014

Lablet Researchers meet at Maryland, share current research and ideas about Science of Security

The SoS quarterly Science of Security Lablet meeting, sponsored by NSA, was hosted by the Lablet at the University of Maryland (UMD) on October 28 and October 29, 2014. Quarterly meetings are held to provide research sharing and coordination, to present interim findings, and to stimulate thought and discussion about the Science of Security. Jonathan Katz, Principal Investigator at UMD, organized the series of talks and discussions about both the technical and behavioral aspects of cybersecurity. Kathy Bogner, Intelligence Community Coordinator for Cybersecurity Research, welcomed the group and described the "excitement" of the government at the efforts they are making. She challenged them to continue to address cybersecurity using strong scientific principles and methods and to share the fruits of their work.

The keynote was presented by John Pescatore of SANS Institute. His provocative talk described the current "sea change" in security engendered by the rapid development and deployment in sensors and actuators, massive new data sources, and in huge increases in M2M (machine to machine) communication-- the Internet of Things. New hacks are occurring in areas traditionally left alone, including hotel door systems, point of sale devices, HVAC systems, medical machinery, ATMs and kiosks. Automobiles are now sensor-laden and are each now generating a terabyte of data a year. With consumer fads driving the tech cycle, the life cycle of computing and data is shifting from every two to three years to a life cycle ranging from as little as two months to as much as twenty years. This shift, said Pescatore, increases the demand for basic computer "hygiene", offers an opportunity to avoid the mistakes of the past, and can drive suppliers and developers to build in higher quality security in their products and services.

Individual researchers and their teams presented materials from their ongoing work and a demonstration of updates to the Cyber-Physical Systems Virtual Organization (CPS-VO) web site. Research in progress that was presented included several briefs on human elements in cybersecurity and a review of Carnegie-Mellon's Security Behavior Observatory, the development of security metrics, a spirited discussion about the twin goals of composability and security, promising approaches to networked systems, resilience, and policy governed secure collaboration.

A special presentation about the challenges of teaching cybersecurity skills concluded the formal offerings. One of the unique features of the Lablets is that, in addition to research, they are charged with providing an educational and informational element to their work.

The next quarterly meeting will be held January 27 and 28, 2015 at North Carolina State University.

(ID#:14-2625)

Note:

Articles listed on these pages have been found on publicly available internet pages and are cited with links to those pages. Some of the information included herein has been reprinted with permission from the authors or data repositories. Direct any requests via Email to SoS.Project (at) SecureDataBank.net for removal of the links or modifications to specific citations. Please include the ID# of the specific citation in your correspondence.

Cyber Aptitude and the Science of Intellectual Assessment

Dr. Mike Bunting, UMD CASL - Center for Advanced Study of Language

Dr. Mike Bunting, of the Center for Advanced Study of Language (CASL) at UMD, addressed the security industry's need for hiring qualified candidates in an increasingly cyber-security-dependent workforce. The problem, according to Dr. Bunting, is the current difficulty in cyber selection, hiring, placement, and training. There are simply not enough candidates in the hiring pool who are able to readily perform their tasks. As hiring managers cannot predict a candidate's potential for job performance with total accuracy, the difficulty in effective cyber placement increases.

To this end, Dr. Bunting and CASL has partnered with the Georgia Institute of Technology (Georgia Tech) and U.S. government researchers to design an aptitude test for the assessment of potential candidates. The researchers will consider multivariate factors in cyber knowledge, skills, cognitive abilities, motivation, and personality attributes of current experts in the field, in order to identify parameters for success. The goal of the aptitude test is to accurately determine applicants' aptitude for cyber analysis.

Dr. Bunting provided a brief timeline of the work still to be performed before the test is operational and able to de deployed. CASL researchers began by studying available literature on cognitive and noncognitive factors existing in successful analytic job performance. Applications of the Psychosomatic Approach, such as data reduction and factor analysis, were employed in the initial data collection process. Researchers then reviewed specific cyber jobs and consulted instructors and experts in the cyber field, in order to identify success traits. Dr. Bunting and his team are working towards finalizing test content and test items.

Dr. Alain Forget presented on the design architecture and deployment of Carnegie-Mellon University's Security Behavior Observatory (SBO), which aims to identify privacy and security challenges faced by users and how to solve them. The SBO studies the user at home, analyzes how malware infects in the wild, and observes changes in computers and their users over time. Participants of the study agree to install CMU software onto their home computers, which allows data to be continually collected. In order to provide usage data for multiple research areas and to answer broad questions, the implemented SBO is a scalable client-server infrastructure designed to collect user behavior data over a long period of time, in this case several years. The SBO infrastructure was designed to scale with the desired length, breadth, and depth of data collection; take extraordinary care to ensure the security and privacy of the collected data, which will inevitably include intimate details about participants' behavior; and serve research interests which will change over the course of the study, as collected data is analyzed, interpreted, and suggest further lines of inquiry.

The pilot study was determined successful since all software functions correctly, all sensors collect the intended data; data is securely transferred and stored, and silent updates push fixes and improvements.

Articles listed on these pages have been found on publicly available internet pages and are cited with links to those pages. Some of the information included herein has been reprinted with permission from the authors or data repositories. Direct any requests via Email to SoS.Project (at) SecureDataBank.net for removal of the links or modifications to specific citations. Please include the ID# of the specific citation in your correspondence.

Developing Security Metrics

"Developing Security Metrics"

The presentation from the NC State Lablet and its collaborators on security metrics offered an overviews and a look at three research projects. The first described the overall security metrics project. Specific projects described included work on vulnerability and resilience prediction metrics and models, attack surface metrics, and using stack traces to approximate attack surfaces.

Metrics are a most important part of the sciences. Scientists use measurements of many kinds and have been in the forefront of developing new measuring tools and standards for a variety of scientific inquiries. Adding sound metrics to SoS is an important part of its development as a true science.

This project is intended to generate better allocation of resources for engineering secure software by contributing evidence-based knowledge, systematizing research on Intrusion Detection Systems, analyzing vulnerability-proneness and resilience of an overall system, and measuring the attack surface to assess risk. The NCSU Lablet's approach is to systematize the knowledge, with metrics of IDS evaluation, and to use those metrics internally. They seek to systematize IDS knowledge by way of classification and methods by evaluation, benchmark performances, and identifying and measuring inherent limitations. Currently, they have conducted a systematic literature review, collected over 300 papers, and classified and narrowed them down.

The goal of the Vulnerability and Resilience Prediction Metrics and Models Project is to develop a science-based understanding of which security metrics can be used to accurately predict its field resilience and vulnerability-proneness. The hypotheses tested are: Measurable properties of a system and associated software development processes are indicative of the presence of vulnerabilities in released software, and Statistical models based upon current (classical) reliability and availability prediction models and attack profiles can accurately predict the resilience of a system.

Their preliminary results indicate "Steady-state" security problem discovery rate in the field for Fedora and Windows are in the range of a few per week (rate is in the 10e-5 to 10e-7 per inservice-week). A large fraction (in the 30+% range) of problems reported weekly for STABLE field versions of Fedora and Windows are security problems. A very large fraction (65% and above) of security problems detected in the field for open-source Fedora (many different releases) belong to epistemic category (flawed process, knowledge, model, ..). Classical reliability models appear to describe and predict well field discovery of security problems for open-source Fedora.

They add that there are two implications. First, once software and its operational profile stabilize in the anomalies the result of sampling low to very low probability input vectors. Second, if a "white list" filter "closes" at that point, software may now be "immune" to further attacks (at the expense of some functional loss). 10e-5 to 10e-7 security problems per inservice week may be the best we can do given current OTS software development processes and usage patterns.

Their current conclusions are that we are making progress towards a good science-based understanding of which security metrics of a system can be used to (accurately) predict its field resilience and vulnerability.

Future hypotheses to be tested are:

Measurable properties of a system and associated software development processes are indicative of the presence of vulnerabilities in released software.

Statistical models based upon current (classical) reliability and availability prediction models and attack profiles can accurately predict the resilience of a system.

The goal of the attack surface metrics project is to assess risk of a software system by way of its input and output space. To do this the team will measure evolution over time assuming more inputs plus more entry points will produce more risk and more outputs plus more exit points will also produce more risk. The object will be to provide an early alert system for developers.

Identifying approaches used so far, they enumerate entry/exit points as functions that call input/output functions and measure ease of attack based on configurations. This methd will allow them to address three research questions:

Do vulnerabilities reside near the attack surface historically?

Do severe vulnerabilities appear in areas of high reachability?

Do applying asset weights and designed defenses improve our measurements?

Attack Surface Approximation via Stacktraces

Laurie Williams, Christopher Theisen, NC State University

The goal of the attack surface approximation via stacktraces project is to aid software engineers in prioritizing security efforts by approximating the attack surface of a system via stack trace analysis. This research will address the following questions:

How effectively can stack traces to be used to approximate the attack surface of a system?

Can the performance of vulnerability prediction be improved by limiting the prediction space to the approximated attack surface?

Articles listed on these pages have been found on publicly available internet pages and are cited with links to those pages. Some of the information included herein has been reprinted with permission from the authors or data repositories. Direct any requests via Email to SoS.Project (at) SecureDataBank.net for removal of the links or modifications to specific citations. Please include the ID# of the specific citation in your correspondence.

This presentation by Jonathan Aldrich from the CMU Lablet described a framework for understanding the hard problem of composability in the setting of security, along with highlights of lablet research results illustrating recent progress in this area and remaining research challenges. The format was an open discussion, and it proved lively.

Prof. Aldrich identified the primary challenge as the need to develop methods to construct secure systems with known security properties from components each of which has known quality and security properties, and avoid full reanalysis of the constituent components. Composition is needed to manage Increasing scale, complexity, dynamism, socio-technical ecosystems, and rich supply chains, and to direct evaluation of artifacts as they are produced and evolved.

The CMU SoS Lablet approach has been to focus on the hardest technical problems, emphasizing composability of modeling and reasoning as a key to scale and incrementality and human behavior and usability for developers, evaluators, operators, and end users. From this work, they seek to advance scientific coherence of cybersecurity technical results, advance most-effective scientific processes, acknowledge the multidisciplinary nature of cybersecurity, enhance the coherence of the body of technical results, enhance productivity, validity, and translation into practice and engage and broaden the cybersecurity technical community. To expand the community, they facilitate community and educational engagement with subcontractor partners, workshops, and conference events.

Work to date includes an initial workshop held in September, 2013. At this workshop, they developed a series of definitions, issues and approaches including crosscutting principles using assume-guarantee reasoning, game theory, and families of systems.

One key element was utilizing the work on sequential compositionality by Ahmad and Harper that produced the logical statement: if two components preserve confidentiality and we compose them in sequence, then the result preserves confidentiality. This premise underlies compositional security and is stated as: If two components preserve confidentiality and we compose them in sequence, then the result preserves confidentiality.

Articles listed on these pages have been found on publicly available internet pages and are cited with links to those pages. Some of the information included herein has been reprinted with permission from the authors or data repositories. Direct any requests via Email to SoS.Project (at) SecureDataBank.net for removal of the links or modifications to specific citations. Please include the ID# of the specific citation in your correspondence.

Remind me Tomorrow: Human Behaviors and Cyber Vulnerabilities

This research consisted of four interrelated projects addressing users and software updates. The research questions addressed included:

How do users update their software?

What are the barriers and facilitators to updates from the user perspective?

Can we compare what users say they do with what they actually do?

What are the implications for improving updating mechanisms and updating interfaces?

Using Symantec's Worldwide Intelligence Network Environment (WINE) data, field data collected from hosts around the world, user studies of patching behaviors, and comparing and contrasting results of these studies, this work in progress is determining whether and how security and software patches are actually being installed and how human behavior impacts cybersecurity.

The first part of the project looked at users' actual patching behaviors. The team analyzed 1,593 vulnerabilities in 10 side applications on Windows from 8.4 million hosts over 5 years. Using this data, they determined that patching behavior is not visible to network vulnerability scanners and is often targeted in spear-phishing attacks.

The second part of the project addressed the goal of measuring patch deployment milestones from the start of patching through time to patch 50%, 90%, 95% of vulnerable hosts and factors influencing the rate of patching. Preliminary conclusions are that start of patching is strongly correlated with the disclosure date--correlation coefficient of r = 0.994; 77% vulnerabilities start patching within 7 days; 92% vulnerabilities start patching within 30 days. The implications for this data are that while software vendors generally respond promptly to disclosures, patch deployment exhibits a long tail so that exploits are generally effective even if not zero-day.

The third part looked at updating mechanisms. It determined that there is considerable difference among updating mechanisms. For example, prompt for download is marginally more effective than manual updates. Auto-download and prompt for install is nearly as effective as silent updates for patching 50% of vulnerable hosts, but less effective for reaching 95% patch completion.

The fourth portion of the research looked at what users say they do. The team conducted online survey and interviews in summer 2014 with good demographic and sampling methods. Then the surveys were statistically analyzed to determine human factors in updating patches. 70.3% of survey respondents felt it is critical to keep software up to date and nearly half of survey respondents updated for security or to fix bugs/enhance performance, but over 1/3 survey respondents felt there were too many updates. Respondents also had clear expectations about patches. They were critical of unexpected changes, especially to the user interface (UI), want to know what has been changed, fear destabilization and incompatibility, and showed specific preferences for patch installation. 42% of survey respondents preferred automatic downloads while 72% of survey respondents preferred manual installation.

Next Steps for the project include completion of collaborative work, continued empirical analysis of WINE data, user studies extended to system administrators and developers and design of improved information about updates, and modeling attacker and defender behavior using game theory and WINE data.

Articles listed on these pages have been found on publicly available internet pages and are cited with links to those pages. Some of the information included herein has been reprinted with permission from the authors or data repositories. Direct any requests via Email to SoS.Project (at) SecureDataBank.net for removal of the links or modifications to specific citations. Please include the ID# of the specific citation in your correspondence.

Science of Secure and Resilient Cyber-Physical Systems

Xenofon Koutsoukos, Vanderbilt University

Network and systems resilience is a critical element in maintaining functionality after an attack. Professor Kousoukos presented on a new initiative separate from the Lablets, but on the related areas of resiliency and security. System Science of SecUrity and REsilience for Cyber-Physical Systems (SURE) is a collaboration among MIT, Hawai'I, California-Berkeley, and Vanderbilt to improve scientific understanding of resiliency, described as having the attributes of functional correctness by design, robustness to reliability failures or faults, and survivability against security failures and attacks. Water distribution and traffic control architectures were offered as examples of the types of cyber physical systems to be examined.

The research problems and questions SURE will address include risk analysis and incentive design, resilient monitoring and control, decentralized security, integrative research and evaluation, and formal reasoning about security in cyber-physical systems. Some of the research questions SURE will address include:

How can the collection of agents in CPS deal with strategic adversaries?

How can strategic agents contribute to CPS efficiency and safety, while protecting their conflicting individual objectives?

What are the control architectures that can improve resilience against intrusions and faults?

What types of dynamics can provide inherent robustness against impacts of faults and cyber-attacks?

What are the physics-based invariants that can be used as "ground truth" in intrusion detection?

How can we design systems that are resilient even when there is significant decentralization of resources and decisions?

How do we formally and practically reason about secure computation and communication?

How do we integrate and evaluate cyber & physical platforms and resilient monitoring & control architectures?

How do we interface and support human decision makers?

The research challenges facing the team include such problems as spatio-temporal dynamics, multiple strategic interactions with network interdependencies, inherent uncertainties in both public & private systems, and tightly coupled control and economic incentives.

Articles listed on these pages have been found on publicly available internet pages and are cited with links to those pages. Some of the information included herein has been reprinted with permission from the authors or data repositories. Direct any requests via Email to SoS.Project (at) SecureDataBank.net for removal of the links or modifications to specific citations. Please include the ID# of the specific citation in your correspondence.

Security of Networked Cyber-Physical Systems: Challenges and Some Promising Approaches

John S. Baras, University of Maryland

Networked cyber-physical systems are becoming ubiquitous. Aircraft, automobiles, and other transportation systems are one example. This proliferation is generating new ideas about security, such as the Trusted Platform Module (TPM) and smarter smart cards. With much at stake, since there are huge commercial markets in this realm, the speaker cautions wariness in the "fusion" of evidence; more granularity may suggest different problems.

Security, resilience, and safety must be linked together. Scalability will derive from compositionality--the ability to develop security and resilience from parts and subassemblies whose security and resilience are already know and which are then linked into networked systems.

Articles listed on these pages have been found on publicly available internet pages and are cited with links to those pages. Some of the information included herein has been reprinted with permission from the authors or data repositories. Direct any requests via Email to SoS.Project (at) SecureDataBank.net for removal of the links or modifications to specific citations. Please include the ID# of the specific citation in your correspondence.

Static Dynamic Analysis of Security Metrics for Cyberphysical Systems

Professor Mitra presented an overview of the UIUC Lablet Science of Security research project targeting security metrics. He described the modeling framework, the approach for modeling adversaries, and security metrics. Then he described two foundational problems related to analysis of metrics: reachability for nonlinear hybrid systems and algorithms and lower-bounds on the cost of privacy in distributed control. Next he presented findings on the application of their reachability-based algorithms in analyzing a parallel landing protocol and a modular cell-pacemaker network. The analysis brings together simulation-based analysis and ideas from input-to-state stability---a composition theorem from control theory. He presented ongoing work on synthesizing controllers for CPS with adversary attacks.

The project goal for the Static-Dynamic Analysis of Security Metrics for Cyber-Physical Systems Research project was to address the hard problems of predictive security metrics and scalability and composability. Objectives were to identify security metrics and adversary models and develop theory, algorithms and tools for analyzing the metrics in the context of those adversary models.

Metrics were developed to address both physical systems to CPS and included safety factors, margin of safety, reserve capacity and their association with availability, the stability envelope, safety margin, and vulnerability level Adversary models looked at access, including actuator intrusion, sensor jamming, and malicious programs, their energy, and whether they were opportunistic, curious, focused, or committed.

Their work confirmed that Static-Dynamic Analysis is a sound and relatively complete algorithm for analysis of nonlinear -nondeterministic models. Symbolic simulation of adversary-free system is an improvement over approximation of leverage. Their method has been effective in enabling them to synthesize controllers and attack strategies and to measure vulnerability of states with regard to attacks.

Articles listed on these pages have been found on publicly available internet pages and are cited with links to those pages. Some of the information included herein has been reprinted with permission from the authors or data repositories. Direct any requests via Email to SoS.Project (at) SecureDataBank.net for removal of the links or modifications to specific citations. Please include the ID# of the specific citation in your correspondence.

Survey on Resilience

Ravi Iyer, UIUC

This discussion session addressed the "ability to sustain damage but ultimately succeed." The related objectives of resilience and security are to face threats directly while maintaining critical functions. The theoretical basis for addressing this problem is coming from control theory. It is related strongly to the hard problem of security and composability.

The participants addressed approaches and the need for more collaboration and communication among academics researchers to have a more structured and longer term discussion.

Articles listed on these pages have been found on publicly available internet pages and are cited with links to those pages. Some of the information included herein has been reprinted with permission from the authors or data repositories. Direct any requests via Email to SoS.Project (at) SecureDataBank.net for removal of the links or modifications to specific citations. Please include the ID# of the specific citation in your correspondence.