GAO: Feds' Einstein Program Comes Up Short

The U.S. government's intrusion detection and prevention program known as Einstein has limited ability to detect breaches of federal information systems, according to a new Government Accountability Office report.

"It doesn't do a very good job in identifying deviations from normal network traffic," says Gregory Wilshusen, the GAO director of information security issues who co-authored the audit of the Department of Homeland Security's National Computer Protection System, or NCPS, which includes Einstein.

Einstein comes up short, according to the report, because it relies on known signatures - patterns of malicious data - to identify intrusions rather than a more complex anomaly-based approach, which compares network activity to predefined "normal behavior" to identify deviations and identify previously unknown threats.

Citing DHS documents and officials, GAO says the DHS always intended the NCPS to be a signature-based intrusion detection system. "By employing only signature-based intrusion detection, NCPS is unable to detect intrusions for which it does not have a valid or active signature deployed," Wilshusen says. "This limits the overall effectiveness of the program."

Overview of NCPS Intrusion Prevention Capability Design

Credit: GAO analysis of DHS data

NCPS is an integrated "system of systems" that delivers a range of capabilities, including intrusion detection, analytics, intrusion prevention and information sharing to federal civilian agencies.

The federal government has spent $1.2 billion from fiscal year 2009 to 2014 on the NCPS system, designed to protect civilian agencies, according to a GAO analysis of unaudited DHS expenditures. For the current fiscal year, DHS requested $480 million for network security deployment to safeguard government networks.

Mark Weatherford, a former DHS deputy undersecretary for cybersecurity, questions whether the government is getting value from its protection systems investments, especially Einstein. "Expectations are that Einstein is this magic pill that will fix everything cyber, and that's just not the case," says Weatherford, chief cybersecurity strategist at the data center security provider vArmour. "... You probably can get something several orders of magnitude less than the cost of that to do the same thing."

Agencies Use Commercial Tools

Indeed, the GAO audit points out that some agencies use commercially available detection and prevention systems, which likely include more signatures than Einstein. "The agencies' intrusion detection systems would be able to compare their network traffic against a larger set of potential exploits," Wilshusen says.

The audit also reveals that NCPS does not evaluate all types of network traffic. For instance, officials tell GAO that no signatures exist with NCPS that would detect threats embedded in some types of network traffic. "Without an ability to analyze all types of traffic, DHS is unable to detect threats embedded in such traffic and [that] increases the risk that agencies could be negatively impacted by such threats," Wilshusen says.

Gregory Wilshusen discusses the findings of a GAO's audit of DHS's National Computer Protection System.

GAO finds that DHS has yet to develop most of the planned functionality for NCPS's information sharing capability, and requirements were only recently approved. Also, the audit says agencies and DHS did not always agree about whether notifications of potentially malicious activity had been sent or received, and agencies gave mixed reviews about the usefulness of these notifications. DHS did not always solicit - and agencies did not always provide - feedback on them.

Adoption Varies

Federal agencies' adoption of NCPS varies widely. The 23 agencies required to implement the intrusion detection capabilities had routed some traffic to NCPS intrusion detection sensors. but only five of them received intrusion prevention services, the audit says.

GAO says agencies have not taken all the technical steps needed to implement the system, such as ensuring that all network traffic is being routed through NCPS sensors. This occurred in part because DHS has not provided network routing guidance to agencies. "As a result," Wilshusen says, "DHS has limited assurance regarding the effectiveness of the system."

A DHS official says the NCPS program office is incorporating information from the department's continuous diagnostic and mitigation initiative to help develop new intrusion-detection and prevention capabilities. The program office is partnering with the General Services Administration to ensure DHS cybersecurity requirements are incorporated into future network services contracts.

Developing Metrics

The official says DHS is developing metrics to clearly measure the effectiveness of NCPS's efforts, including the quality, efficiency and accuracy of actions related to detecting and preventing intrusions.

GAO offered nine recommendations for improving NCPS, including determining the feasibility of adding functionality to detect deviations from normal network behaviors as well as scan more traffic. DHS concurs with all of the recommendations, "many of which are already being addressed as part of ongoing efforts to improve system functionality and customer satisfaction," a department spokesman says.

Operation Success!

Risk Management Framework: Learn from NIST

From heightened risks to increased regulations, senior leaders at all levels are pressured to
improve their organizations' risk management capabilities. But no one is showing them how -
until now.

Learn the fundamentals of developing a risk management program from the man who wrote the book
on the topic: Ron Ross, computer scientist for the National Institute of Standards and
Technology. In an exclusive presentation, Ross, lead author of NIST Special Publication 800-37
- the bible of risk assessment and management - will share his unique insights on how to:

Understand the current cyber threats to all public and private sector organizations;

Develop a multi-tiered risk management approach built upon governance, processes and
information systems;