According to the Breach Level Index, between July and September of this year, an average of 23 data records were lost or stolen every second – close to two million records every day.1 This data loss will continue as attackers become increasingly sophisticated in their attacks. Given this stark reality, we can no longer rely on traditional means of threat detection. Technically advanced attackers often leave behind clue-based evidence of their activities, but uncovering them usually involves filtering through mountains of logs and telemetry. The application of big data analytics to this problem has become a necessity.

To help organizations leverage big data in their security strategy, we are announcing the availability of an open source security analytics framework: OpenSOC. The OpenSOC framework helps organizations make big data part of their technical security strategy by providing a platform for the application of anomaly detection and incident forensics to the data loss problem. By integrating numerous elements of the Hadoop ecosystem such as Storm, Kafka, and Elasticsearch, OpenSOC provides a scalable platform incorporating capabilities such as full-packet capture indexing, storage, data enrichment, stream processing, batch processing, real-time search, and telemetry aggregation. It also provides a centralized platform to effectively enable security analysts to rapidly detect and respond to advanced security threats.

The OpenSOC framework provides three key elements for security analytics:

Context

A mechanism to capture, store, and normalize any type of security telemetry at extremely high rates. OpenSOC ingests data and pushes it to various processing units for advanced computation and analytics, providing the necessary context for security protection and the ability for efficient information storage. It provides visibility and the information required for successful investigation, remediation, and forensic work.

Real-time

Real-time processing and application of enrichments such as threat intelligence, geolocation, and DNS information to collected telemetry. The immediate application of this information to incoming telemetry provides the greater context and situational awareness critical for detailed and timely investigations.

Centralized Perspective

The interface presents alert summaries with threat intelligence and enrichment data specific to an alert on a single page. The advanced search capabilities and full packet-extraction tools are available for investigation without the need to pivot between multiple tools.

During a breach, sensitive customer information and intellectual property is compromised, putting the company’s reputation, resources, and intellectual property at risk. Quickly identifying and resolving the issue is critical, but, traditional approaches to security incident investigation can be time-consuming. An analyst may need to take the following steps:

Review reports from a Security Incident and Event Manager (SIEM) and run batch queries on other telemetry sources for additional context.

Research a network forensics tool with full packet capture and historical records in order to determine context.

Apart from having to access several tools and information sets, the act of searching and analyzing the amount of data collected can take minutes to hours using traditional techniques.

When we built OpenSOC, one of our goals was to bring all of these pieces together into a single platform. Analysts can use a single tool to navigate data with narrowed focus instead of wasting precious time trying to make sense of mountains of unstructured data.

No network is created equal. Telemetry sources differ in every organization. The amount of telemetry that must be collected and stored in order to provide enough historical context also depends on the amount of data flowing through the network. Furthermore, relevant threat intelligence differs for each and every individual organization.

As an open source solution, OpenSOC opens the door for any organization to create an incident detection tool specific to their needs. The framework is highly extensible: any organization can customize their incident investigation process. It can be tailored to ingest and view any type of telemetry, whether it is for specialized medical equipment or custom-built point of sale devices. By leveraging Hadoop, OpenSOC also has the foundational building blocks to horizontally scale the amount of data it collects, stores, and analyzes based on the needs of the network. OpenSOC will continually evolve and innovate, vastly improving organizations’ ability to handle security incident response.

We look forward to seeing the OpenSOC framework evolving in the open source community. For more information and to contribute to the OpenSOC community, please visit the community website at http://opensoc.github.io/.

Even as the latest breach headline fades away, we all know there is another waiting in the wings (read Part I of my blog). How can organizations protect themselves? There is no panacea for securing a payment environment, and implementing advanced technology alone will not make an organization compliant with the Payment Card Industry (PCI) Data Security Standard (DSS). The PCI DSS provides a solid foundation for a security strategy that covers payment and other types of data, but overall security does not begin and end with PCI compliance. Therefore, an organization’s security strategy should employ best practices and an architecture that will not only facilitate PCI compliance, but also help secure the cardholder environment, prevent identity theft, reliably protect brand image and assets, mitigate financial risk, and provide a secure foundation for new business services.

Last weekend was a typical one, nothing out of the ordinary: errands, science fairs, softball practice with the kids. However, I found myself hesitating a number of times, thinking twice, before I handed my credit card to the cashier at the mall for to purchase a pair of shoes and again as I typed in my credit card number and security code online to purchase some items for a school fund raiser. In the past, I hadn’t given this much thought, but with yet another data breach in the news, it seems that the breaches are continuing to occur – and as consumers, we will continue getting those ‘Dear John’ letters informing us we were one of the unlucky ones…

With news of another data breach of up to 1.5 million credit and debit cards compromised last month as well as high-profile data attacks against the International Monetary Fund, National Public Radio, Google and Sony’s PlayStation Network, data security should be top of mind to all of us. So, how are these breaches continuing despite all of the efforts to secure customer data? In a series of blog entries to follow, we’ll outline the anatomy of a data breach, steps you can take to reduce your risk, and how Cisco can help keep your organization from being the topic of the next breach headline.

Anatomy of a Data Breach:

It used to be that hackers were in the business of hacking for fame or infamy… mostly individuals or groups of friends were doing small-time breaches, leaving digital graffiti on well-known websites. Although these breaches demonstrated security gaps among those affected, there was little financial impact compared to today. It should come as no surprise in a world of big data, that it is harder than ever for organizations to protect their confidential information. Complex, heterogeneous IT environments make data protection and threat response very difficult.

You’re out on a relaxing evening at a public place. Maybe you’re having dinner with your family, or maybe you’re having drinks with some coworkers after a long day. As you’re reaching for your wallet, your heart suddenly drops into your stomach as you notice that your wallet is not there. You pat your pockets or check your purse and subject yourself to looking in ridiculous places, but your wallet is nowhere to be found. Your driver’s license, credit cards, insurance ID, all of the data that’s crucial to your life is suddenly gone from you.

Some of the individuals posting to this site, including the moderators, work for Cisco Systems. Opinions expressed here and in any corresponding comments are the personal opinions of the original authors, not of Cisco. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Cisco or any other party. This site is available to the public. No information you consider confidential should be posted to this site. By posting you agree to be solely responsible for the content of all information you contribute, link to, or otherwise upload to the Website and release Cisco from any liability related to your use of the Website. You also grant to Cisco a worldwide, perpetual, irrevocable, royalty-free and fully-paid, transferable (including rights to sublicense) right to exercise all copyright, publicity, and moral rights with respect to any original content you provide. The comments are moderated. Comments will appear as soon as they are approved by the moderator.