Big data security context

I just finished up a lengthy tour through Latin America and Asia, as described in many of my latest blogs. Most recently I was in Australia and New Zealand (ANZ). I had the opportunity to work with various government agencies, organizations within critical infrastructure and general enterprise businesses across ANZ. Their primary topic of interest: big data. More specifically, they were interested in determining what needs to be part of a successful big data security strategy.

Years ago some organizations throughout ANZ viewed cyber security in the same way they viewed physical security in response to nation-state threats. Because ANZ has a land and sea gap physically separating them from other countries, there was a feeling of separation and protection from the nefarious activities that might be happening around the world. Of course others realized, as almost all do today, that cyber attacks have grater range than a jet fighter or ICBM regardless of whether they're perpetrated by nation-states, cyber criminals or activists. To address this issue, organizations are trying to optimize their use of big data security by letting the machines do the heavy lifting and allowing the humans to manage by exception.

Big Data

Big data has already proven its value outside of security across many areas such as space exploration, sports, retail and insurance.

When we think of big data, it doesn't get much bigger than space. Big data analytics have lead to corrections--rest in peace Pluto--and countless discoveries such as:

Consider the value it afforded baseball as portrayed in the 2011 movie Moneyball. We've moved from just relying upon visceral reactions by scouts and gut feel to also incorporating math and science.

Think of your latest online shopping experience. Chances are that the webpage the retailer displays to you has been customized for your interests based on a variety of factors ranging from age and gender to purchase history and geography. And consider how this experience will mature with mobile devices, the Internet of Things, and apps when you visit a brick-and-mortar establishment or drive past a location that has a sale on a brand you like and as such you are alerted with a map, item photo, sale price, inventory, etc.

Finally, remember when getting car insurance was a few simple questions like make, model and year of vehicle, driving record and age? Now questions include marital status, number of children, your highest level of education and home ownership, because they can be measured against a statistical model to help develop a risk score and ultimately determine what you should be charged.

Big Data Security

Before the term "big data" became common nomenclature in the security industry, there was a trend largely inspired by SIEM and log management solutions. This trend resulted in the mass collection and storage of log data. This helped placate auditors and make storage vendors a lot of money, but without capabilities like threat intelligence feeds, automation, analytics such as correlation, anomaly detection, pattern discovery and prioritization, their effectiveness was limited. Simple collection and storage isn't enough. Today, with big data being measured at levels never before operationalized, such as the Yottabytes of storage that some military-level data centers are being built to handle and the Undecillion IP addresses in IPv6.

Perhaps the most important variable, so that security can be managed by exception in the face of staggering data volumes, velocity and variety, is context.

Big Data Security Context

Folks I spoke with in ANZ want to move beyond thinking of data, regardless of that data being logs, alerts, packet captures, metadata, flows, threat feeds, malware detonation outputs and the like, in terms of what they can collect and store. They want to automatically extract value from it. They want machines to:

More simply put, they want to have context delivery automated so security analysts are given a prioritized list of "stories" to review as opposed to some sentence fragments that they need to piece together.

Reality

In ANZ--and frankly everywhere in the world--deriving this level of context is a bit of a utopia at least today. All the pieces of the puzzle are being provided at some level by disparate solutions. Some of these solutions are even integrated. But having a unified, inclusive solution made up of all the necessary best-in-breed technologies that's scalable and effective and will allow security analysts to truly mange by exception is still a ways off but certainly worth striving for.

As organizations begin to embrace big data security, or are already starting to tune their program, context must be at the core of the requirements list. Without context, the simple math of the problem will introduce far too much complexity to be of value and big data security will become be a big waste.

Latest Videos

Hear from Invictus Games Sydney 2019 CEO, Patrick Kidd OBE and Head of Technology, @James-d-smith -share their insights on how they partnered with Unisys to protect critical data over an open, public WiFi solution.

With so much change all the time, how can executives best prepare their businesses to meet the security challenges of the coming years? CSO Australia, in conjunction with Mimecast, explored this question in an interactive Webinar that looks at how the threat landscape has evolved – and what we can expect in 2019 and beyond.

According to new research conducted by the Ponemon Institute, Australia and New Zealand have the highest levels of data breaches out of the nine countries investigated. This was linked to heavy investment in security detection and an under-investment in security and vulnerability response capabilities

Copyright 2019 IDG Communications. ABN 14 001 592 650. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of IDG Communications is prohibited.