Using Big Data Analytics to Detect Cyberattacks

The number and complexity of cyberattacks has been increasing steadily in the last few years. The major players in today’s cyberconflicts are not skilled individuals, but well-organized and heavily funded teams with specific goals and objectives. Adversaries are targeting the communication and information systems of organizations and are willing to expend large amounts of money, time and expertise to reach their goals. It is well-known that current intrusion detection systems (IDS) and security information and event management (SIEM) systems are imperfect at detecting these advanced persistent threats (APTs), as their limitations have been discussed in my recent Journal article. Moreover, adversaries have proven their ability to exploit these limitations to evade detection. The underlying problems are not addressable by writing more accurate signatures or providing more information to correlate; they are structural and require reconsideration of the architectural design of these kinds of cybersecurity applications.

The use of big data analytics for cybersecurity might enable the detection of highly sophisticated attacks that cannot be detected using the current approach of deploying commercial products such as IDS, SIEM and deep packet inspection systems. Currently, a small number of proof of concept deployments that utilize big data analytics for security event detection exist and show promising results. I believe that research on this very promising field needs to be intensified in order to create robust solutions that can address the multidimensional problem of APTs. Such a solution will need to enable the overall management of network traffic and event logs captured at various points of a computer network, including retaining various data sources long-term, providing analytic tools to analyse that data (not necessarily in real time) and providing tools for the generation and deployment of new detection algorithms.

My recentJournal article expands on this concept and justifies how big data analytics can significantly enhance the detection capabilities of the defenders, possibly enabling the detection of APT activities that are passing under the radar of traditional security solutions.