Preempt Blog

The latest insights and advice to keep your company protected from insider threats and breaches.

Getting to the Root of Security Alert Fatigue

Something is very wrong in the security industry and “security alert fatigue” is one of the most obvious symptoms. Most enterprises generate far more security alerts than their security staff can analyze. Typically it’s not even close. The problem extends to all industries, but a recent survey of banking security leaders brought the issue into sharp focus. The study found that 61% of the organizations generate at least 100,000 events per day. 37% of organizations generated more than 200,000 events per day. That is simply too many events to process even for the largest of security teams. This shouldn’t be the norm, but virtually anyone who works in security can attest that it is. So let’s take a looks at why this is happening and what we can do to fix it.

Humans Working for the Machines

If we look at the heart of the problem, it boils down to this – security products are creating data, not value. More often than not, a person is required in order to turn security data into value. A security product may find a potential threat or an anomaly, but a human analyst still has to verify the issue, understand the impact, and decide what to do. As a result, we have created a situation where machines are mass-producing manual work for humans. This is backwards whether it occurs in a security team or a physical factory. It is simply a broken process.

Growing Out of Automation Adolescence

The problem is that the security process is only half-automated. Today the process of detecting a threat is highly automated. Security products dutifully generate security events and fill up colorful dashboards with data. But detecting an issue is only the beginning of a solution, and soon teams are drowning in data and dashboards with no real solution in sight.

To get back on a level playing field we have to complete the automated process. This may sound futuristic, but it isn’t. Good information security solutions have always been automated. A firewall, for example, can analyze thousands of sessions per second, evaluate them against policy, and take action. But if a firewall only evaluated network traffic, but then relied on a human to pull a lever to allow or deny, the process would never work. We couldn’t hire enough staff to pull levers. It sounds farcical, but this is almost exactly what we ask of security staff today when chasing countless alerts.

Modernizing Policy

The good news is that this analogy gives us a solid blueprint for how we can get better. What the firewall has is this – an automated policy that arrives at hard answers and takes action on them. Cybersecurity often lacks all of these elements. Alerts are typically not definitive, so the policy reverts to manual analysis and manual remediation.

To solve this we need to take a fresh look at policy and how it can be automated. Even though most security events will not be definitive, our policies can enrich those alerts by asking automated questions. Consider a situation where a user is accessing an unusual application. That alone is probably not be enough to warrant a block, but by policy we could automatically trigger a multi-factor authentication challenge. Based on the result, we can automatically go from a low-confidence alert to a high-confidence alert that we can take action on.

Next we can further refine the policy with a wide variety of context. What privileges does the user have? What is the value of the application being accessed? Is the connection coming from an unmanaged device? Or remotely over a VPN? Are there any other risky behaviors associated with the user or the device?

And as we get more fine-grained with our policies, we must likewise get more fine-grained with our responses. We can always block, but a modern policy should be able to automate more than that. By policy we can trigger additional authentication challenges or temporarily demote a user to restrict privileges without disrupting basic services. We can set authorizers where staff can quickly approve or deny a request.

These are just a few examples, but the important point is that we get back to automating policies that deliver appropriate actions. The good news is that we have the tools to make this change today. And as the rate of alerts continues to accelerate, how well we make this change will determine if we end up working for the machines, or if we put them back to work for us.