We had no problem when SIEM monitoring was done in-house. My security team consists of two Level 3 analysts, who are well seasoned and very familiar with our company and have more than seven years of experience in information security. But it was recently decided that SIEM monitoring was something we could offshore. There's no arguing with a decision like that. The cost of offshore services such as this is compellingly competitive, and we've had good experiences with our offshoring of the help desk, network operations and development. Besides, my team wouldn't be laid off but instead freed to do more pressing things.

And so we found ourselves executing the statement of work and creating dozens of operating procedures, an escalation matrix, incident response training materials and a fairly comprehensive incident analysis reporting tool. We conducted countless interviews and endured weeks of knowledge transfers, several overseas trips, negotiations, dog and pony shows and other activities, bringing us to where we are now: disappointed.

The thing about SIEM analysts is that they need to be judicious. Hundreds of devices report their logs to the SIEM, which currently is identifying about 1 billion events per quarter. Clearly, we can't investigate them all, and in fact we average about 1,500 events of interest per quarter. We do far fewer investigations than that, though, since some events that wouldn't be worthy of investigation on their own become of interest when they are correlated with others to show a pattern. But when you're dealing with a billion events every quarter, managing false positives is essential, and that's why it's necessary to properly tune both the SIEM and the analysts.

For example, like most companies, mine is constantly getting scanned. Many years ago, when port scanning was less common, a single incident might have warranted action. Today, though, if we responded to every port scan, we would need an army of analysts. Therefore, we've created rules to trigger only when a port scan exceeds certain thresholds, suggesting that someone is being very persistent, versus just "driving by." So when one of the new offshore analysts began escalating dozens of port scans that didn't meet our pre-defined criteria, we had to spend time tuning the analyst.

Another example had to do with one of the analysts responding to a single FTP "change working directory" alert. This event triggers when someone who has accessed an FTP server attempts to enter more than a certain number of characters when changing to a different working directory. Such activity can be a prelude to a denial-of-service or buffer overflow attack. However, none of our FTP servers are vulnerable to attacks related to misuse of the FTP CWD command. The analyst should have made sure of this before escalating the event.

The problem with false positives, of course, is that my staff, instead of having extra time to respond to real events now that they have been freed from SIEM monitoring, are losing time by having to look into events that don't warrant the attention. And some of these ludicrous security incidents are forwarded to IT operations team members for action. If they see too many of these false positives, they could end up with "the boy who cried wolf" syndrome and begin ignoring incidents. Besides all that, false positives are tracked; when you have a lot of them, management receives the wrong message.

All we can do, it seems, is to hold lots of training sessions and reduce the number of false positives coming from overseas. Just as important, of course, is the need to do that while making sure that the analysts don't miss the events that really do deserve our attention. When you think about it, going too far the other way would be even worse.

This week's journal is written by a real security manager,"Mathias Thurman," whose name and employer have been disguised for obvious reasons. Contact him at mathias_thurman@yahoo.com.

Latest Videos

​Email fraud is nothing new, but online criminals have become ever more-effective at spoofing their identities to trick employees into sending them money. The Australian Centre for Cyber Security (ACSC) recorded losses of over $20M to business email compromise (BEC) attacks last year alone, up 230 percent over the previous year – and the full amount is certain to be much larger.​

No matter how robust your security, or how diligent your employees, network credentials are a free pass for cybercriminals. This is mostly because employees are relied upon for their own password management. And with more than 4.8 billion sets of stolen credentials said to be available online, odds are that at least a few of your employees’ user IDs and passwords are just waiting to be used by unscrupulous outsiders. Are you ready to stop them?

Cyber resilience will be particularly important as Australian organisations face increased pressure to quickly detect, respond to, and manage the repercussions of breaches in the wake of 2018’s Notifiable Data Breaches (NDB) scheme.

Copyright 2018 IDG Communications. ABN 14 001 592 650. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of IDG Communications is prohibited.