I've seen several recent comments elsewhere questioning the value of IDS. I'd like to what other people think about it.

I've managed an IDS in the past and conducted some IDS research for a former employer, but it's been several years since I did any hands-on IDS monitoring so I feel like I'm lacking a current perspective.

The argument that I've always accepted is Bejtlich's "prevention eventualy fails." I still think that argument is valid and can see the value of monitoring systems, logging, keeping session/statistical data both for detection and response, etc. But, I wonder what value IDS actually gives us.

Consider Snort, let's say we remove all of the signatures that aren't applicable to our environment (e.g. remove Oracle rules if we don't run Oracle), remove all of the rules that are too out of date to matter (e.g. teardrop), and also remove all of the rules for things that we're blocking anyway. Once we do that, how much is really left and what are the odds that, if we do undergo a serious attack, that the remaining rules will alert us to it?

Although prevention eventually fails, the detection systems that we put into place is only valuable if they are able to detect malicious activity when prevention fails. Otherwise, we don't gain any additional security.

It is only that: a tool. By itself without staff who can interpret, other security controls, etc.... it probably would be a paperweight.

I believe a properly configured and maintained IDS used along with other aids adds value to a security program. Ultimately any additional alerts, data, insight, etc... I can get I will be more than happy to take.

My immediate concern isn't managing, tuning, or responding to attacks. Assuming an organization can handle that, what value does the IDS actually bring? What is its capability to actually detect attacks?

I've seen numbers for AV (they aren't reassuring), but not IDS. I'd love to have some hard data or even casual observations from the field as to what various IDS are actually capable of.

Put another way, I'd like to know how many false negatives there are. I want to know how much passes "under the radar". If IDS can only detect 10, 20 or 30% of attacks, then it's not a very valuable tool. If it can detect 70% or more, the benefits become a lot more significant.

An IDS (or IPS, or any other security measure) is only as good as the person / people who configure them. I've seen folks see significant value from theirs, when the right folks set them up, and then CONTINUE to proactively monitor and tune / adjust. Then there are others who complain, but when I look at their configurations, it's obvious why (and under the same configs, I'd be displeased, too.)

Do I rely solely on them? Heck no. But do I feel they CAN be of value / benefit? Absolutely.

~ hayabusa ~

"All men can see these tactics whereby I conquer, but what none can see is the strategy out of which victory is evolved." - Sun Tzu, 'The Art of War'

I've brought this up elsewhere and a common response is that the value depends on the person running it, but the person running it is responsible for tuning the IDS for performance and sifting through false positives. Let's assume tuning is not an issue. I can enable all the rules I want without running into performance problems and, by magic, the false positive will still be low. Will the IDS catch the stuff I really care about? Looking at Snort, it seems to me that a lot of the rules catch older attacks or very generic patterns that are easy to avoid (e.g. 0x90 NOP sleds).

Have you used them to successfully detect any attacks that weren't super-obvious (e.g. a Nessus scan)? Have you had any slip by (that you know of)?

there's too much grey area in what you're asking. The person tuning the IDS/IPS is the one that will catch potentially malicious traffic using a variety of resources including the IDS. You can't rely on any one tool to do the work. Can you tune the device and write custom signatures to help aid in the work and narrow it down to what you want? of course but ultimately the staff is the focus not a specific device.

I'm focused on the existing rule sets, either those available from the IDS vendor or a third-party project such as Emerging Threats. What you can actually detect with the default rule set? Most of what I see in the rule sets is old, easy to avoid, or too noisy (e.g. port scans if you looking at Internet-originating traffic).

My concern is that we don't spend much time evaluating how effective IDS actually is. This has been the case with anti-virus as well although it's easier to find data for that since AV is more widely deployed.

The ultimate answer may be that IDS is extremely effective, but I don't want to assume that. I want to challenge our assumptions to get to something more objective.

I understand the concept of tuning the IDS, turning off rules that generate too many false positives, creating some custom rules to fit local policy, etc. That's all well and good. But, underneath that we're depending on someone else to deliver a base set of rules and capabilities that are supposed to detect malicious traffic and I'm not convinced about the effectiveness of what is being delivered.

unicityd wrote:I'm focused on the existing rule sets, either those available from the IDS vendor or a third-party project such as Emerging Threats.

When just talking about standard default rules I think is where having mutiple IDS devices working together becomes valuable. High level example being maybe you run Cisco IDS/IPS along with something like SecurityOnion on the same network segment to get potentially differing views/alerts of traffic going through to compare against each other.

Well in most cases your IDS and IPS are one in the same. Only major differences is you tell your IPS side to block specific signatures. Other than that you tell them both to ignore traffic related to platforms you are not using. You want them both to log and in some cases you allow certain devices to pass through unhindered. For instance anything going to your honeypot, you might want to allow through without blocking but you definitely want to log it for analysis.

Also it is about placement. My last job we had an IDS/IPS that first was sitting in the rack for like 2 years and all it was doing was passing traffic between the internet and LAN. I had to call support to get the thing updated since no one bothered doing that for the 2 years. So once I got that all squared away I turned turned on the logging to get a baseline and disabled the logging for the platforms I know I didn't have. After that it was a couple weeks of tweaking until I knew I could turn on the IPS part without breaking the network. I was getting some decent traffic, most of which was valid. Then some genius decided to have it moved from the main internet line where most of our high traffic devices where to a secondary line where the only thing that existed was email which was going through a filtered service. Guess what happened.... nothing.

And I was happy to report the plan that the ISO and network engineer implemented failed miserably. Oh and yes, we (my boss and I) did recommend alternatives but there was a trust issue with the support staff due to previous members who were no longer there. I was not sad when they announced they were outsourcing all our jobs, I laughed and gave my notice. Told them where I was going and almost felt like telling them yeah, that's right bitches, going to move into some real shit

A big part of this is going to be how well you baseline and how well you know your environment. The initial attack may be able to circumvent the IDS, but if a database server establishes an outbound HTTPS connection when it has never done that before, you can be sure something abnormal is underway. It may be an attacker, it may be your DBA checking his email because that server isn't subject to web filtering. Regardless, it signifies activity that's probably worth looking into.

This is why you need to correlate IDS with netflow, local system logs, etc., so you can connect the dots and get better visibility into network and systems operations. Just standing up Snort, Proventia, etc. on the perimeter is wholly inadequate.

unicityd wrote:My immediate concern isn't managing, tuning, or responding to attacks. Assuming an organization can handle that, what value does the IDS actually bring? What is its capability to actually detect attacks?

I've seen numbers for AV (they aren't reassuring), but not IDS. I'd love to have some hard data or even casual observations from the field as to what various IDS are actually capable of.

Put another way, I'd like to know how many false negatives there are. I want to know how much passes "under the radar". If IDS can only detect 10, 20 or 30% of attacks, then it's not a very valuable tool. If it can detect 70% or more, the benefits become a lot more significant.

I don't think you're being very fair to IDS. People criticize IDS because they can be bypassed, but really what preventative measure can't be bypassed? Intrusion detection is a lot more than an IDS, just like preventing attacks is a lot more than a firewall. Neither is meant to be the only thing needed for detection/prevention.

My concern isn't that there is some way to bypass an IDS. As you point out, everything can be bypassed, at least in certain circumstances. My concern is that the effectiveness of IDS at detecting malicious traffic is so low that it is not cost effective. Even if IDS only detected 5% of malicious traffic, it would be useful if it were cheap and required little maintenance. But, it costs money to deploy IDS and, as many others pointed out, you need one or more good analysts to run them. For the level of investment that it actually takes to run and maintain and IDS, the benefits need to be greater.

I don't think that IDS is worthless. I just suspect that it's not the best use of scarce resources.

1. You don't have to monitor the entire organization. You can monitor your most important or most vulnerable systems.2. There are around 8000 vulnerabilities found a year, but only around 13 that are commonly exploited [source]. So I think you can get by with a far more conservative ruleset than the default.3. You don't have to investigate and respond to every alert in real time. You could only resond to critical alerts and leave others for forensic purposes.

I wouldn't say an IDS is appropriate for everyone, but an IDS is what you make it. You can make a firewall a nightmare to manage by using a really conservative ruleset where you have to continually make exceptions. You could also make an IDS a nightmare to manage by using a liberal ruleset where you get overwhelmed by false positives. Not many people do the former, but the latter is a common mistake.

Cheap IDS software is not the issue. It still takes time/money to setup sensors and management consoles (hardware/VMs are cheap at least) and a lot more to staff them. Even with a low false positive rate, IDS need love and affection.

IDS may in fact be very valuable. I'd really like to see some objective evidence that it is valuable rather than make the assumption that it is valuable based on a general notion of defense in depth, prevention fails, etc. We ought to have some idea of what capabilities IDS actually provides, how well it detects malicious traffic, etc. What percentage of actual attacks in the wild can IDS detect? What if we exclude the attacks that are easily prevented? How do the capabilities of the IDS vary by attack type (e.g. web app vs client-side web vs botnet CnC)?

There has been some research on the effectiveness of anti-virus and it's not encouraging (act surprised). My impression is that IDS is further behind the curve than AV.