Posted
by
timothy
on Thursday December 31, 2009 @04:42PM
from the open-but-not-promiscuous dept.

richrumble writes "The OISF has released the beta version of the Suricata IDS/IPS engine: The Suricata Engine is an Open Source Next Generation Intrusion Detection and Prevention Engine. This engine is not intended to just replace or emulate the existing tools in the industry, but will bring new ideas and technologies to the field. This new Engine supports Multi-Threading, Automatic Protocol Detection (IP, TCP, UDP, ICMP, HTTP, TLS, FTP and SMB! ), Gzip Decompression, Fast IP Matching and coming soon hardware acceleration on CUDA and OpenCL GPU cards."

That is interesting to me. CUDA can easily provide parallelization of bulk mathematical operations, but it's notoriously weak with conditional logic. Are they doing a whole lot of math on the side -- perhaps with some fancy anomaly-detection algorithms that work by clustering packet attributes in multidimensional spaces, or approximate nonnegative matrix factorization, or such?

Maybe I'm missing something, but as someone who has working with the techniques referenced in the parent post - I'm not sure where the funny mod came from. Both clustering packet attributes and nonnegative matrix factorizations could be used for anomaly detection. And as someone who has also worked on CUDA a good bit, I think both of those problems have solutions that fit CUDA's concurrency model.

I get the impression that the mods saw big words and assumed this was a joke about buzzwords, but in fact that's a reasonable approach to this problem.

To be fair, I think this is from 1994. You gotta give Gibson some slack on that one.

If his writing had any redeeming qualities, he'd have earned some slack... but I've just found him to kinda suck. His writing style is torturous reading at best. This holds true in both his newest and oldest work.

While there is some information available on the site, it's still pretty sparse. Is this a whole framework? They refer to engine, but do they mean a detection engine or also a correlation engine? This area really needs more open source innovation, commercial solutions are ridiculously expensive for small / mid sized companies, and the only "complete" IDS option I know of for the moment is Ossim (which has extremly lacking documentation).

From the INSTALL doc:Suricata is compatible with standard Snort rulesets. A sample standard configuration file can be found in the Suricata base directory. This file is called 'suricata.yaml'. I've just now got it installed, going to investigate further...

I thought that the "Open Source Intrusion Detector" spotted intrusions of open source software in the company. I'm sure that Microsoft would have loved to have one of those for the Windows 7 USB/DVD download tool.

I'm not trying to be a troll here or anything, but are IDS/IPS systems actually worth while?

We started using Snort back around 2002 when I worked at a hosting provider and it was one of the biggest waste of resources in the NOC department.

The first issue is that there was no way we were going to inject such a box that could ever modify the packets going through the border routers/switches (no server was fast enough for starters), so that eliminated any "prevention" from happening.

You are. Your IDS was incredibly poorly-tuned, a very common problem in IT. First guideline: turn off signatures for anything that you're not running. It makes no sense to watch your inbound traffic for Windows signatures if you run Apache on RHEL. If all you have are web servers and they do only HTTP, there's no reason to watch for SMTP.

Making the move to IPS is always tricky. You have to figure out what level of false positives you're willing to accept. If it's zero, well, you don't need an IPS. But odds are that you will come across some strange but innocuous traffic that the IPS doesn't like, and it trips a rule and blocks the traffic.

In addition, you need to get the hardware for the solution. A server-based Snort solution works well for low-bandwidth scenarios, but at most hosting providers, you need a dedicated appliance solution built on ASICs. If you like Snort, you go to Sourcefire. Otherwise, you find solutions from McAfee (Intrushield), Tipping Point, IBM, etc. They have boxes that scale into the gigabit-per-second range, with latencies under 1ms in most cases, and there are a few true-10Gbps solutions out there now. Yes, they can be quite expensive, but the low-end systems (essentially highly-tuned servers) can start at as low as a few thousand dollars.

But in any case, rule tuning is an ongoing item, and anyone that tells you that an IDS/IPS will reduce your time requirements is probably trying to get you to sign a contract. It can reduce overall time requirements by alerting you early in the attempt to compromise a system and save you all the time of recovery, but that is not a certain thing.

Say it's HTTP only. What can they do for you?So far have there actually been attacks your IPS systems have _stopped at your sites that would have actually caused problems - wouldn't have been shrugged off by your servers being not vulnerable in the first place (patched, not vulnerable to SQL injection etc).

I'm just wondering how much security and availability they'd really add over just having a firewall (some even have IPS features themselves, but let's ignore that for now).

They can protect in situations involving unpatched vulnerabilities, actually. In many cases, once a vulnerability is publicized, a signature can go out within a few hours, sometimes even within minutes. Whether you add that in, being new, is something for you to decide based on policy. But IDS/IPS is, as you mention, reactive in part, but anomaly engines are getting better.

I come from an environment where we moved from simple port-based firewalls to Sidewinders, which are application proxy firewalls. Th

A lot of companies don't know enough to understand security, some well dressed salesman comes through the door and tells them whatever he's selling will make them more secure so they buy it...

I went to a company where every box was running a commercial implementation of SSH for just this reason, none of their staff actually used it, every box also ran telnet and rlogin and that's what the staff used.They believed that simply by having it there they had improved security, when in fact they had weakened it by

Try the sguil console, and you'll be happier with handling alerts. It presents the data from full content pcaps, Snort alerts, and session data, together with a handy window to to reverse DNS and whois. It will give you the signature that fired the alert, or, if no alert fired (say someone emailed abuse@yourdomain.tld with an IP and time range) you can look back in time and see what connections your host had open when. It will even help you decide which alerts are useful and which are useless, but you st

If it were really that good,
it would sniff the referrer on all
the HTTP requests and throttle Slashdot.

I'm sure this was meant in jest, but it doesn't work that way. They could instantly drop every packet in a DDoS attack and it wouldn't matter. By the time it hits their network the bandwidth is already gone.

Now that you mention it, recognizing the referrer would most
likely be of marginal benefit at best. You
have to SYN all those connections to
do it in the first place, and in a true DoS attack
you probably do have your entire network saturated
with SYNs alone, nevermind data.

In other words, you're right. The IPS
is a doorman; but it can't prevent a crowd from
forming outside the door and preventing the band
from getting on stage.

No, it would help a lot... Most of the bandwidth from being hit by web requests is actually outbound traffic when the server actually responds and tries sending the site content... Also since you would be blocking the first get request, the client would never receive the html content and therefore not try to retrieve any images, css files, javascript etc.

Also, sometimes a site goes down not for lack of bandwidth but because the page is dynamically generated and too complex for its processor to handle so man