Why Sagan?

In mid 2009, Quadrant Information Security. staff monitored an attacker breaking into a network and modifying logs. The attacker was attempting to "hide" traces of the break-in. Nothing new there. However, the company that had hired us had a centralized logging system. The problem was, it wasn't in real time and the attacker could modify the logs before they were sent to the centralized logging server! In order to retrieve evidence from the centralized logging system, you had to know what you were looking for. For the average administrator, this might not be an easy task.

The thought occurred to build a real time monitoring system that would tell the administrator when "bad things" are happening.

There's already plenty of software to do this type of task. However, most of it is "near real time". That's the type of software the attacker above took advantage of.

The other issue is what to do with the attack data once you detect it? At Quadrant, we're fans of Sourcefire's Snort IDS/IPS (Intrusion Detection/Prevention System) software. Rather than having "Yet another console" to monitor, we decided to put the data into a Snort database as a separate "sensor ID". This has since been been expanded to Unified2 output support, the Prelude framework and many others.

If you don't use Sourcefire's Snort, you probably should. However, even if you don't, don't worry, Sagan supports multiple output formats.

From Quadrant's stand point, Sagan gives us a broad range of devices, services, applications that we can monitor. For example, if your organization is a "Cisco shop" and you don't want to deploy Snort based IDS/IPS sensors, it really doesn't matter to our staff. We can monitor the Cisco devices just as we would a Snort based IDS/IPS solution.

What license does Sagan use?

We believe in open source. Sagan is licensed under the GNU General Public License version 2. You're welcome to test, deploy and use it as you see fit! However, if you need professional Sagan support, 24/7 managed network security services or have other security related needs, consider talking to Quadrant Information Security! You can contact us at 1-800-538-9357 (U.S.A - 24/7).

What operating systems does Sagan run on?

Sagan was primarily developed to run on Unix and Linux based systems. Sagan has been tested on various Linux platforms (Gentoo, Ubuntu) and FreeBSDOpenBSD, etc. Please let us know if you've successfully used Sagan on other platforms. Sagan may compile under Microsoft Windows, but we don't have any plans at this time to port it.

Why the name "Sagan"?

Sagan is named after my dog (a mutt). You can see a picture of her in the top left hand corner of this page. Sagan, the dog, is named after Carl Sagan. One of my favorite teachers/scientists. The software is named after both

Why is Sagan multi-threaded? / Why does Sagan use pthreads?

Mostly to prevent I/O blocking. On some systems, we receive millions of logs per day that need to be checked for "bad things". We cannot afford to "drop" any log entries, so when an event needs to be logged, a "thread" is created. This allows Sagan to continue processing event/syslog data in real time without dropping events.

Sagan version 0.1.8 added support for the Unified2 output format. This means using you can use Barnyard2 with Sagan. Sagan with unified2 output support and Barnyard2 is practically identical to how Snort uses unified2 with Barnyard2. The main advantages of using Sagan with unified2 and Barnyard2 is that the logging processes is seperated from the detection engine. Barnyard2 and the unified2 output format were created to separate the Snort IDS/IPS engine from having to deal with logging. That had a lot to do with I/O blocking. Without separating the processes, Snort would sometimes "drop" packets. This isn't a "Snort" issue, I'm simply explaining the existence of Barnyard2. Since Sagan is multi-threaded, I/O blocking isn't an issue. However, Barnyard2 and Unified2 have other advantages than precenting I/O blocking.

In particular, the ability to "queue events". For example, imagine you're logging to a remote database. If that database where to go down at any point, when an events is triggered, there would be no way for Sagan to store the event. Basically, the event would be "lost". This is not good. With Barnyard reading the Unified2 output, if the database connection is lost, Sagan can continue "recording events" and Barnyard2 will insert them when the database connection is re-established.

Barnyard2 also has the ability to record information in many other formats. Sagan + Unified2 + Barnyard2 == many forms of output. For example, Barnyard2 can write to Microsoft MS-SQL, Oracle SQL, MySQL, PostgreSQL and can use ODBC drivers. Barnyard2 also supports the Prelude Framework and Sguil support.

There are current no plans for Sagan to support the older "Unified" format, only Unified2

For a simple Barnyard2 example configuration, see the Barnyard2HowTo page. For general information Unified2 information, see the SaganHOWTO.

How does Sagan do the event correlation with Snort IDS/IPS events?

It should be pointed out, when using Sagan with a Snort database event's are kept separate from IDS/IPS events. That is, while your IDS/IPS system might be reporting as "sensor ID: #1", Sagan will startup as "sensor ID: #2". This way you can isolate between Sagan events and Snort events without an issue.

Sagan simply logs to the database in a way that if a Snort event correlates with it, it will. That is, just because there might not be a Snort event, doesn't mean Sagan won't log "bad things".

Sagan attempts to correlate events using these methods:

Time stamp - This should be fairly obvious.

Destination TCP/IP address - This being the machine that is being targeted or has a problem.

Source TCP/IP address - The source of the attacker or problem. This could be the machine directly, or the TCP/IP address of the attacker. Sagan attempts to pull the source of the attack or problem when it can.

Destination TCP/IP port - In many cases, we can tie the "service" reporting to a TCP/IP port. For example, we know OpenSSH runs on port 22 (normally), so we log it that way.

Source TCP/IP port: Sagan attempts to pull the remote TCP/IP port information, when it can. However, not all services report this.

Protocol used - For example, if the service being attacked or has a problem is OpenSSH, we can assume the protocol is TCP. So we log the information in the TCP database.

Classification - For example, with Snort a SMTP session with the command "expn root" (smtp.rules) is classified as an "attempted-recon". Sagan can see the same event, and will classify it the same ("attempted-recon").

How does Sagan use "liblognorm" for correlation?

Sagan uses liblognorm to gather information from log messages for better correlation of events. For more information about liblognorm, how it's used, how to compile and install please see the LibLogNorm page.

First off, Sagan can't always get remote TCP/IP address because they aren't always applicable. For example, if you're monitoring a Microsoft Windows environment and Sagan alerts you when an application crashes, there will not be a TCP/IP addresses tied to that event. In that case, Sagan "assumes" that the message it's receiving was generated by that machine.

Network services are a bit difference. For example, let's say you have OpenSSH running on 10.0.0.1 and a centralized logging server at 10.0.0.50. If an attacker attempts a brute force attack from 192.168.0.1 to 10.0.0.1, normally the centralized logging server will see the "source" of the message from 10.0.0.1 with a destination of 10.0.0.50.

.... And that is true, but it doesn't help the network administrator or security staff...

Sagan will attempt to identify the true attacker TCP/IP address, port being targeted, the protocol (TCP/UDP/etc) and in some cases the remote port. This way, the event shows the attacker from 192.168.0.1, and he's attacking 10.0.0.1.

This is useful information you can use. If the service is attempting to resolve (DNS lookup) the IP address of the attacker and put that in the event/syslog data, this needs to be disabled! For example, if your OpenSSH log entries look like this:

Sagan won't like this. There's no way for Sagan to determine the remote system's address. Also, from a logging stand point, you want the TCP/IP address, not what a DNS server "told" you. In this example, it'd mean in your "sshd_config" on the server you have the "UseDNS" flag enabled. Disable it, and your log entries will look like:

This Sagan can deal with! Fortunately, the majority of services report TCP/IP addresses rather than using DNS lookup information.

Another way Sagan can extract useful information is via the "liblognorm" library. This actually works much better than "parse_ip" and "parse_port" in many cases. For more information, please see the LibLogNorm page.

Do I have to run Sagan on all workstations/machines/systems?

You can. It depends on how you want to use Sagan. If you're looking to catch "bad things" happening on say, your personal laptop, you can configure Sagan to run locally. This way, when something bad happens, you can be notified by a popup message (via xmessage) or what not. Building Sagan without MySQL/PostgreSQL/libesmtp saves a considerable amount of RAM and it'll give the desired results for notifying you on the local workstation.

However....

Sagan was originally designed to be run on a secure, centralized log server. This way, all systems within your network (network switches, routers, Windows server, Linux boxes, etc) can send events to a centralized server which Sagan can examine, correlate and warn you of potential bad things happening. This also makes Sagan easier to manage instead of having to manage workstations with Sagan installed, including rule set updates, you have a central point of management.

Sagan deals with log input in two different formats. The first is via a FIFO. When Sagan is in this mode, it'll watch for events through the FIFO. The second mode is via Standard input (stdin) . In the syslog-ng world, this is known as the "program" mode in the syslog-ng.conf file. Syslog-ng will work with both formats.

Rsyslog will work fine with Sagan, however, only in FIFO/named pipe mode. This is typically fine, as it's usually preferred that Sagan run in FIFO mode. For more information about configuration of Sagan with Rsyslog or Syslog-ng, see our SaganHOWTO.

It's likely that Sagan will work with other syslog daemons. If the syslog daemon in question can format the message properly, so Sagan can "read" it, in either program or FIFO mode, then it'll likely work.

If you find another syslog daemon that Sagan works with, please let us know!

No. Sagan performs a different role. Actually, these types of software can work in conjunction with Sagan and you'll benefit from it. Sagan simply identifies "bad things" in your network and has the ability to correlate that data with your Snort IDS/IPS data.

I'm not a Snort user, can I still benefit from Sagan?

Yes. Sagan has multiple output formats, with more to come. See the SaganTODO list for upcoming formats we'll be supporting. Right now, Sagan can e-mail you when "bad things" happen (via libesmtp), create "Snort like" alert log files or call an external program.

Calling an external program is interesting. This gives you the power to write your own routine(s) in the language of your choosing to deal with events how you see fit. When using this function, external calls are threaded. This insures that your program wouldn't be blocking any I/O and everything remains "real time".

This typically means that Sagan is receiving a 'host name' and not an 'IP address'. This means that your receiving server is using DNS, when is shouldn't be. (Hint: "use_dns(no);" in the syslog-ng.conf).

Sagan gives me the message, "Sagan received a malformed (host|date|tag|etc)". Why is that?

You shouldn't see the message very often. When it does happen, it means that Sagan didn't receive a fully formated syslog message. Typically this might happen on startup of Sagan where 'half' a message was received. Sagan will recover from this by filling in the missing information and continuing to the next message. If you are receiving a lot of these messages, then it likely means that a Sagan is receiving improperly formated messages. Look into your configuration and examine the output from the FIFO or syslog program.

Sagan isn't building correct under FreeBSD/OpenBSD/etc. What's the problem?

"If you are a FreeBSD or OpenBSD user, you might need to do the following. In most cases, the required libraries get installed in the /usr/local/lib and include files into the /usr/local/include directory. If you are getting an error in BSD stating the libraries and/or include files can't be found and you are sure they've been installed, try the following. This assumes you're using the 'bash' shell."

How do I use oinkmaster with Sagan rules?

Pretty much the exact same why you use it with Snort rules. However, you'll need to add the "\.rulebase$|" to your update_files (for liblognorm rulebase files). For a simple example configuration file, see the SaganOinkmaster page.

I get "fatal error: libpq-fe.h: No such file or directory" when building with PostgreSQL support. How do I fix this?

This means the PostgreSQL include/header files can't be found. Use the "--with-postgresql-includes" to fix this. For example, "--with-postgresql-includes=/usr/include/postgresql". Make sure you point it to the location of your libpq-fe.h file.

I have a question that isn't covered here, what do I do?

Join the Sagan mailing list and ask! The mailing list is low volume, and by asking in that forum, you might be helping other people with a similar problem.

You can also see if we're on IRC. You can point your favorite IRC client at:

Server: irc.2600.net Channel: #sagan

If you don't have an IRC client installed, you can use the online client.