Security Onion – from traffic to analyst

In the past months I’ve been usingSecurity Onion in relation to one of my school projects and lately to my internship. Security Onion has a lot of useful programs, on which you could literally spend days to configure to work properly on the same server. The fact that it just works, does not save you the headaches of figuring out how it works and tweaking it to suit your needs. (but it saved me from a lot of headaches caused by other issues).

In order to understand how it works I had to get the big picture in mind, and there’s no better way to do that than by a visual representation followed by an appropriate description. The present article will show and explain the roadmap that Security Onion creates between the network traffic that it’s monitoring to the user that analyzes and classifies the data.

Legend:

blue boxes – traffic capturing/analysis programs

geen boxes – user interfaces

yellow boxes – “intermediate” programs

circles – data formats

cylinders – databases

Description:

In order to understand how the system is working I’m going to split it down to 4 descriptions, related to the specific programs and the “chain” that they’re involved in.

Snorby – is a web interface built in ruby on rails that shows a nice overview of Snort alerts. It can be accessed by default on HTTPS on por t 444

snort processes the network traffic, saves some usage info in a snort.stat log file (/nsm/sensor_data/HOSTNAME-IFACE/snort.stats) and outputs the results in unified2 format (/nsm/sensor_data/HOSTNAME-IFACE/snort/snort.unified2.TIMESTAMP)

the unified2 log files are being processed by Barnyard2 which passes the information to syslog-ng, the Snorby database and to Sguil

Snorby uses the info from the Snorby database to represent it visually

ELSA – is a collection of perl scripts which uses syslog-ng, patterndb, sphinx search and mysql to gather massive amounts of logs and to query through them (really fast btw). It can receive logs from multiple location and in Security Onions case it receives logs from BRO, Snort and Syslog

the snort information that has been passed by Barnyard2 to syslog-ng is being processed by ELSA in realtime

BRO processes the network traffic and outputs the result in log files (/nsm/bro/logs/current/TYPE.log)

the log files location is defined as a log source in syslog-ng for ELSA, and ELSA processes them directly as they are being saved to disk by BRO

ELSA writes to the ELSA database, where it stores usage information (query history and others)

Sguil and Squert – these 2 work hand-in-hand, as Squert is a web-interface for Sguil which provides access to realtime events, session data, and raw packet captures

the snort information that has been passed by Barnyard2 to Sguil is being shown in Sguils GUI

Sguil writes the information into the Sguil database

Squert uses the information in the Sguil database to display information in the web-interface

Other

netsniff-ng saves the network traffic from the monitoring interface to pcap files (nsm/sensor_data/HOSTNAME-IFACE/dailylogs/DATESTAMP) that have a a default size of 150MB

capME is being by Snorby to get from a specific alert to the traffic that generated it