What are we doing here?

What is PSAD?

psad is a collection of three lightweight system daemons (two main daemons and one helper daemon) that run on Linux machines and analyze iptables log messages to detect port scans and other suspicious traffic. A typical deployment is to run psad on the iptables firewall where it has the fastest access to log data.

What is Logstash?

logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs.

Using PSAD and Logstash

Using Logstash, you can process and tag PSAD events (logs) based on their significance. Also, as an alternative to using the PSAD email alerts, you could use one of Logstash' many output filters to send notifications of important events. For example, you could use Nagios to notify you of any port scans PSAD has detected. As the number of servers you install PSAD on grows, receiving email alerts from all of them can become cumbersome and could become a 'boy who cried wolf' situation. Logstash in combination with Kibana will let you chart all port scans and attempted attacks that PSAD can discover in a unified way.

This gist will explain how to configure both PSAD and Logstash to play nice together. This gist is focused on PSAD 2.2.4, Logstash 1.4.2 and Ubuntu Server 14.04; if you're using a different OS or different versions of Logstash/PSAD you may have to tweak the instructions a bit.

This gist assumes you have PSAD and Logstash up and running individually.

First, configure rsyslog

The first thing we need to do is configure rsyslog on Ubuntu to write any PSAD events to a separate logfile. Although it is perfectly possible to parse the /var/log/syslog directly for any PSAD messages, I prefer to write PSAD events to its own logfile; it makes it easier to determine its actions and makes our logstash configuration simpler as well, as we don't have to filter out any other kernel messages.

To do this, we need to tell rsyslog to send all messages that originate from PSAD to a different logfile. Create a file called /etc/rsyslog.d/30-psad.conf. The 30 is added to make sure the settings in this configuration file are parsed before the default settings, which start with 50-.

The first line ensures that any log lines from the application psad are sent to the file /var/log/psad.log. The second line ensures that any lines matching the previous criteria will not be processed by any other filters. This second line is needed to prevent the log lines from also showing up in your /var/log/messages or /var/log/kern.log file.

Note:with older versions of rsyslog, you may have to replace the word stop on the second line with the ~ character; on newer version of Ubuntu the ~ character as a stop indicator has been deprecated and will cause warning messages when (re)starting the rsyslog service.

Then, restart the rsyslog service to enable the changes:

sudo service rsyslog restart

Then, configure logstash patterns

To make processing PSAD messages easier, I have created a set of GROK patterns which should allow you to detect and act upon the various PSAD messages without too much effort.

First, you need to create a logstash patterns file. Depending on your logstash configuration, the location of these pattern files may vary but I store mine in /etc/logstash/patterns/. Here are the patterns:

PSAD_AUTO_BLOCK

PSAD_AUTO_UNBLOCK

PSAD_NOTICE

Used to detect PSAD notice messages; usually generated by PSAD upon starting the service or updating the signatures
Fields: message

PSAD_WARNING

Used to detect PSAD warning messages
Fields: log_level, message

Available log parameters

Here's a list of all paramters which are available for use in Logstash (or elasticsearch, if that's what you're using to store your logs; these are also the fields you can use to create your dashboards in Kibana).

[psad][start_port] / [psad][end_port]

Start and end port of a port scan range

[psad][scan_port]

Port scanned or found in signature match

[psad][scan_type]

Indicates the type of scan, if any

[psad][proto]

Protocol used in scan (TCP/UDP)

[psad][flags]

Any attack flags found (SYN, etc)

[psad][proto_num_pkts]

The number of packets received in the attack/scan

[psad][danger_level]

The PSAD danger lever assigned to the attack/scan

[psad][signature]

The psad/fwsnort signature detected

[psad][signature_id]

The psad/fwsnort signature ID detected

[psad][block_time]

The time an IP address was auto-blocked

[psad][log_level]

The log level of a PSAD warning message

message

A generic message for the logline; varies per type of log line

src_ip

The source IP of the attack/scan

dst_ip

The destination IP of the attack/scan

Note:Why not place the src_ip and dst_ip in the [psad] array as well? Since I use logstash to process almost all my logfiles, I prefer to keep things consistent. This way I can create a separate GEOIP filter to process all fields called src_ip and dst_ip and I don't have to create a separate filter for each logfile.

Next up: logstash input

The logstash input filter will assure that our psad log is parsed by logstash. Here's what it looks like:

input {file {path => '/var/log/psad.log' # the location of our psad logfiletype => 'psad' # assign a type so we can easily process this in the filters}}

Roundup

Now you've configured rsyslog to write your psad messages to a separate file, told logstash where to find this file and what to do with each type of log line. Use your creativity to send your logstash filtered logs to any of the available outputs; I personally use elasticsearch to store my logs and kibana to visualize them. In a next gist I will likely publish my Kibana PSAD dashboard, but I still need to do some finetuning there first.