Archiving PF Firewall Logs

In the previous installment of this series we learned how to fine-tune the process of saving and rotating pf logs to match our preferences. Today we'll look at the problem of automating the transfer of logs from the firewall to one of the workstations connected to the internal private network segment. But, you may ask, why won't we analyze pf logs on the firewall instead? Well, while we could analyze the logs on the firewall, it is usually more convenient, efficient and safe to do it on another computer with a faster processor, more memory, and larger hard disks. We shouldn't really ask the firewall to do anything more than packet filtering. The task of log archiving and analysis is best left to another computer.

The monitoring workstation need not be connected to the internal private network segment. Logs can be sent to a trusted machine located on the outside of our network, although that is less common in small to medium-size networks. Some larger networks connecting computers at more than one physical location, though within the same organization often send logs to one central location for analysis, but still keep the copies of logs archived on-site as a security precaution.

Many administrators managing several client's networks from a single location, send logs to a single machine located inside the administrator's own network. Although this might seem a little unwise from the point of view of security, if strong encryption is used and the administrator follows all basic security precautions, the chances of hackers getting at the precious data is very low. One important drawback of sending logs outside your own network is the fact that they consume a lot of bandwidth, and bandwidth costs money.

Which method of archiving logs is best for you?

The choice of the method for remote logging and archiving depends on many factors, but in most cases, it will all boil down to the security policy of your organization, the amount of time and money you can devote to overseeing and monitoring the whole setup, as well as what you feel is most convenient to you. But in any situation, it is highly advisable that you choose a method that is the easiest to build, manage and secure within the constraints of law, money and time.

There are literally dozens of solutions for archiving logs and we cannot possibly describe them all in this article, because the exact details of every configuration are highly dependent on each site's security policies and the personal preferences of its administrators. However, since we are using OpenBSD, we can assume that certain things are going to look and work the same on most OpenBSD installations. Software tools like pf, newsyslog, or syslogd are available on all OpenBSD 3.0 (and later) systems; similarly ssh and Perl are also standard fare on all free and commercial UNIX derivatives (Linux, BSD, Mac OS X, etc.).

Figure 1: Monitoring station connected to the internal private network segment. (You can click on the figure to open a full-size view.)

For the sake of simplicity we'll use a network split into two segments: private network and DMZ (Figure 1), both protected by the same firewall, which will automatically collect and send logs to one of the workstations connected to the internal private network segment. (The details of the configuration of such network design were discussed in part 1 and part 2 of this series, and can be easily adapted to other network configurations.)

This solution is convenient and cost-effective, because the administrator can log into the monitoring station from other hosts and can use the same workstation for doing things other than log collection, archiving, and analysis. Also, because the monitoring station is connected to the network, which itself is connected to the outside world, upgrades of the station's operating system and tools are quick and easy. (The monitoring station need not be running one of UNIX variants, but our job will be much easier, if it does.)

Figure 2: Monitoring station connected to a separate network segment. (You can click on the figure to open a full-size view.)

There is a lot to like about this configuration, but the administrator should be careful to not overload the internal network with additional traffic generated when logs are transmitted from the firewall to the monitoring station. If our firewall is set up to log all traffic going in and out of the private segments and the DMZ segment, then sending the logs to the workstation connected to the internal private segment of your network (Figure 1) more than doubles the amount of data transferred in that segment. When the network becomes congested, there are two things we can do:

if you are still using 10Mb cards, replace all of them with faster 100Mb or 10/100Mb cards; or

connect the monitoring station to the firewall via a separate interface, this requires another network card added to the firewall machine (Figure 2).

If you want to connect the monitoring station via a separate interface, you will need to do the following things:

add another network card to the firewall, a cheap second-hand 10Mb (compatible with OpenBSD, check the OpenBSD Platforms page) will probably be quite enough, but if you can spend 5-10 bucks more, use a 100Mb or 10/100Mb card as it certainly is more future-proof;

connect the monitoring station to the firewall via a cross-over cable (no hub is required, because there are only two interfaces on that network segment: one in the firewall, and one in the monitoring station);

add a new set of filtering rules for the new network segment (you can copy the rules for the private network segment, changing the segment IP numbers);