Garbage Procedure

The garbage attack, or random fragment attack, observes how each firewall behaves against an attack that uses random datatypes on random port numbers. The Netwag program random fragment attack also has the ability to spoof the source IP address.

Open Netwag and select "Flood a host with random fragments."

Check the Destination IP Address checkbox.

Enter the target IP address.

Select Generate It (bottom of screen).

Select Run It.

UDP Ping Procedure

The UDP Ping attack observes how each firewall behaves against ping attacks using UDP. The Netwag program's UDP Ping also has the ability to spoof the source IP address.

Open Netwag and select Ping UDP.

Check the Destination IP Address checkbox.

Check the Destination Port Number checkboxes.

Enter the target IP address.

Enter the target port number.

Select Generate It (bottom of screen).

Select Run It.

TCP Ping Procedure

The TCP Ping attack observes how each firewall behaves against the ping attack using TCP. The Netwag program's TCP Ping also has the ability to spoof the source IP address.

Open Netwag and select Ping TCP.

Check the Destination IP Address checkbox.

Check the Destination Port Number checkboxes.

Enter the target IP address.

Enter target port number.

Select Generate It (bottom of screen).

Select Run It.

Ping of Death Procedure

The ping of death attack observes how each firewall behaves against attacks that send over-sized packets. Our goal was not to determine how many over-sized packets are required to shut down each firewall.

Open a Windows command prompt window.

Enter ping -l 65000 <target_IPaddress>.

Configuration

The aim of this configuration is to simulate a condensed, real-world, corporate network layout. We placed one server on the outside router's external interface to act as the Internet in order to demonstrate how the internal network could gain permitted access to the Internet. The server on the inside network provides the outside world with a specific target. The access list for all three firewalls permits WWW traffic to pass out on port 80 while at the same time allowing for FTP to pass in on port 21. Because the PIX implicitly denied anything not on the access list, we had to create rules to allow these transmissions to pass through. We created the baseline (PIX Firewall) by modifying the Advanced Router Lab's configuration. SmoothWall and openBSD layouts modified this layout even further. Neither software firewall has an inside router in its configuration. We removed the inside router due to issues it caused in allowing connections to the SmoothWall web-based administrative console.

We kept this configuration for simplicity and uniform results in the later configuration and testing of the openBSD firewall.

Test Results

Our test results were interesting.

Cisco PIX Results

Regardless of which port the attack used, with the state full packet inspection activated, the Cisco PIX blocked all transmissions on every test we conducted. The PIX also continued to allow the proper connections that were not considered attacks during the tests. The PIX effectively blocked the outgoing and incoming packets. One of the few issues with the PIX is finding proper documentation. The PIX was designed with a professional support team in mind, not the typical home user.

SmoothWall Express Results

Compared to the PIX, SmoothWall was more simplistic in design and easier to configure, but also less robust. Unlike the PIX, SmoothWall uses stateless packet inspection. Attacks on specific ports locked up the firewall system until the attack stopped. SmoothWall was designed with the home user in mind, not corporations.

The documentation provided for SmoothWall is centered on the web-based GUI, which does a good job detailing how to set up and configure the system. SmoothWall also uses the open source intrusion detection system Snort. One of SmoothWall's problems is that the GUI does not list any outbound packet inspection options. (You can activate stateful inspection by modifying the source code manually.) Another issue is that SmoothWall has a limit of three interfaces: an inside, an outside, and a DMZ.

OpenBSD Results

OpenBSD is everything one might expect from an open source firewall. It has the power and potential of the PIX without the cost. As for performance, OpenBSD performed just as well as the Cisco PIX at blocking unwanted incoming or outgoing packets with no degradation to the system. OpenBSD also kept detailed text logfiles of each attack, which were fairly easy to read. Like SmoothWall, OpenBSD does not provide any type of graphical results analysis of the logged attacks; the PIX does provide this.

The main issue with OpenBSD is that you may require professional support. But without the cost of the hardware PIX, you might consider this an even offset. OpenBSD also does stateful or stateless packet inspections, remembers sessions, and modulates the session to assist with preventing the data connections from being hijacked. Because it is an operating system, OpenBSD has the ability to add Snort or other advanced IDS options. OpenBSD also has the option of creating VPN connections.

When installed, OpenBSD is secure by default. As with SmoothWall, the documentation for BSD is very detailed. However, unlike SmoothWall, the configuration instructions direct you to perform manual command-line operations. OpenBSD is our runner-up firewall. The available interfaces are only limited by the number of possible interface cards you can install in the PC.

Conclusion

The Cisco PIX behaved as expected and is an outstanding choice if cost is not an issue. Cisco's built-in graphical results are effective quick-references for observing the firewall status. For cost effectiveness and features, OpenBSD is an excellent choice. Out of the three firewalls tested, SmoothWall Express is our least preferred because it is the least powerful. However, SmoothWall Express is a good choice for a home-based network. This version is not recommended for business or corporate use. SmoothWall Express is open source; however, it does have a corporate professional edition that we did not evaluate for this project.

Choosing a firewall depends on the needs of your business or network. If you have a large corporate network behind the firewall, it would probably suit your needs to invest in a system like Cisco; however, a small entrepreneur should consider OpenBSD or SmoothWall, depending on the required security level denies and experience level.

When should a corporation consider using one of these three firewalls? The choice depends on its needs. If it wants top of the line defense that provides detailed reports (including graphs), then the Cisco PIX is the best choice. If the corporation needs a good defense but cannot afford the PIX, OpenBSD is an effective, inexpensive choice. Based on the testing results of this project, SmoothWall Express is not an effective option for a corporation.

Small business or home users are most likely unable to afford the Cisco PIX. They are just as unlikely to be able to maintain such a device, assuming they are not a technology-based business or user. Because most small business and home users do not have the money or technical experience to use a PIX or OpenBSD, the best option for them is SmoothWall.

One of the key differences between a corporation and a small business (including home users) is that a corporation stands a high risk of being a target of script kiddies and professional hackers. With a small business, this risk is low, although small business and home users are still at risk. Because of this difference, and in light of maintainability concerns, SmoothWall is a good choice for a small business or home user.