Minggu, 26 April 2009

Virtual Honeynets

Virtual Honeynets

First discussed in the paper Know Your Enemy: Virtual Honeynets, these solutions havethe advantage of being easier to deploy and simpler to manage. The Honeynet Project has also found VMware tomake an excellent development environment for Honeynet technologies. In this paper, we will take you throughstep-by-step how to build and deploy such a solution using the commercial software VMware. In this case, we willbuild a GenII (2nd Generation) Virtual Honeynet with five different honeypots. It is assumed you have read andunderstand the concepts discussed in both KYE: Virtual Honeynets and KYE: Honeynets. Also, if this is the firsttime you have ever worked with Honeynet technologies, it is highly recommended you work in a lab environment.Last, as with all virtual software, you need to be aware of the risk of attackers identifying, and potentially breakingout of, the virtual environment. You have been warned.Plan of AttackThe format of this paper is similar to KYE: User-Mode Linux, its broken down into five parts. In the first part we willdescribe what VMware is, how it works, and how to install it. In the second part, we describe how to configureVMware and install your honeypots. In the third part we describe how to implement Data Control on your VMwareHoneynet using IPTables. In the fourth part we describe how to implement Data Capture using Snort. Finally, inthe fifth part, we describe how to test your setup.Part I: VMwareVMware is virtualization software that allows you to run multiple operating systems at the same time. Unlike User-Mode Linux, VMware allows you to run different operating systems, as long as they can run on Intel X86architecture. Developed and sold by VMware Inc, there are actually three different software products you canchoose; Workstation, GSX, or ESX. Of the three, we will be using GSX. GSX is more powerful then Workstation,designed to run more then two operating systems at the same time and supports remote administration. However,most of the information discussed here can also be applied to Workstation. For the purpose of this paper, we aregoing to build our Virtual Honeynet on a laptop, specifically an IBM Thinkpad T23, utilizing a PIII 1Ghz processorand 768 MB RAM. The base operating system is Red Hat 7.3.VMware works by installing virtualization software on your computer. This virtualization software then allows youto boot and run multiple operating systems at the same time. The very first operating system you install, the baseOS, is called the HostOS. This is the operating system you install VMware on. Once you have installed theHostOS and VMware, you can then install additional operating systems that run within the virtual environment. Allof these additional operating systems are called the GuestOS's, as they are 'guests' on the Host operatingsystem. To get a better understanding of how this works, refer to Figure 1. Installing VMware on our LinuxHostOS is very easy, you simply install a single RPM package. The command looks similar to this.host #rpm -vi VMware-gsx-2.0.1-2129.i386.rpmPreparing packages for installation...VMware-gsx-2.0.1-2129There are additional software packages we can install, such as the remote administration package. However, ourlaptop does not require this as all administration will be done locally.

Part II: Configuring VMware and Installing your HoneypotsOnce installed, the next step is to configure the VMware software. Configuration is done by executing thecommand 'vmware-config.pl'. During the configuration process, VMware will most likely have to recompile severalof its own kernel modules. This means you need both a compiler and the source code for your kernel. On ourlaptop, we are running kernel 2.4.18-19.7.x. We then confirm we have the source code.host #uname -r2.4.18-19.7.xhost #host #rpm -qa | grep sourcekernel-source-2.4.18-19.7.xmarge $ls -l /usr/srctotal 8lrwxrwxrwx 1 root root 19 Dec 26 13:53 linux-2.4 -> linux-2.4.18-19drwxr-xr-x 17 root root 4096 Dec 26 13:53 linux-2.4.18-19.7.xdrwxr-xr-x 7 root root 4096 Jul 12 11:52 redhatOnce you have the source code installed, you can begin the installation. During the process, the only real optionis Networking, which we want. Remember, the goal is to have all the GuestOS's route through the HostOS, ourgateway. During the installation process, select Networking. Later in the installation process, you will be asked ifyou want HostOnly networking. Select this option also, and give the interface an IP address. This is the gatewayIP address, which we will be using 10.10.10.1. Linked below is the series of commands executed during theconfiguration process.vmware-config.plOnce you have completed the configuration process, VMware should now be running. However, we have oneproblem. During the default configuration, VMware enabled three interfaces; vmnet0, vmnet1, and vmnet8. Ofthose three networking interfaces, we want only one interface, vmnet1. vmnet0 is used for bridging, so theGuestOS talks directly to the network, bypassing the HostOS. vmnet8 is used for NAT Networks. Only vmnet1gives us the control of having the GuestOS's go through the HostOS. Thus, we have to run vmware-config.plagain, then using the editor, remove the two unwanted interfaces vmnet0 and vmnet8.vmware-config.pl (being ran for the second time)Once you have configured VMware, the next step is to install and configure the individual honeypots. For ourHoneynet, we are going to install and run five different honeypots. The requirements for running so manyoperating systems is not as intensive as you may think. Think about it, no one is going to be using them exceptattackers, so there is little system activity. Also, for the Unix based systems, there is no need for a GUI, you canadminister the systems at the Command Line Interface. Thus, no need for running X-Windows. As such, memoryrequirements are minimal. Also, each operating system needs no more the 2 GB of hard drive space. Red Hat Linux 8.0 (64 MB RAM, do not run X-Windows) Solaris8 X86 (64 MB RAM, do not run X-Windows) OpenBSD 3.1 (64 MB RAM, do not run X-Windows) Windows2000 (128 MB RAM) WindowsXP (128 MB RAM)Installing the individual honeypots is simple. First, make sure the vmware virtualization software is running with"ps aef | grep vmnet" and that you have the interface vmnet1 "ifconfig -a". Once you have confirmed that isrunning, create a new VMware window to install your honeypot. The command ishost #vmware -G &

When you start the Window, you have the option of selecting an existing GuestOS to boot up, or start theinstallation of a new GuestOS. For installing a new GuestOS, select "Run the Configuration Wizard". Here youselect which type of GuestOS you are installing, the directory you will store the file system, create a new virtualdisk for the OS, size, enable the CDROM (be sure to disable the floppy, unless its attached) and HostOnlynetworking. Once you have configured the GuestOS, you simply insert the installation CDROM of the Guestoperating system, and Power On the system. From there, the boot and installation of the GuestOS is as younormally would with any operating system. You then proceed to repeat these steps for all five GuestOShoneypots. Once installed, you have the option of installing VMware tools on the honeypots. This helps increasethe resolution of the GUI interface. For the Unix systems, you do not need VMware tools, as you can administerfrom the command line. For Window based honeypots, may want to install this on the honeypots to increaseresolution, however, it will be easier for attackers to fingerprint the system as a VMware virtual system. For moreinformation on VMware configuration and GuestOS installation, refer to the VMware documentation.Before we go any further, you will want to backup your installed honeypots. VMware stores each of yourhoneypots in a seperate file under its own directory. To backup each honeypot, you only merely have to copythese files. This makes recovering or rebuilding your honeypot extremly easy. With traditional Honeynets, after ahoneypot has been compromised and you are done analyzing the attack, you have to rebuild the honeypot beforeputting it back online. This can be a time consuming process. However, with VMware, rebuilding a honeypot is assimple as copying over your backups. You can have your honeypots up and running within seconds. For example,VMware by default stores the images of each honeypot under the directory /root/vmware. You can backup all ofthe honeypots by copying this directory. Whenever you want to rebuild a honeypot, you can merely copy over thehoneypot directory containing its files.host #ls -l /root/vmwaretotal 28drwxr-xr-x 2 root root 4096 Oct 10 01:10 linux-6.2drwxr-xr-x 2 root root 4096 Jan 14 19:00 linux-7.2drwxr-xr-x 2 root root 4096 Jan 14 22:14 linux-7.3drwxr-xr-x 2 root root 4096 Jan 25 15:15 openbsddrwxr-xr-x 2 root root 4096 Jan 25 15:15 solarisdrwxr-xr-x 2 root root 4096 Dec 16 08:47 win2000Servdrwxr-xr-x 2 root root 4096 Jan 25 15:15 winXPProhost #host #cp -a /root/vmware /root/vmware-backupPart III: Data ControlOnce you have setup VMware and the honeypots, the next step is Data Control. The purpose of Data Control is tocontain what the attacker can do inbound and outbound of the Honeynet. Typically, we allow anything inbound tothe Honeynet systems, but limit outbound connections. For the purpose of this paper, we will use IPTables, anOpenSource firewall solution that comes with Linux. IPTables is a highly flexible stateful firewall, including theability for connection limiting, network address translation, logging, and many other features. We will configureIPTables to act as a filter on our HostOS, counting outbound packets. Once a limit has been met for outboundconnections, all further attempts are blocked, preventing the compromised honeypot from harming other systems.Configuring and implementing these capabilities can be extremely complex. However, the Honeynet Project hasdevelop an IPTables script called rc.firewall that does all the work for you. You merely have to modify the scriptvariables as they apply to your Honeynet, then run the script.The first thing you have to decide is if you want your gateway to run in layer three routing mode, or layer twobridging mode. Layer two bridging (also known as GenII, or 2nd generation) is the preferred method. When yourgateway is acting as a bridge, there is no routing or TTL decrement of packets, it acts as an invisible filteringdevice, making it much more difficult for attackers to detect. However, for IPTables to work in bridging mode, yourkernel must be patched to support it. By default, most kernels do not support IPTables in bridging mode. Red Hatkernel 2.4.18-3 is one of the few that does support this by default. If you want to patch your kernel, you can findthe patch at http://bridge.sourceforge.net/download.html. we will assume yourkernel DOES support IPTables in bridging mode. If your kernel does not support bridging mode, the refer to thepaper KYE: UML for more information on configuring the rc.firewall script for layer three routing.Lets cover how to configure the rc.firewall script to implement GenII functionality. There are two critical areas to

configure, the networking issues and control issues. Actually, networking is much simpler in bridging mode then inrouting mode. In bridging mode there is no routing, nor any Network Address Translation issues. We simplyconvert the HostOS to a bridge, and the GuestOS's interact directly with other networks. For connection issues,we have to configure how many outbound connections we allow. The options we will have to configure are asfollows. First, you will need to set the public IP addresses of the Guest operating systems. These are the IPaddresses that hackers will attack, the valid IP addresses of our honeypots. Since we have five honeypots, we willneed to list the five IP addresses. The firewall filters need to know who they are.PUBLIC_IP="10.10.10.201 10.10.10.202 10.10.10.203 10.10.10.204 10.10.10.205"Second, you will need to identify the name of the internal interface of the HostOS. By default, this is eth1.However, we are using the virtual interface vmnet1, and have to modify this variable.LAN_IFACE="vmnet1"Third, since we are building a GenII Honeynet, you may want to consider trying Snort-Inline capabilities to dropknown outbound attacks. It is beyond the scope to describe the details of Snort-Inline, that will bediscussed in the future GenII Honeynet. However, you may want to consider using theHoneynet Snort-Inline Toolkit, which has static, precompiled binaries, configuration files, rulebases, anddocumentation, which you will find . in the Honeynet Tools section. If you do want to test this capability, you willneed to enable the QUEUE option. NOTE: If you enable this option, you MUST be running Snort-Inline, or ALLoutbound packets will be dropped. If you are not sure at this point, do NOT enable this feature.#QUEUE="yes" # Use experimental QUEUE supportQUEUE="no" # Do not use experimental QUEUE supportThese are the minimum variables you will want to consider, there may be others depending on the configurationof your system. There are additional options you can update, such as remote management, limiting whatconnections the firewall can initiate, and giving your honepyots unrestricted DNS access. Also, by default, thescript limits each honeypot to the following outbound connections per hour; 9 TCP connections, 20 UDPconnections, 50 ICMP connections, and 10 IP other. Details of the script are beyond the scope of this paper. Tobetter understand these variables, we recommend you review the script in detail and try out the different optionsin a lab environment. Once you have configured the rc.firewall script, you implement it by executing the script.Remember, you are going to be putting your HostOS into bridging mode. For this, your HostOS must have thebridging utilities. For Red Hat systems, this is known as "bridge-utils-0.9.3-4".There are two gotchas when using bridging. First, you have to boot up all of your GuestOS's before enablingbridging. When the GuestOS's boot, they look for and use the vmnet1 interface. If this interface has already beenset to bridging mode, the GuestOS will not find the interface and cannot talk to the network. So, start all of yourhoneypots BEFORE you run the rc.firewall script. The second gotcha is time, it takes about 10-30 seconds for thebridging to take effect. You have to give the bridge time to learn where all the MAC's are before it can startbridging packets.host #/.rc.firewallTo confirm the script was successful, there are several things to check. First, ensure that bridging has beenenabled. You can confirm this by checking your /var/log/messages file, your kernel should log going into bridgingmode. Second, you should have a new interface called 'br0', which is your bridge. Third, use the 'brctl' commandto see what interfaces are bound to the bridge. Fourth, your external and internal interfaces will no longer have anIP address, as they are now in bridging mode. Last, review your IPTables rules to ensure you are filteringconnections.host #tail /var/log/messageshost #ifconfig -ahost #brctl show

host #iptables -L -nIf successful, your Data Control is in place. There are also other options for implementing Data Control, such asbandwidth throttling.Part IV: Data CaptureOnce we have completed Data Control, the next step is Data Capture. The purpose of Data Capture is to captureall of the attacker's activity, without them knowing. There are a variety of methods to implement this, however wewill focus on two, IPTable logs and Snort. IPTable logs are the logs generated by the firewall whenever there is aninbound or outbound connection. Snort is an OpenSource IDS solution which we will use to capture all networkactivity, and generate alerts for known attacks.For IPTables, the logging has already been configured for us with the rc.firewall script. It is configured to log allnew inbound and outbound connection to /var/log/messages. Any inbound connection is an indication of a probe,scan, or attack. Any outbound connection indicates that a honeypot has been compromised. The value of IPTablelogs is one primarily for alerting. The logs do not have enough information to tell us what the attacker is doing. ForSnort, we configure it to capture every packet and its full payload that enters or leaves the Honeynet. Linked hereis a Snort config file that will capture and log attacker activity.. You will find a simple Snort startup script that startsSnort and uses the recommended Snort config file. Be sure to update the startup script to monitor the vmnet1interface of the HostOS. You will most likely want to run this script daily, running the script from cron.host #./snort-start.shSince this is a GenII Honeynet, you may want to consider using more advance Data Capture techniques, such asSebek. This allows you to capture the attacker's activities from kernel space. There are also a variety of otheroptions for implementing Data Capture which are beyond the scope of this paper. For additional options, checkout Honeynet Tools Section.Part V: Testing Your VMware HoneynetThe fifth, and final step, of building our VMware Honeynet is to test our configuration, specifically Data Controland Data Capture. We want to ensure that our Honeynet requirements are behaving as expected. Testing DataControl is relatively simple. We want to ensure that any attempt by the honeypot to initiate an outboundconnection is both logged and controlled. By logged, all connection attempts should log an entryto /var/log/messages, alerting us that an outbound connection has been initiated, and the honeypot has mostlikely been compromised. Also, once the limit has been met, we want to ensure that no more outboundconnections are allowed. There is one trick to testing our Honeynet, since we are bridging we need a secondcomputer, the attacker. The bridge will not forward any packets if it cannot match the destination IP to a validMAC address. If no packets are forwarded, we cannot test IPTables. For those of you who don't have a secondcomputer (or who are just hard core geeks), you can run a second computer virtually by starting up a UMLsystem. The UML system will bind to the tap0 virtual interface, while all of our VMware honeypots will bind to thevmnet1 virutal interface. This way your HostOS is bridging two different virtual networks. Remember, you willhave to modify your rc.firewall script with tap0 being the external interface. To learn more about running UML,refer to the paper KYE: UML. The UML can be the attacker, probing the VMware honeypots. For the purpose ofthis paper, that is the testing concept we will demonstrate. Our UML attacker's IP address will be 10.10.10.100.Yes, this really does work :)We will test outbound TCP connections, which by default are limited to 9 attempts per hour. To test this we willneed two terminal windows open. First we open a terminal on the HostOS and monitor the IPTable logsin /var/log/messages. When we initiate outbound connections from the GuestOS through our Host gateway, weshould see the attempts logged there. This information is critical for alerting purposes, indicating the honeypot hasbeen hacked, and the attacker (or automated tool) is attempting outbound connections. Starting with the 10thoutbound attempt, the TCP connections should be blocked (the limit was met) and logged. Below is the commandyou want to execute before attempting any outbound connection.host #tail -f /var/log/messages

Next, open a terminal on the honeypot system, our GuestOS. Initiate a variety of outbound TCP connections tothe external IP, in this case 10.10.10.100 (our UML systems). You will most likely have to repeat the attemptsseveral times.Trying 10.10.10.100...telnet: connect to address 10.10.10.100: Connection refusedIf you see the attempts logged, and blocked after the limit, you have successfully implemented Data Control.Next, we want to ensure that Data Capture is happening, specifically that the Snort process is capturing allpackets and their full payload that are entering and leaving the Honeynet. A Snort process should be monitoringthe internal interface of the HostOS, specifically vmnet1. To test this, attempt to ping the external system, in thiscase once again 10.10.10.100.guest #ping -c 3 10.10.10.100The Snort process should have captured the three ICMP Echo Request packets and their full payload. It shouldhave logged the activity to tcpdump binary log format. To confirm, review the log file, an example is below. Whatis critical is that you are not only capture every packet and its header, but you are capturing the fully payload ofevery packet.host #snort -vdr *snort.logThats it! You have just completed a very basic test of your Data Control and Data Capture capabilities. There arefar more advance tests you can attempt, such as using a second, seperate computer to act as a system on theInternet and interact with the honeypot. Specifically, the Suspend feature of VMware. Suspend allows you toliterally suspend a GuestOS (or honeypot) image. It freezes all the running processes and saves the memoryimage to a file. This means you can Suspend your honeypot, turn off your computer, turn it on a week later, bringback up the honeypot, un-suspend it, and you will have it exactly where it was before. This has some incredibleforensic applications. The Project has begun saving suspended images of hacked computers, then sending thoseimages to others for analysis. This allows us to analyze a hacked honeypot while its still in its running state. Theconcern here is when analyzing suspended images, you have to ensure you are doing this on an isolatednetwork, or your hacked honey pot will attempt to connect to any systems it was communicating with before beingsuspended.