chkrootkit Rootkit Scanner

chkrootkit is a collection of small utilities used to detect the presence of known rootkits on a system. Although it is possible to find stand-alone detection scripts for almost all rootkits, chkrootkit differentiates itself by its ability to detect a large number of different rootkits using a single application. In addition to detecting known rootkit signatures, it also runs some generic tests that might aid in discovering a rootkit that is not actually supported by the application.

The chkrootkit package contains seven small applications. The main progam, chkrootkit, is used to detect known rootkit signatures. In addition to searching for a rootkit's default files, it also checks core system binaries for malicious content. The remaining applications in the package are ifpromisc, which helps to find out whether a network interface is in promiscuous mode (remember you can't trust netstat), chklastlog, chkwtmp, and check_wtmpx, all of which detect deletions in various log files, chkproc, which detects Loadable Kernel Module (LKM) trojans and hidden processes, and finally strings, which is a simple implementation of the Unix strings utility.

Before going into how to use chkrootkit, let's see how to obtain and install it. The official chkrootkit site is http://www.chkrootkit.org. Download the latest .tar.gz file from this Web site. After getting the file, put it in a temporary directory and decompress it (tar xfvz chkrootkit.tar.gz) and cd into the newly created directory (cd chkrootkit-0.35). Finally, type make sense to compile it. Almost always the application will compile without a problem, but if you get error messages try building the software on another, similar machine and moving the binaries to the server you want to scan. Although it is possible to copy the newly created binaries to a central location in order to install them, this is not recommended because the software is updated frequently with new capabilities and it's best to get a new copy each time.

After building the binaries, my personal preference is to run ./ifpromisc right away to see if any of the interfaces are in promiscuous mode, because in such a case I might want to pull the server off the network immediately. It is also a good idea to run ./chkproc right after ifpromisc to see if there are any hidden processes or LKM trojans on the system. After running these two stand-alone utilities, you can run the main program ./chkrootkit without parameters. This will perform the rootkit tests as well as the tests provided by the rest of the utilities in the package, including ifpromisc and chkproc, which we had run earlier.

Here's some sample output from chkrootkit:

[root chkrootkit-0.34]# ./ifpromisc
eth0 is not promisc
eth0:0 is not promisc
eth0:1 is not promisc
eth0:2 is not promisc
eth0:3 is not promisc
eth0:4 is not promisc
eth0:5 is not promisc
eth0:6 is not promisc
eth0:7 is not promisc
eth0:8 is not promisc
[root chkrootkit-0.34]#

[root chkrootkit-0.34]# ./chkrootkit
ROOTDIR is `/'
Checking `amd'... not found
Checking `basename'... not infected
Checking `biff'... not found
Checking `chfn'... not infected
Checking `chsh'... not infected
Checking `cron'... not infected
Checking `date'... not infected
Checking `du'... not infected
Checking `dirname'... not infected
Checking `echo'... not infected
Checking `egrep'... not infected
Checking `env'... not infected
Checking `find'... not infected
Checking `fingerd'... not found
Checking `gpm'... not infected
Checking `grep'... not infected
Checking `hdparm'... not infected
Checking `su'... not infected
Checking `ifconfig'... not infected
Checking `inetd'... not infected
Checking `inetdconf'... INFECTED
Checking `identd'... not found
Checking `killall'... not infected
Checking `login'... not infected
Checking `ls'... not infected
Checking `mail'... not infected
Checking `mingetty'... not infected
Checking `netstat'... not infected
Checking `named'... not infected
Checking `passwd'... not infected
Checking `pidof'... not infected
Checking `pop2'... not found
Checking `pop3'... not found
Checking `ps'... not infected
Checking `pstree'... not infected
Checking `rpcinfo'... not infected
Checking `rlogind'... not infected
Checking `rshd'... not infected
Checking `slogin'... not infected
Checking `sendmail'... not infected
Checking `sshd'... not infected
Checking `syslogd'... not infected
Checking `tar'... not infected
Checking `tcpd'... not infected
Checking `top'... not infected
Checking `telnetd'... not infected
Checking `timed'... not found
Checking `traceroute'... not infected
Checking `write'... not infected
Checking `aliens'... no suspect files
Searching for sniffer's logs, it may take a while... nothing found
Searching for t0rn's default files and dirs... nothing found
Searching for t0rn's v8 defaults... nothing found
Searching for Lion Worm default files and dirs... nothing found
Searching for RSHA's default files and dir... nothing found
Searching for RH-Sharpe's default files... nothing found
Searching for Ambient's rootkit (ark) default files and dirs... nothing found
Searching for suspicious files and dirs, it may take a while...
/usr/lib/perl5/site_perl/5.005/i386-linux/auto/mod_perl/.packlist /usr/lib/perl5/site_perl/5.005/i386-
linux/auto/MD5/.packlist /usr/lib/perl5/site_perl/5.005/i386-linux/auto/Quota/.packlist
/usr/lib/perl5/site_perl/5.005/i386-linux/auto/XML/Parser/.packlist /usr/lib/perl5/site_perl/5.005/i386-
linux/auto/Devel/Symdump/.packlist /usr/lib/perl5/site_perl/5.005/i386-linux/auto/Image/Magick/.packlist
/usr/lib/perl5/5.00503/i386-linux/.packlist /lib/... /lib/.../BitchX/.config.h
/lib/...
Searching for LPD Worm files and dirs... nothing found
Searching for Ramen Worm files and dirs... nothing found
Searching for Maniac files and dirs... nothing found
Searching for RK17 files and dirs... nothing found
Searching for Ducoci rootkit... nothing found
Searching for Adore Worm... nothing found
Searching for ShitC Worm... nothing found
Searching for Omega Worm... nothing found
Searching for anomalies in shell history files... nothing found
Checking `asp'... not infected
Checking `bindshell'... not infected
Checking `lkm'... ./chkrootkit: [: integer expression expected before -gt
not tested
Checking `rexedcs'... not found
Checking `sniffer'...
eth0 is not promisc
eth0:0 is not promisc
eth0:1 is not promisc
eth0:2 is not promisc
eth0:3 is not promisc
eth0:4 is not promisc
eth0:5 is not promisc
eth0:6 is not promisc
eth0:7 is not promisc
eth0:8 is not promisc
Checking `wted'... nothing deleted
Checking `z2'... user ap deleted or never loged from lastlog!

As you can see, on this system the interfaces are not promiscuous, but there is an infected file. Chkrootkit also detected a suspicious file in a '...' directory (/lib/.../BitchX/.config.h). It is common for intruders to create directories whose names start with a decimal dot character, which makes them invisible or hard to find. You should also be looking for ' .' (space dot) files, which are difficult to differentiate from a '.' (current dir) on a terminal screen. The error on the 'Checking lkm' line is nothing to worry about because that's the error message you get when your kernel does not support loadable kernel modules, and there's nothing wrong with that--it is an almost desirable configuration for a server. Finally, the last line shows that a user 'ap' has either not logged in or was deleted later on from the log. I strongly recommend reading the FAQ on the chkrootkit Web site for more information.

A Rootkit Was Detected, Now What?

The only way you can be 100 percent sure your system is clean after a rootkit infection is if you format your drive and reinstall the operating system again from the original media. However,
real-life conditions can force you to keep an infected system running at least for a little while. In such a case you should at least clean up your server and get rid of the rootkits and trojans.

Although the chkrootkit output is clear and to the point, it doesn't give you any pointers as to how you get rid of any given rootkit or trojan once it's detected. You need to familiarize yourself with the way rootkits work. You can do so by frequenting security-related Web sites such as cert.org or sites that explain how a specific rootkit works. This
article at Teep's site has a comprehensive report on the anatomy of a particular rootkit that also provides a lot of valuable information applicable to other rootkits. Another way to familiarize yourself with rootkits is to find and install them on a test environment and compare the effects by using before-and-after snapshots of the system. If you know what a rootkit does, which directories and files it creates, and which binaries it infects, you can go to the infected server and remove the trojan files and replace the infected binaries with clean ones.

On RPM-based systems you can identify the infected packages as described earlier in this article. Once you have a list of the infected RPM packages, you can reinstall them using your original source.

The following syntax will force an RPM package to be reinstalled, overwriting the old files in the process:

rpm -U --force rpm_package_name.rpm

After reinstalling all infected RPM packages, run the RPM and chkrootkit checks again to see if there are still any unresolved issues. If your system doesn't use RPM, you can follow a similar procedure that applies to the particular operating system you're running. If you're not using any packaging system at all, you'll need to recompile those applications from source
and overwrite the old ones.

After reinstalling packages that are supposed to be on your system, you need to clean up alien files that are introduced to your system by the intruder. The chkrootkit output will help you identify the location of these files. Also, after cleaning up core system utilities such as ifconfig, netstat, find, ls, top, and ps you can gather more accurate information that might reveal even more alien files and processes. You should go through and remove all these files and directories. If you take a few minutes to read the config files that might be included in the rootkits, you might find clues as to what other trojan files and processes exist on the server.

After removing all the alien files you can identify, run 'top' and 'ps' again to identify and kill undesired processes that are still running. You can also use 'netstat -nap' on most systems to see the process name and ID that started a daemon. You might also want to check your initialization scripts and make sure no alien process is started using those scripts. Intruders usually place their startup lines at the bottom of these scripts but they can also hide them in the middle of the file to make it harder to identify. Some common locations that intruders use to place their startup calls are the inetd.conf file, rc.local file, and any other file in the initialization directory for the run-level you run the server at. You should also pay special attention to helper function files such as /etc/rc.d/init.d/functions, which other initialization scripts include, thereby allowing any valid command in them to be executed indirectly when the system is rebooted.

When you're confident there are no sniffers or trojans left running, change all the passwords on the server. If this is not possible at least change the passwords for users who have root or equivalent access to the server. Also go through your password file and remove any users that you think shouldn't be there, especially those with root equivalency. You may optionally reboot the server and perform another security scan to see if you were able to clean up all startup scripts and infected binaries.

Conclusion

Related article:

Understanding Rootkits -- Hackers have many tools that allow them to remain undetected during an attack. Understanding these tools is key to recognizing and cleaning up after an attack.

In theory, it is possible to detect and clean up all effects of a rootkit or a trojan on a compromised machine. However, it is extremely important to secure a system so that rootkits and trojans can't be introduced to the system in the first place. Most of the intrusion activity on the Internet targets servers that respond positively to a network
scan probing for a specific operating system or software weakness. If you keep up with software patches and updates, you are less likely to be targeted in the first place because a potential intruder will usually pick a server that he or she can get into with the least effort, and there's never a shortage of these weak servers on the Internet.

Furthermore, if your server still gets broken into, you must unplug that server from the network immediately, reinstall the OS, and apply all security patches before introducing it to the network again.

The information I provide is not meant to replace common security practices, but might prove useful when used in conjunction with these practices.

May your servers always be rootkit free.

Oktay Altunergil
works for a national web hosting company as a developer concentrating on web applications on the Unix platform.