Examine log files
Look for setuid and setgid Files
Check system binaries
Check for packet sniffers
Examine files run by 'cron' and 'at'.
Check for unauthorized services
Examine /etc/passwd file
Check system and network configuration
Look everywhere for unusual or hidden files
Examine all machines on the local network

This document outlines suggested steps for determining if your system has been compromised. System administrators can use this information to look for several types of break-ins. We encourage you to review all sections of this document and modify your systems to close potential weaknesses.

In addition to the information in this document, we provide three companion documents that may help you:

We also encourage you to check with your vendor(s) regularly for any updates or new patches that relate to your systems.

Look For Signs That Your System May Have Been Compromised

Note that all action taken during the course of an investigation should be in accordance with your organization's policies and procedures.

Examine log files for connections from unusual locations or other unusual activity. For example, look at your 'last' log, process accounting, all logs created by syslog, and other security logs. If your firewall or router writes logs to a different location than the compromised system, remember to check these logs also. Note that this is not foolproof unless you log to append-only media; many intruders edit log files in an attempt to hide their activity.
Look for setuid and setgid files (especially setuid root files) everywhere on your system. Intruders often leave setuid copies of /bin/sh or /bin/time around to allow them root access at a late time. The UNIX find(1) program can be used to hunt for setuid and/or setgid files. For example, you can use the following commands to find setuid root files and setgid kmem files on the entire file system:

Note that the above examples search the entire directory tree, including NFS/AFS mounted file systems. Some find(1) commands support an "-xdev" option to avoid searching those hierarchies. For example:

find / -user root -perm -4000 -print -xdev

Another way to search for setuid files is to use the ncheck command on each disk partition. For example, use the following command to search for setuid files and special devices on the disk partition //[b][/b]dev[b][/b]/rsd0g:

ncheck -s //[b][/b]dev[b][/b]/rsd0g

Check your system binaries to make sure that they haven't been altered. We've seen intruders change programs on UNIX systems such as login, su, telnet, netstat, ifconfig, ls, find, du, df, libc, sync, any binaries referenced in /etc/inetd.conf, and other critical network and system programs and shared object libraries. Compare the versions on your systems with known good copies, such as those from your initial installation media. Be careful of trusting backups; your backups could also contain Trojan horses.

Trojan horse programs may produce the same standard checksum and timestamp as the legitimate version. Because of this, the standard UNIX sum(1) command and the timestamps associated with the programs are not sufficient to determine whether the programs have been replaced. The use of cmp(1), MD5, Tripwire, and other cryptographic checksum tools is sufficient to detect these Trojan horse programs, provided the checksum tools themselves are kept secure and are not available for modification by the intruder. Additionally, you may want to consider using a tool (PGP, for example) to "sign" the output generated by MD5 or Tripwire, for future reference.

Check your systems for unauthorized use of a network monitoring program,

commonly called a sniffer or packet sniffer. Intruders may use a sniffer to capture user account and password information. For related information, see CERT advisory CA-94:01 available in http://www.cert.org/advisories/CA-94.01.ongoing.network.monitoring.attacks.html
Examine all the files that are run by 'cron' and 'at.' We've seen intruders leave back doors in files run from 'cron' or submitted to 'at.' These techniques can let an intruder back on the system (even after you believe you had addressed the original compromise). Also, verify that all files/programs referenced (directly or indirectly) by the 'cron' and 'at' jobs, and the job files themselves, are not world- writable.
Check for unauthorized services. Inspect /etc/inetd.conf for unauthorized additions or changes. In particular, search for entries that execute a shell program (for example, /bin/sh or /bin/csh) and check all programs that are specified in /etc/inetd.conf to verify that they are correct and haven't been replaced by Trojan horse programs.

Also check for legitimate services that you have commented out in your /etc/inetd.conf. Intruders may turn on a service that you previously thought you had turned off, or replace the inetd program with a Trojan horse program.

Examine the /etc/passwd file on the system and check for modifications to that file. In particular, look for the unauthorized creation of new accounts, accounts with no passwords, or UID changes (especially UID 0) to existing accounts.
Check your system and network configuration files for unauthorized entries. In particular, look for '+' (plus sign) entries and inappropriate non-local host names in /etc/hosts.equiv, /etc/hosts.lpd, and in all .rhosts files (especially root, uucp, ftp, and other system accounts) on the system. These files should not be world-writable. Furthermore, confirm that these files existed prior to any intrusion and were not created by the intruder.
Look everywhere on the system for unusual or hidden files (files that start with a period and are normally not shown by 'ls'), as these can be used to hide tools and information (password cracking programs, password files from other systems, etc.). A common technique on UNIX systems is to put a hidden directory in a user's account with an unusual name, something like '...' or '.. ' (dot dot space) or '..^G' (dot dot control-G). Again, the find(1) program can be used to look for hidden files, for example:

find / -name ".. " -print -xdev

find / -name ".*" -print -xdev | cat -v

Also, files with names such as '.xx' and '.mail' have been used (that is, files that might appear to be normal).
Examine all machines on the local network when searching for signs of intrusion. Most of the time, if one host has been compromised, others on the network have been, too. This is especially true for networks where NIS is running or where hosts trust each other through the use of .rhosts files and/or /etc/hosts.equiv files. Also, check hosts for which your users share .rhosts access.

Review Other CERT Documents

For further information about the types of attack that have recently been reported to the CERT Coordination Center and for a list of new or updated files that are available for anonymous FTP, see our past CERT Summaries, available in the directory

Also review other appropriate files in our tech_tips directory.
To report a computer security incident to the CERT Coordination Center, please complete and return a copy of our Incident Reporting Form, available from

The information on the form helps us provide the best assistance, as it enables us to understand the scope of the incident, to determine if your incident may be related to any other incidents that have been reported to us, and to identify trends in intruder activities.

Examine files run by 'cron' and 'at'.i can resist to make a reply here. sorry if it sounds like i want to bash several things in your post. its not meant that way, just maybe a proof that this wont help to get full security.

first of all, you got hacked, and you wont notice:

Quote:

Examine log files

i use a logcleaner, you wouldnt see my connections then, not even what commands i typed in. only if it is a bad logcleaner, that leaves zeros behind, then you might have a chance to see me.

Quote:

Look for setuid and setgid Files

ok, thats true, this might help. but going through all those files as a normal user? hrm, not sure, and what do you do with a file named "sysdll"? do you remove it? maybe not? maybe its important?

Quote:

Check system binaries

errr, for what and how? compromised files maybe, yes, then use chkrootkit (get it at freshmeat.net) to remove those rootkits that infected your system.

Quote:

Check for packet sniffers

alright, how? ettercap and most others run stealthy. you wont even notice them. if there is a packet sniffer, you should change all your pwd, remove the whole OS and start over again. they sniffed your whole life. AIM, ICQ; mail pwd, even SSH pwds (yes, thats working, not talking about ssh2 )

Quote:

Examine files run by 'cron' and 'at'.

hrm, to check if anything runs that you dont want to. maybe check init too, or the kernel start-up at booting. there are so many ways, too many to check them all i guess. backdoors that react to a ICMP packet, shellcode, etc etc.

Quote:

Check for unauthorized services

i suppose you mean a rootkit. there are ways that you wont see them running, not even as root. services like sendmail and stuff should be restricted anyway. get chkrootkit from freshmeat, thats all i can say.

Quote:

Examine /etc/passwd file

why? i have your root pwd from my sniffer, why should i need to make another root account. you wont find much there. only if pwds are altered, which is unnecessary.

Quote:

Check system and network configuration

good, might help to check if the network runs in promiscous mode. really tough for the normal user i guess :\

Quote:

Look everywhere for unusual or hidden files

wow, everywhere? man, im fux0red, now i wont have time to do anything else. whats an unusual file anyway? lets see, crackers mostly put them into /tmp, or even delete their packages. wont help to see if you got hacked, sorry.

Quote:

Examine all machines on the local network

if you are a sysadmin and follow this advices, that means you got hacked, get a job that you know. maybe M$ tech support. stay away from administration, this is basic stuff and the hackers/crackers out there eat you alive.

do i get a cookie now? im sorry to look like an idiot with that reply, but it teaches you that you never know if you got hacked or not. tripwire isnt free, use their free replacement maybe, run nessus and nmap. thats all you can do. oh, maybe tiger scripts too. everything at freshmeat.net.
btw, cert is only for the USA i guess? ah damn, im from russia im fux0red :\
did i already mention that cracker mostly replace ps and find and some other system binaries? no? then i did now. have fun looking for that service with the altered ps

Or in fact you should be using Osiris, Tripwire or Samhain anyway, so you should know your system binaries have been changed.

Samhain is my software of choice.

CoyoteX wrote:

Quote:

Check for packet sniffers

alright, how? ettercap and most others run stealthy. you wont even notice them. if there is a packet sniffer, you should change all your pwd, remove the whole OS and start over again. they sniffed your whole life. AIM, ICQ; mail pwd, even SSH pwds (yes, thats working, not talking about ssh2 )

wow, everywhere? man, im fux0red, now i wont have time to do anything else. whats an unusual file anyway? lets see, crackers mostly put them into /tmp, or even delete their packages. wont help to see if you got hacked, sorry.

do i get a cookie now? im sorry to look like an idiot with that reply, but it teaches you that you never know if you got hacked or not. tripwire isnt free, use their free replacement maybe, run nessus and nmap. thats all you can do. oh, maybe tiger scripts too. everything at freshmeat.net.
btw, cert is only for the USA i guess? ah damn, im from russia im fux0red :\

I'll give you a cookie if you think you deserve one

Tripwire is not free not, but Samhain is.

There are plenty of things you can do, most of them are on this forum.

You can subscribe to the CERT list anywhere in the world, you don't have to be from the US.

i use a logcleaner, you wouldnt see my connections then, not even what commands i typed in. only if it is a bad logcleaner, that leaves zeros behind, then you might have a chance to see me.

If ext2fs is used, it is possible to undelete the sauber'd logfiles (or any other source-code the intruder compiled and then deleted for instance). This will give some interesting information to work on.

Also, when using a loghost (which does *nothing* more then accepting loggings, no other services of course), it is impossible for the intruder to clean the files on that machine (unless that machine is also compromized), but noticed that was already mentoined.

Not only `ps' can be trojaned, but also the find binary cannot be trusted on a root-ed machine. Before you use find, you must verify it's integrity (or upload a staticly linked binary on the host you want to investigate / analyze).

The fact that it's kept up to date, it's still under active development, it's not commercial, it has a neat stealth mode, centralised logging and management/config/updates (with DB support). I think that's enough reasons