Troubles with Debuggers

A simple measure against crackers is detecting whether a program is being debugged. There is a race between defense adding new recognition features and attackers finding ways to fool defenders. In fact, White Hawk Software does not have the illusion we can always recognize when an attacker uses a debugger. So we use a large number of different methods, annoying any attacker so that he just never knows when he is finished after finding the next hurdle.

For an extra kick the WHS protected program makes its defense in such a subtle way, that the attacker might not recognize the defense. Maybe a wrong algorithm is chosen, or the precision will become dismal… Sadly, we cannot create subtle misbehavior automatically: in practice it turns out to either not be subtle enough, or not to disturb the attacker at all. Real subtlety is the domain for manual protections and therefore restricted for use in very high-end defenses. Here, White Hawk Software can offer to give the user enough control and enable manual creation and integration of such a protection. One thing however we can do automatically: Add a delay between detection and acting, so the attacker might not see the spot where his presence has been detected. Another trick against scripted attackers is to not always trigger but to use some randomness.

We want to present some tricks to recognize a debugger. The absolute last thing we want to do is educating attackers, and that is indeed not happening. Rest assured, attackers already know about what is written here, it’s ok to also let the other people know. By the way: When researching known anti-debugger tricks we have found a large number to be broken. Traditionally it is the good guy which uses a debugger, and malware trying to prevent debugging. For this reason (at least we occasionally think so) text-books describe detection methods which don’t quite work completely. We will stay within that tradition, we will stay far too general and high-level to risk exposure of new information to criminals.

There are very different classes of debuggers. For example: Breakpoints can be created by modifying the code. Breakpoints can be implemented in hardware. A debugger can stop all threads, or could stop just one single thread. An emulated processor could be used. The program could be run on special hardware for debugging. No single measure will recognizes all.

Finally, here are some ways to detect your program may be debugged

The operating system has an API to detect debuggers. Look for that call.

There are some bits set by debuggers which can be checked directly without using the API. (E.g. Windows Process Environment Block)

Recognize presence of trap instructions. (And don’t trip over data with the same encoding.)

Special-purpose breakpoint registers can be “used” by us, so use by a debugger becomes conflicted.

Debuggers themselves can have bugs which are known and exploited by malware. (But eventually those will become fixed.)