Older Information

Here's an article from 2002 on KDB vs. KGDB: http://kerneltrap.org/node/112
It has a good discussion excerpt between Andrew Morton and Keith Owens about the relative merits of KDB versus KGDB.

General Information

Kernel Versions supported

kgdb was added to the mainline Linux kernel in version 2.6.26.

kdb support was added to the mainline Linux kernel in version 2.6.35.

Before those versions, kgdb and kdb were available as patches which could be
applied to the Linux kernel source.

Kernel configuration

The following descriptions are for a 2.6.35 kernel, using KDB over a serial line between host and target:

All these options on are the "Kernel Hacking" menu.

In order to support KDB, "KGDB" support must be turned on first (even if you aren't using kgdb/gdb)

CONFIG_DEBUG_KERNEL=Y - includes debug information in the kernel compilation - required for basic kernel debugging support

CONFIG_KGDB=Y - turn on basic kernel debug agent support

CONFIG_KGDB_SERIAL_CONSOLE=Y - to share a serial console with kgdb.

Sysrq-g must be used to break in initially.

Selecting this will automatically set:

CONFIG_CONSOLE_POLL=N

CONFIG_MAGIC_SYSRQ=Y - turn on MAGIC-SYSRQ key support

CONFIG_KGDB_KDB=Y - actually turn on the KDB debugger feature

Optional other configuration settings:

CONFIG_FRAME_POINTER=Y - this allows for better backtrace support in KDB

CONFIG_DEBUG_RODATA=N - disable this in order to support breakpoints on data accesses

CONFIG_KALLSYMS=Y - this adds symbolic information to the kernel, useful to see symbols instead of addresses

CONFIG_KDB_KEYBOARD - use KDB with an attached keyboard (not for use with serial console)

CONFIG_KGDB_TESTS - used to turn on kgdb internal self-tests - see the config help for this for more information

Enabling kdb

At runtime

Once the kernel is compiled with kdb support and is running on your target board, you need to
enable it. This can be done on a running system, binding the kdb/kgdb feature to a serial port,
by writing a value into the sys filesystem.

If your machine starts a serial console on ttyS0, you can bind kdb/kgdb to this serial console
by writing the string "ttyS0" to /sys/module/kgdboc/parameters/kgdboc. The kernel will respond
with a message indicating that that driver is registered.

Finally, you can set a breakpoint (either hardware or software), such that the debugger is
invoked when the breakpoint condition is met. For a code breakpoint, this means when
the instruction is executed at the breakpoint location, and for a data breakpoint, when the
particular access is made at the breakpoint address.

Invoking with Magic SysRq 'g'

To invoke the debugger using the Magic SysRq command, you use the 'g' command, which
can be issued any of the ways supported by the Magic SysRq feature. This can be done
by 1) typing the key sequence on a connected keyboard, 2) echoing a value to /proc/sysrq-trigger, or
3) sending a break key sequence through the serial console

Invocation by panic

When a kernel panic occurs, then something has gone seriously wrong and the kernel automatically enters
kdb. From here you can look at memory, do a traceback, examine registers, and do other operations to
find out more about the state of the system and debug the problem.

Invocation by breakpoint

To enter kdb using a breakpoint, first invoke kdb using the Magic SysRq key (see above), then set a breakpoint.
Then type 'go' to continue execution. When the breakpoint is hit, the debugger shell will appear.

In the example that follows, items in italics are commands typed by a user.
Items following a '$' are commands entered at a shell command (normal
Linux user-space runtime), and items following 'kgdb>' are commands entered
at the kdb interactive shell.

This example shows an invocation of kdb, followed by setting a breakpoint, then
resuming execution with 'go'. Then, at the Linux user-space shell, the 'sync' command
is run to cause the breakpoint to occur. When kdb is entered due to the breakpoint,
then 'bt' is run to get a backtrace from the stack of the current process.

Using gdb to see the kernel source listing

You can use the addresses printed out in kdb, with a host-side gdb session, to see
the source code or assembly instructions around a particular address.

The target address can come from a backtrace or register dump (e.g. instruction pointer).

To load the source for a kernel, start gdb (or the appropriate arch-specific gdb) with the
vmlinux that matches the image running on target.
The kernel should have been compiled with debug symbols (CONFIG_DEBUG_KERNEL=y).
gdb will start, and load the symbol information for the kernel.

Use the following commands to see various bits of information:

source file and line number for an instruction address

info line *0x<target_addr>

source lines around an instruction address

list *0x<target_addr>

assembly instructions at an address

disas 0x<target_addr>, or

x/20i 0x<target_addr>

KDB environment variables

Here are some environment variables supported by kdb (in 2.6.35):

LINES - set the number of lines for paging output from KDB (default 24)

PS - specify a list of task states (one letter per state) for filtering the 'ps' command (default 'DRSTCZEU')

letters can be used from the list: DRSTCZEUIMA, each letter corresponding to a task state (see the PS man page for task states)

D = uninterruptible sleep

R = running

S = interruptible sleep

T = stopped

C = traced

Z = exited and zombied

E = exited and dead

U = unrunnable

I = idle

M = daemon

A = all

Unused args (in 2.6.35):

BTARGS - number of arguments to show in bt command (default 9)

is used for argcount for internal routine kdb_bt1, but argcount is never used in that routine

BTSYMARG - ??? [set in kdb_cmds 'dumpall' and 'dumpcpu', but not used in code)

appears to be an integer

Hints and tips

Command line completion

kdb supports command line completion using the tab key. While typing at the interactive shell, you can press the tab key to get a list of symbols to use as part of the command. If you type the first few letters of a symbol name, then press the tab key, the shell will complete the name for you. If more than one symbol matches
what you have typed, then a list is shown of matching symbols. The number of symbols shown is controlled by
the DTABCOUNT variable.

Defining a set of commands

You can define a set of commands to be run as a group (in sequence), using the 'defcmd' command.
This group essentially becomes a new command which you can run, using the indicated name.

The second two strings after the new command name are the Usage string and the Description string.
These appear in the online help when the 'help' or '?' commands are run.

Executing commands on kdb initialization

In the kernel source, you can place commands in the file 'kdb_cmds'. These commands will be executed
by kdb on it's initialization. By default, a few command sets are declared: 'dumpcommon', 'dumpall', and 'dumpcpu'.

Piping command results through 'grep'

kdb includes a grep-like capability, which provides the ability to filter command results using a pattern. To
use this, type the regular command, and at the end of the line add '| grep <pattern>', where pattern is the pattern to match.

To see online help for this feature, type 'grephelp' at the kdb shell prompt.

The pattern may include a very limited set of metacharacters:

pattern or ^pattern or pattern$ or ^pattern$

And if there are spaces in the pattern, you may quote it:

"pat tern" or "^pat tern" or "pat tern$" or "^pat tern$"

Command history

You can access the command history by typing up-arrow and down-arrow, and select a previous command to run.

If you hit <enter> without typing a command, sometimes (when it makes sense) the last command run will be run again. For 'md' commands, the next command will continue at the address where the last command left off.

Feature extension notes (by Tim)

Can break on __do_user_fault, to break into kdb on user process SIGSEGV (on ARM, anyway)

See CONFIG_DEBUG_USER (on ARM) for extra information shown and code paths used, on user-space on faults