GDB

The GNU Debugger GDB is the most common debug tool for Linux. It features most used features one can think of, including server-client debug architecture (so you run the heavy debug part on your host/pc machine), but lack some bits as checkpoint-restart during execution.

Documentation

Basic Usage

Documentation is so large that sometimes its hard to get started, so most simple tasks can be done with the following commands, but please read GDB docs as soon as possible!

Debug version

Debug packages

First note that in order to use GDB in an efficient way,you need the debug packages,that is to say the -dbg version of your package.
Under the opkg package management if you installed a binary like that:

opkg install binary

you need to install the debug part of it:

opkg install binary-dbg

Without debug packages you will be limited to assembly debugging(no source level debugging)

Note that if you have restricted space on the target,you could use NFS root if you have an ethernet connection.

No packages

As I don't have packages which segfault at hand,I'll create a binary which segfault
First start exporting the path of the toolchain if it's not done yet:

-static is only for our test,it's for avoiding runtime issues in case the libc is different or incompatible between codesourcey and our target(in case of ucilbc for instance)

Cross or not crosss

A common error while using gdb,is using the wrong gdb. In order to debug a program that runs on a target from your host you need a cross gdb,that is to say a GDB that runs on your computer but can debug the target architecture.
Such version is normally included in your toolchain/SDK or buildable if you use a build system

If you want to debug use GDB on target you can too,it's easier but has a huge drawback(amongs other) : ram usage (you can easily use too much memory and have an OOM(out of memory) )

Cross GDB

Code sourcey Linux

Let's say that you downloaded arm-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2

You will need to unpack it,and to run the following command on your host computer (instead of just running gdb):

cd arm-2010q1/bin
./arm-none-linux-gnueabi-gdb

Openembedded

If your build system is also your debug workstation do:

bitbake gdb-cross gdbserver

And it will build a cross gdb for your host and gdbserver for your target
The resulting binaries will be found in your TMPDIR here:

/home/embedded/tmpdir/cross/armv6/bin/

replace "/home/embedded/tmpdir" by your tmpdir and armv6 by your target architecture
The binary name changes according to your distro settings,for me it was:

arm-angstrom-linux-gnueabi-gdb

So in order to run it do:
./arm-angstrom-linux-gnueabi-gdb

Howto

If you have networking,it's advised to choose it because it will be faster than serial,but if you only have serial choose serial

Core dump

Sometimes you can get a coredump.
A man page describing the core dump,and in which condition it can occur can be found here
In order to make possible core dumps(that unlimit the size limit on coredumps)

target/device$ ulimit -c unlimited

Then run your binary:

target/device$ ./binary

Then reproduce the conditions under which the coredump happen.The core file will appear in your current working directory and will be named core

Then copy the coredump to your workstation,and run gdb on your workstation(core is assumed to be in the current directory)

Networking

Fist start gdbserver on the target:

target/device$ gdbserver :2345 ./binary arg1 arg2

Note that will expose a gdb session to all networking interfaces,for instance if you have wifi on,someone could connect to it.
So if you don't want that,make it listen only on your the ip of the target interface you want to use for debugging:

target/device$ gdbserver 192.168.0.202:2345 ./binary arg1 arg2

Then start GDB on your workstation,but do not connect to the target yet(else you'll have some undefined symbols):

host/pc$ ./arm-angstrom-linux-gnueabi-gdb
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-linux --target=arm-angstrom-linux-gnueabi".

Then you need access to the target filesystem,knowing that we have networking we could use the following options:

sshfs:

In order to use sshfs you need sftp-server installed on the target,and sshfs installed on the host computer.
To mount the filesystem do:

sudo sshfs root@192.168.0.202:/ /mnt/target/

NFS(maybe you already had nfsroot on the target)

any other system that export the filesystem,and that is mountable

Then you need to set a variable called sysroot in order to make the cross gdb find the libraries to load

Serial

First set the serial port speed on the target(before you issue the gdbserver command):

stty speed 115200 < /dev/ttyS1

Then start gdbserver on the target:

target/device$ gdbserver /dev/ttyS1 ./binary arg1 arg2

That will freeze the program until you connect with the cross gdb version and that you type "c" for continuing the execution of the program.
Note that GDB will slow the program a lot. if you have a program that uses a lot of resources,it's advised to attach to it just before the crash instead like that:

Shared Object Paths

Often your cross compile root is not /, so you might have to add new paths to the search list.

Unset absolute prefix:

(gdb) set solib-absolute-prefix null

Add paths to search paths:

(gdb) set solib-search-path /path1:/path2

Alternatively you can choose to set the prefix to the root of your target file system. Specially if you are doing
embedded development and already exporting your root file system from you host machine to your target machine it can
be very rewarding so simply use that as root: