tech blog, another one …

HowTo: Debug core dumps with ptxdist 2017.07.0

Debugging for embedded projects is a little harder than for your own computer. In many cases you can not run gdb on the target, and even if you can use gdbserver1 this does not cover all use cases. For post mortem analysis (e.g. after a segmentation fault of your program) you want to examine so called core dumps. Given you successfully found out how to let your target create those2, copied it to your workstation and unpacked it, you still need to know how to analyze it.

With the release of ptxdist 2017.07.0 the handling of debug information changed. Quoting from the release announcement:

The debug symbol handling was reworked. The debug files are now named based
on build-ids and (optional) debug IPKGs can be created. They are not
installed by default but can be installed manually as needed. This is
useful to gdb on the target or with valgrind and perf.

In my BSP those debug info is put to /usr/lib/debug in the root folder from which the target files are copied. This looks like this now:

1

2

3

4

5

6

7

8

9

10

11

12

% tree -a ./platform-foo/root/usr/lib/debug | head

platform-foo/root/usr/lib/debug

└── .build-id

├── 00

│ └── ba20cb0e075c4dc0a792a9062b0864ced517b1.debug

├── 03

│ └── 3b4fc351317376388fadb19fc63b4c8ab6c0d9.debug

├── 04

│ ├── 01fe993aa2bed2155514c676d7001625732396.debug

│ ├── 7bdbc5fd44a4444de24762c76a3313d1fda2c0.debug

│ └── d0ecb6611e590a036cbdd5909cc5bfc9158af8.debug

You can imagine it is not possible anymore to load this manually, the debugger will have to find out by itself. Getting this to work caused me some headaches, but this it how I got it work: Create a file ‘gdb-config’ with the following content:3

1

2

set debug-file-directory /home/adahl/Work/bsp/foo/platform-foo/root/usr/lib/debug

set sysroot /home/adahl/Work/bsp/foo/platform-foo/root

Note: the order of the commands is important, it does not work the other way round!

So you run the gdb from your toolchain, load the previously crafted file with gdb commands with -x, give the path to your executable with -e and finally the core dump file with -c and that’s it. You may now have a look at a backtrace and find out what caused the segfault …

Update: I was pointed to an easier possibility to invoke the right gdb with the necessary options in #ptxdist IRC channel (on freenode). The previous call would be like this:

No need to pick the correct gdb and find and configure the right directories, just add the path to your tool and your core file, ptxdist handles everything else. I bet this would also work with older ptxdist versions, where the debug symbols where placed somewhere else, but didn’t try it. This however was also just added with ptxdist 2017.07.0:

There is also a new ptxdist command ‘gdb’ for remote debugging that sets up
the sysroot correctly and a wrapper script that can be used by graphical
development environments.