Red Hat Enterprise Linux also provides -debuginfo packages for all architecture-dependent RPMs included in the operating system. A packagename-debuginfo-version-release.architecture.rpm package contains detailed information about the relationship of the package source files and the final installed binary. The debuginfo packages contain both .debug files, which in turn contain DWARF debuginfo and the source files used for compiling the binary packages.

Note

Most of the debugger functionality is missed if attempting to debug a package without having its debuginfo equivalent installed. For example, the names of exported shared library functions will still be available, but the matching source file lines will not be without the debuginfo package installed.

Use gcc compilation option -g for your own programs. The debugging experience is better if no optimizations (gcc option -O, such as -O2) is applied with -g.

For Red Hat Enterprise Linux 6, the debuginfo packages are now available on a new channel on the Red Hat Network. To install the -debuginfo package of a package (that is, typically packagename-debuginfo), first the machine has to be subscribed to the corresponding Debuginfo channel. For example, for Red Hat Enterprise Server 6, the corresponding channel would be Red Hat Enterprise Linux Server Debuginfo (v. 6).

Red Hat Enterprise Linux system packages are compiled with optimizations (gcc option -O2). This means that some variables will be displayed as <optimized out>. Stepping through code will 'jump' a little but a crash can still be analyzed. If some debugging information is missing because of the optimizations, the right variable information can be found by disassembling the code and matching it to the source manually. This is applicable only in exceptional cases and is not suitable for regular debugging.

For system packages, GDB informs the user if it is missing some debuginfo packages that limit its functionality.

A core file is a representation of the memory image at the time of a process crash. For bug reporting of system program crashes, Red Hat recommends the use of the ABRT tool, explained in the Automatic Bug Reporting Tool chapter in the Red Hat Deployment Guide. If ABRT is not suitable for your purposes, the steps it automates are explained here.

If the ulimit -c unlimited setting is in use when a process crashes, the core file is dumped into the current directory. The core file contains only the memory areas modified by the process from the original state of disk files. In order to perform a full analysis of a crash, a core file is required to have:

the core file itself

the executable binary which has crashed, such as /usr/sbin/sendmail

all the shared libraries loaded in the binary when it crashed

.debug files and source files (both stored in debuginfo RPMs) for the executable and all of its loaded libraries

For a proper analysis, either the exact version-release.architecture for all the RPMs involved or the same build of your own compiled binaries is needed. At the time of the crash, the application may have already recompiled or been updated by yum on the disk, rendering the files inappropriate for the core file analysis.

The in-memory address where the specific binary was mapped to (for example, 0x400000 in the first line).

The size of the binary (for example, +0x207000 in the first line).

The 160-bit SHA-1 build-id of the binary (for example, 2818b2009547f780a5639c904cded443e564973e in the first line).

The in-memory address where the build-id bytes were stored (for example, @0x400284 in the first line).

The on-disk binary file, if available (for example, usr/bin/sleep in the first line). This was found by eu-unstrip for this module.

The on-disk debuginfo file, if available (for example, /usr/lib/debug/bin/sleep.debug). However, best practice is to use the binary file reference instead.

The shared library name as stored in the shared library list in the core file (for example, libc.so.6 in the third line).

For each build-id (for example, ab/cdef0123456789012345678901234567890123) a symbolic link is included in its debuginfo RPM. Using the /usr/bin/sleep executable above as an example, the coreutils-debuginfo RPM contains, among other files:

In some cases (such as loading a core file), GDB does not know the name, version, or release of a name-debuginfo-version-release.rpm package; it only knows the build-id. In such cases, GDB suggests a different command:

The version-release.architecture of the binary package packagename-debuginfo-version-release.architecture.rpm must be an exact match. If it differs then GDB cannot use the debuginfo package. Even the same version-release.architecture from a different build leads to an incompatible debuginfo package. If GDB reports a missing debuginfo, ensure to recheck:

rpm -q packagename packagename-debuginfo

The version-release.architecture definitions should match.

rpm -V packagename packagename-debuginfo

This command should produce no output, except possibly modified configuration files of packagename, for example.

rpm -qi packagename packagename-debuginfo

The version-release.architecture should display matching information for Vendor, Build Date, and Build Host. For example, using a CentOS debuginfo RPM for a Red Hat Enterprise Linux RPM package will not work.

If the required build-id is known, the following command can query which RPM contains it:

Similar methods are available if the binaries are not packaged into RPMs and stored in yum repositories. It is possible to create local repositories with custom application builds by using /usr/bin/createrepo.