This chapter describes advanced aspects of the Valgrind core
services, which are mostly of interest to power users who wish to
customise and modify Valgrind's default behaviours in certain useful
ways. The subjects covered are:

The "Client Request" mechanism

Debugging your program using Valgrind's gdbserver
and GDB

Function Wrapping

3.1. The Client Request mechanism

Valgrind has a trapdoor mechanism via which the client
program can pass all manner of requests and queries to Valgrind
and the current tool. Internally, this is used extensively
to make various things work, although that's not visible from the
outside.

For your convenience, a subset of these so-called client
requests is provided to allow you to tell Valgrind facts about
the behaviour of your program, and also to make queries.
In particular, your program can tell Valgrind about things that it
otherwise would not know, leading to better results.

Clients need to include a header file to make this work.
Which header file depends on which client requests you use. Some
client requests are handled by the core, and are defined in the
header file valgrind/valgrind.h. Tool-specific
header files are named after the tool, e.g.
valgrind/memcheck.h. Each tool-specific header file
includes valgrind/valgrind.h so you don't need to
include it in your client if you include a tool-specific header. All header
files can be found in the include/valgrind directory of
wherever Valgrind was installed.

The macros in these header files have the magical property
that they generate code in-line which Valgrind can spot.
However, the code does nothing when not run on Valgrind, so you
are not forced to run your program under Valgrind just because you
use the macros in this file. Also, you are not required to link your
program with any extra supporting libraries.

The code added to your binary has negligible performance impact:
on x86, amd64, ppc32, ppc64 and ARM, the overhead is 6 simple integer
instructions and is probably undetectable except in tight loops.
However, if you really wish to compile out the client requests, you
can compile with -DNVALGRIND (analogous to
-DNDEBUG's effect on
assert).

You are encouraged to copy the valgrind/*.h headers
into your project's include directory, so your program doesn't have a
compile-time dependency on Valgrind being installed. The Valgrind headers,
unlike most of the rest of the code, are under a BSD-style license so you may
include them without worrying about license incompatibility.

Here is a brief description of the macros available in
valgrind.h, which work with more than one
tool (see the tool-specific documentation for explanations of the
tool-specific macros).

RUNNING_ON_VALGRIND:

Returns 1 if running on Valgrind, 0 if running on the
real CPU. If you are running Valgrind on itself, returns the
number of layers of Valgrind emulation you're running on.

VALGRIND_DISCARD_TRANSLATIONS:

Discards translations of code in the specified address
range. Useful if you are debugging a JIT compiler or some other
dynamic code generation system. After this call, attempts to
execute code in the invalidated address range will cause
Valgrind to make new translations of that code, which is
probably the semantics you want. Note that code invalidations
are expensive because finding all the relevant translations
quickly is very difficult, so try not to call it often.
Note that you can be clever about
this: you only need to call it when an area which previously
contained code is overwritten with new code. You can choose
to write code into fresh memory, and just call this
occasionally to discard large chunks of old code all at
once.

Alternatively, for transparent self-modifying-code support,
use--smc-check=all, or run
on ppc32/Linux, ppc64/Linux or ARM/Linux.

VALGRIND_COUNT_ERRORS:

Returns the number of errors found so far by Valgrind. Can be
useful in test harness code when combined with the
--log-fd=-1 option; this runs Valgrind silently,
but the client program can detect when errors occur. Only useful
for tools that report errors, e.g. it's useful for Memcheck, but for
Cachegrind it will always return zero because Cachegrind doesn't
report errors.

VALGRIND_MALLOCLIKE_BLOCK:

If your program manages its own memory instead of using
the standard malloc /
new /
new[], tools that track
information about heap blocks will not do nearly as good a
job. For example, Memcheck won't detect nearly as many
errors, and the error messages won't be as informative. To
improve this situation, use this macro just after your custom
allocator allocates some new memory. See the comments in
valgrind.h for information on how to use
it.

VALGRIND_FREELIKE_BLOCK:

This should be used in conjunction with
VALGRIND_MALLOCLIKE_BLOCK.
Again, see valgrind.h for
information on how to use it.

VALGRIND_RESIZEINPLACE_BLOCK:

Informs a Valgrind tool that the size of an allocated block has been
modified but not its address. See valgrind.h for
more information on how to use it.

These are similar to
VALGRIND_MALLOCLIKE_BLOCK and
VALGRIND_FREELIKE_BLOCK
but are tailored towards code that uses memory pools. See
Memory Pools for a detailed description.

VALGRIND_NON_SIMD_CALL[0123]:

Executes a function in the client program on the
real CPU, not the virtual CPU that Valgrind
normally runs code on. The function must take an integer (holding a
thread ID) as the first argument and then 0, 1, 2 or 3 more arguments
(depending on which client request is used). These are used in various
ways internally to Valgrind. They might be useful to client
programs.

Warning: Only use these if you
really know what you are doing. They aren't
entirely reliable, and can cause Valgrind to crash. See
valgrind.h for more details.

VALGRIND_PRINTF(format, ...):

Print a printf-style message to the Valgrind log file. The
message is prefixed with the PID between a pair of
** markers. (Like all client requests,
nothing is output if the client program is not running under Valgrind.)
Output is not produced until a newline is encountered, or subsequent
Valgrind output is printed; this allows you to build up a single line of
output over multiple calls. Returns the number of characters output,
excluding the PID prefix.

VALGRIND_PRINTF_BACKTRACE(format, ...):

Like VALGRIND_PRINTF (in
particular, the return value is identical), but prints a stack backtrace
immediately afterwards.

VALGRIND_MONITOR_COMMAND(command):

Execute the given monitor command (a string).
Returns 0 if command is recognised. Returns 1 if command is not recognised.
Note that some monitor commands provide access to a functionality
also accessible via a specific client request. For example,
memcheck leak search can be requested from the client program
using VALGRIND_DO_LEAK_CHECK or via the monitor command "leak_search".
Note that the syntax of the command string is only verified at
run-time. So, if it exists, it is preferrable to use a specific
client request to have better compile time verifications of the
arguments.

VALGRIND_STACK_REGISTER(start, end):

Registers a new stack. Informs Valgrind that the memory range
between start and end is a unique stack. Returns a stack identifier
that can be used with other
VALGRIND_STACK_* calls.

Valgrind will use this information to determine if a change
to the stack pointer is an item pushed onto the stack or a change
over to a new stack. Use this if you're using a user-level thread
package and are noticing crashes in stack trace recording or
spurious errors from Valgrind about uninitialized memory
reads.

Warning: Unfortunately, this client request is
unreliable and best avoided.

Warning: Unfortunately, this client request is
unreliable and best avoided.

VALGRIND_STACK_CHANGE(id, start, end):

Changes a previously registered stack. Informs
Valgrind that the previously registered stack with stack id
id has changed its start and end
values. Use this if your user-level thread package implements
stack growth.

Warning: Unfortunately, this client request is
unreliable and best avoided.

3.2. Debugging your program using Valgrind gdbserver and GDB

A program running under Valgrind is not executed directly by the
CPU. Instead it runs on a synthetic CPU provided by Valgrind. This is
why a debugger cannot debug your program when it runs on Valgrind.

This section describes how GDB can interact with the
Valgrind gdbserver to provide a fully debuggable program under
Valgrind. Used in this way, GDB also provides an interactive usage of
Valgrind core or tool functionalities, including incremental leak search
under Memcheck and on-demand Massif snapshot production.

3.2.1. Quick Start: debugging in 3 steps

The simplest way to get started is to run Valgrind with the
flag --vgdb-error=0. Then follow the on-screen
directions, which give you the precise commands needed to start GDB
and connect it to your program.

Otherwise, here's a slightly more verbose overview.

If you want to debug a program with GDB when using the Memcheck
tool, start Valgrind like this:

valgrind --vgdb=yes --vgdb-error=0 prog

In another shell, start GDB:

gdb prog

Then give the following command to GDB:

(gdb) target remote | vgdb

You can now debug your program e.g. by inserting a breakpoint
and then using the GDB continue
command.

This quick start information is enough for basic usage of the
Valgrind gdbserver. The sections below describe more advanced
functionality provided by the combination of Valgrind and GDB. Note
that the command line flag --vgdb=yes can be omitted,
as this is the default value.

3.2.2. Valgrind gdbserver overall organisation

The GNU GDB debugger is typically used to debug a process
running on the same machine. In this mode, GDB uses system calls to
control and query the program being debugged. This works well, but
only allows GDB to debug a program running on the same computer.

GDB can also debug processes running on a different computer.
To achieve this, GDB defines a protocol (that is, a set of query and
reply packets) that facilitates fetching the value of memory or
registers, setting breakpoints, etc. A gdbserver is an implementation
of this "GDB remote debugging" protocol. To debug a process running
on a remote computer, a gdbserver (sometimes called a GDB stub)
must run at the remote computer side.

The Valgrind core provides a built-in gdbserver implementation,
which is activated using --vgdb=yes
or --vgdb=full. This gdbserver allows the process
running on Valgrind's synthetic CPU to be debugged remotely.
GDB sends protocol query packets (such as "get register contents") to
the Valgrind embedded gdbserver. The gdbserver executes the queries
(for example, it will get the register values of the synthetic CPU)
and gives the results back to GDB.

GDB can use various kinds of channels (TCP/IP, serial line, etc)
to communicate with the gdbserver. In the case of Valgrind's
gdbserver, communication is done via a pipe and a small helper program
called vgdb, which acts as an
intermediary. If no GDB is in use, vgdb can also be
used to send monitor commands to the Valgrind gdbserver from a shell
command line.

3.2.3. Connecting GDB to a Valgrind gdbserver

To debug a program "prog" running under
Valgrind, you must ensure that the Valgrind gdbserver is activated by
specifying either --vgdb=yes
or --vgdb=full. A secondary command line option,
--vgdb-error=number, can be used to tell the gdbserver
only to become active once the specified number of errors have been
shown. A value of zero will therefore cause
the gdbserver to become active at startup, which allows you to
insert breakpoints before starting the run. For example:

valgrind --tool=memcheck --vgdb=yes --vgdb-error=0 ./prog

The Valgrind gdbserver is invoked at startup
and indicates it is waiting for a connection from a GDB:

Note that vgdb is provided as part of the Valgrind
distribution. You do not need to install it separately.

If vgdb detects that there are multiple Valgrind gdbservers that
can be connected to, it will list all such servers and their PIDs, and
then exit. You can then reissue the GDB "target" command, but
specifying the PID of the process you want to debug:

Once GDB is connected to the Valgrind gdbserver, it can be used
in the same way as if you were debugging the program natively:

Breakpoints can be inserted or deleted.

Variables and register values can be examined or modified.

Signal handling can be configured (printing, ignoring).

Execution can be controlled (continue, step, next, stepi, etc).

Program execution can be interrupted using Control-C.

And so on. Refer to the GDB user manual for a complete
description of GDB's functionality.

3.2.4. Connecting to an Android gdbserver

When developping applications for Android, you will typically use
a development system (on which the Android NDK is installed) to compile your
application. An Android target system or emulator will be used to run
the application.
In this setup, Valgrind and vgdb will run on the Android system,
while GDB will run on the development system. GDB will connect
to the vgdb running on the Android system using the Android NDK
'adb forward' application.

Example: on the Android system, execute the following:

valgrind --vgdb-error=0 --vgdb=yes prog
# and then in another shell, run:
vgdb --port=1234

On the development system, execute the following commands:

adb forward tcp:1234 tcp:1234
gdb prog
(gdb) target remote :1234

GDB will use a local tcp/ip connection to connect to the Android adb forwarder.
Adb will establish a relay connection between the host system and the Android
target system. Be sure to use the GDB delivered in the
Android NDK system (typically, arm-linux-androideabi-gdb), as the host
GDB is probably not able to debug Android arm applications.
Note that the local port nr (used by GDB) must not necessarily be equal
to the port number used by vgdb: adb can forward tcp/ip between different
port numbers.

In the current release, the GDB server is not enabled by default
for Android, due to problems in establishing a suitable directory in
which Valgrind can create the necessary FIFOs (named pipes) for
communication purposes. You can stil try to use the GDB server, but
you will need to explicitly enable it using the flag
--vgdb=yes or
--vgdb=full.

Additionally, you
will need to select a temporary directory which is (a) writable
by Valgrind, and (b) supports FIFOs. This is the main difficult
point. Often, /sdcard satisfies
requirement (a), but fails for (b) because it is a VFAT file system
and VFAT does not support pipes. Possibilities you could try are
/data/local,
/data/local/Inst (if you
installed Valgrind there), or
/data/data/name.of.my.app, if you
are running a specific application and it has its own directory of
that form. This last possibility may have the highest probability
of success.

You can specify the temporary directory to use either via
the --with-tmpdir= configure time
flag, or by setting environment variable TMPDIR when running Valgrind
(on the Android device, not on the Android NDK development host).
Another alternative is to specify the directory for the FIFOs using
the --vgdb-prefix= Valgrind command
line option.

We hope to have a better story for temporary directory handling
on Android in the future. The difficulty is that, unlike in standard
Unixes, there is no single temporary file directory that reliably
works across all devices and scenarios.

3.2.5. Monitor command handling by the Valgrind gdbserver

The Valgrind gdbserver provides additional Valgrind-specific
functionality via "monitor commands". Such monitor commands can be
sent from the GDB command line or from the shell command line or
requested by the client program using the VALGRIND_MONITOR_COMMAND
client request. See
Valgrind monitor commands for the
list of the Valgrind core monitor commands available regardless of the
Valgrind tool selected.

An example of a tool specific monitor command is the Memcheck monitor
command leak_check full
reachable any. This requests a full reporting of the
allocated memory blocks. To have this leak check executed, use the GDB
command:

(gdb) monitor leak_check full reachable any

GDB will send the leak_check
command to the Valgrind gdbserver. The Valgrind gdbserver will
execute the monitor command itself, if it recognises it to be a Valgrind core
monitor command. If it is not recognised as such, it is assumed to
be tool-specific and is handed to the tool for execution. For example:

As with other GDB commands, the Valgrind gdbserver will accept
abbreviated monitor command names and arguments, as long as the given
abbreviation is unambiguous. For example, the above
leak_check
command can also be typed as:

(gdb) mo l f r a

The letters mo are recognised by GDB as being
an abbreviation for monitor. So GDB sends the
string l f r a to the Valgrind
gdbserver. The letters provided in this string are unambiguous for the
Valgrind gdbserver. This therefore gives the same output as the
unabbreviated command and arguments. If the provided abbreviation is
ambiguous, the Valgrind gdbserver will report the list of commands (or
argument values) that can match:

Instead of sending a monitor command from GDB, you can also send
these from a shell command line. For example, the following command
lines, when given in a shell, will cause the same leak search to be executed
by the process 3145:

vgdb --pid=3145 leak_check full reachable any
vgdb --pid=3145 l f r a

Note that the Valgrind gdbserver automatically continues the
execution of the program after a standalone invocation of
vgdb. Monitor commands sent from GDB do not cause the program to
continue: the program execution is controlled explicitly using GDB
commands such as "continue" or "next".

3.2.6. Valgrind gdbserver thread information

Valgrind's gdbserver enriches the output of the
GDB info threads command
with Valgrind-specific information.
The operating system's thread number is followed
by Valgrind's internal index for that thread ("tid") and by
the Valgrind scheduler thread state:

3.2.7. Examining and modifying Valgrind shadow registers

When the option --vgdb-shadow-registers=yes is
given, the Valgrind gdbserver will let GDB examine and/or modify
Valgrind's shadow registers. GDB version 7.1 or later is needed for this
to work. For x86 and amd64, GDB version 7.2 or later is needed.

For each CPU register, the Valgrind core maintains two
shadow register sets. These shadow registers can be accessed from
GDB by giving a postfix s1
or s2 for respectively the first
and second shadow register. For example, the x86 register
eax and its two shadows
can be examined using the following commands:

(gdb) p $eax
$1 = 0
(gdb) p $eaxs1
$2 = 0
(gdb) p $eaxs2
$3 = 0
(gdb)

Float shadow registers are shown by GDB as unsigned integer
values instead of float values, as it is expected that these
shadow values are mostly used for memcheck validity bits.

Intel/amd64 AVX registers ymm0
to ymm15 have also their shadow
registers. However, GDB presents the shadow values using two
"half" registers. For example, the half shadow registers for
ymm9 are
xmm9s1 (lower half for set 1),
ymm9hs1 (upper half for set 1),
xmm9s2 (lower half for set 2),
ymm9hs2 (upper half for set 2).
Note the inconsistent notation for the names of the half registers:
the lower part starts with an x,
the upper part starts with an y
and has an h before the shadow postfix.

The special presentation of the AVX shadow registers is due to
the fact that GDB independently retrieves the lower and upper half of
the ymm registers. GDB does not
however know that the shadow half registers have to be shown combined.

3.2.8. Limitations of the Valgrind gdbserver

Debugging with the Valgrind gdbserver is very similar to native
debugging. Valgrind's gdbserver implementation is quite
complete, and so provides most of the GDB debugging functionality. There
are however some limitations and peculiarities:

Precision of "stop-at" commands.

GDB commands such as "step", "next", "stepi", breakpoints
and watchpoints, will stop the execution of the process. With
the option --vgdb=yes, the process might not
stop at the exact requested instruction. Instead, it might
continue execution of the current basic block and stop at one
of the following basic blocks. This is linked to the fact that
Valgrind gdbserver has to instrument a block to allow stopping
at the exact instruction requested. Currently,
re-instrumentation of the block currently being executed is not
supported. So, if the action requested by GDB (e.g. single
stepping or inserting a breakpoint) implies re-instrumentation
of the current block, the GDB action may not be executed
precisely.

This limitation applies when the basic block
currently being executed has not yet been instrumented for debugging.
This typically happens when the gdbserver is activated due to the
tool reporting an error or to a watchpoint. If the gdbserver
block has been activated following a breakpoint, or if a
breakpoint has been inserted in the block before its execution,
then the block has already been instrumented for debugging.

If you use the option --vgdb=full, then GDB
"stop-at" commands will be obeyed precisely. The
downside is that this requires each instruction to be
instrumented with an additional call to a gdbserver helper
function, which gives considerable overhead (+500% for memcheck)
compared to --vgdb=no.
Option --vgdb=yes has neglectible overhead compared
to --vgdb=no.

Processor registers and flags values.

When Valgrind gdbserver stops on an error, on a breakpoint
or when single stepping, registers and flags values might not be always
up to date due to the optimisations done by the Valgrind core.
The default value
--vex-iropt-register-updates=unwindregs-at-mem-access
ensures that the registers needed to make a stack trace (typically
PC/SP/FP) are up to date at each memory access (i.e. memory exception
points).
Disabling some optimisations using the following values will increase
the precision of registers and flags values (a typical performance
impact for memcheck is given for each option).

--vex-iropt-register-updates=allregs-at-mem-access (+10%)
ensures that all registers and flags are up to date at each memory
access.

--vex-iropt-register-updates=allregs-at-each-insn (+25%)
ensures that all registers and flags are up to date at each instruction.

The Valgrind gdbserver can simulate hardware watchpoints
if the selected tool provides support for it. Currently,
only Memcheck provides hardware watchpoint simulation. The
hardware watchpoint simulation provided by Memcheck is much
faster that GDB software watchpoints, which are implemented by
GDB checking the value of the watched zone(s) after each
instruction. Hardware watchpoint simulation also provides read
watchpoints. The hardware watchpoint simulation by Memcheck has
some limitations compared to real hardware
watchpoints. However, the number and length of simulated
watchpoints are not limited.

Typically, the number of (real) hardware watchpoints is
limited. For example, the x86 architecture supports a maximum of
4 hardware watchpoints, each watchpoint watching 1, 2, 4 or 8
bytes. The Valgrind gdbserver does not have any limitation on the
number of simulated hardware watchpoints. It also has no
limitation on the length of the memory zone being
watched. Using GDB version 7.4 or later allow full use of the
flexibility of the Valgrind gdbserver's simulated hardware watchpoints.
Previous GDB versions do not understand that Valgrind gdbserver
watchpoints have no length limit.

Memcheck implements hardware watchpoint simulation by
marking the watched address ranges as being unaddressable. When
a hardware watchpoint is removed, the range is marked as
addressable and defined. Hardware watchpoint simulation of
addressable-but-undefined memory zones works properly, but has
the undesirable side effect of marking the zone as defined when
the watchpoint is removed.

Write watchpoints might not be reported at the
exact instruction that writes the monitored area,
unless option --vgdb=full is given. Read watchpoints
will always be reported at the exact instruction reading the
watched memory.

It is better to avoid using hardware watchpoint of not
addressable (yet) memory: in such a case, GDB will fall back to
extremely slow software watchpoints. Also, if you do not quit GDB
between two debugging sessions, the hardware watchpoints of the
previous sessions will be re-inserted as software watchpoints if
the watched memory zone is not addressable at program startup.

Stepping inside shared libraries on ARM.

For unknown reasons, stepping inside shared
libraries on ARM may fail. A workaround is to use the
ldd command
to find the list of shared libraries and their loading address
and inform GDB of the loading address using the GDB command
"add-symbol-file". Example:

You must use a GDB version which is able to read XML
target description sent by a gdbserver. This is the standard setup
if GDB was configured and built with the "expat"
library. If your GDB was not configured with XML support, it
will report an error message when using the "target"
command. Debugging will not work because GDB will then not be
able to fetch the registers from the Valgrind gdbserver.
For ARM programs using the Thumb instruction set, you must use
a GDB version of 7.1 or later, as earlier versions have problems
with next/step/breakpoints in Thumb code.

Stack unwinding on PPC32/PPC64.

On PPC32/PPC64, stack unwinding for leaf functions
(functions that do not call any other functions) works properly
only when you give the option
--vex-iropt-register-updates=allregs-at-mem-access
or --vex-iropt-register-updates=allregs-at-each-insn.
You must also pass this option in order to get a precise stack when
a signal is trapped by GDB.

Breakpoints encountered multiple times.

Some instructions (e.g. x86 "rep movsb")
are translated by Valgrind using a loop. If a breakpoint is placed
on such an instruction, the breakpoint will be encountered
multiple times -- once for each step of the "implicit" loop
implementing the instruction.

Execution of Inferior function calls by the Valgrind
gdbserver.

GDB allows the user to "call" functions inside the process
being debugged. Such calls are named "inferior calls" in the GDB
terminology. A typical use of an inferior call is to execute
a function that prints a human-readable version of a complex data
structure. To make an inferior call, use the GDB "print" command
followed by the function to call and its arguments. As an
example, the following GDB command causes an inferior call to the
libc "printf" function to be executed by the process
being debugged:

The Valgrind gdbserver supports inferior function calls.
Whilst an inferior call is running, the Valgrind tool will report
errors as usual. If you do not want to have such errors stop the
execution of the inferior call, you can
use v.set vgdb-error to set a
big value before the call, then manually reset it to its original
value when the call is complete.

To execute inferior calls, GDB changes registers such as
the program counter, and then continues the execution of the
program. In a multithreaded program, all threads are continued,
not just the thread instructed to make the inferior call. If
another thread reports an error or encounters a breakpoint, the
evaluation of the inferior call is abandoned.

Note that inferior function calls are a powerful GDB
feature, but should be used with caution. For example, if
the program being debugged is stopped inside the function "printf",
forcing a recursive call to printf via an inferior call will
very probably create problems. The Valgrind tool might also add
another level of complexity to inferior calls, e.g. by reporting
tool errors during the Inferior call or due to the
instrumentation done.

Connecting to or interrupting a Valgrind process blocked in
a system call.

Connecting to or interrupting a Valgrind process blocked in
a system call requires the "ptrace" system call to be usable.
This may be disabled in your kernel for security reasons.

When running your program, Valgrind's scheduler
periodically checks whether there is any work to be handled by
the gdbserver. Unfortunately this check is only done if at least
one thread of the process is runnable. If all the threads of the
process are blocked in a system call, then the checks do not
happen, and the Valgrind scheduler will not invoke the gdbserver.
In such a case, the vgdb relay application will "force" the
gdbserver to be invoked, without the intervention of the Valgrind
scheduler.

Such forced invocation of the Valgrind gdbserver is
implemented by vgdb using ptrace system calls. On a properly
implemented kernel, the ptrace calls done by vgdb will not
influence the behaviour of the program running under Valgrind.
If however they do, giving the
option --max-invoke-ms=0 to the vgdb relay
application will disable the usage of ptrace calls. The
consequence of disabling ptrace usage in vgdb is that a Valgrind
process blocked in a system call cannot be woken up or
interrupted from GDB until it executes enough basic blocks to let
the Valgrind scheduler's normal checking take effect.

When ptrace is disabled in vgdb, you can increase the
responsiveness of the Valgrind gdbserver to commands or
interrupts by giving a lower value to the
option --vgdb-poll. If your application is
blocked in system calls most of the time, using a very low value
for --vgdb-poll will cause a the gdbserver to be
invoked sooner. The gdbserver polling done by Valgrind's
scheduler is very efficient, so the increased polling frequency
should not cause significant performance degradation.

When ptrace is disabled in vgdb, a query packet sent by GDB
may take significant time to be handled by the Valgrind
gdbserver. In such cases, GDB might encounter a protocol
timeout. To avoid this,
you can increase the value of the timeout by using the GDB
command "set remotetimeout".

Ubuntu versions 10.10 and later may restrict the scope of
ptrace to the children of the process calling ptrace. As the
Valgrind process is not a child of vgdb, such restricted scoping
causes the ptrace calls to fail. To avoid that, Valgrind will
automatically allow all processes belonging to the same userid to
"ptrace" a Valgrind process, by using PR_SET_PTRACER.

Unblocking processes blocked in system calls is not
currently implemented on Mac OS X and Android. So you cannot
connect to or interrupt a process blocked in a system call on Mac
OS X or Android.

Changing register values.

The Valgrind gdbserver will only modify the values of the
thread's registers when the thread is in status Runnable or
Yielding. In other states (typically, WaitSys), attempts to
change register values will fail. Amongst other things, this
means that inferior calls are not executed for a thread which is
in a system call, since the Valgrind gdbserver does not implement
system call restart.

Unsupported GDB functionality.

GDB provides a lot of debugging functionality and not all
of it is supported. Specifically, the following are not
supported: reversible debugging and tracepoints.

Unknown limitations or problems.

The combination of GDB, Valgrind and the Valgrind gdbserver
probably has unknown other limitations and problems. If you
encounter strange or unexpected behaviour, feel free to report a
bug. But first please verify that the limitation or problem is
not inherent to GDB or the GDB remote protocol. You may be able
to do so by checking the behaviour when using standard gdbserver
part of the GDB package.

3.2.9. vgdb command line options

Usage: vgdb [OPTION]... [[-c] COMMAND]...

vgdb ("Valgrind to GDB") is a small program that is used as an
intermediary between Valgrind and GDB or a shell.
Therefore, it has two usage modes:

As a standalone utility, it is used from a shell command
line to send monitor commands to a process running under
Valgrind. For this usage, the vgdb OPTION(s) must be followed by
the monitor command to send. To send more than one command,
separate them with the -c option.

In combination with GDB "target remote |" command, it is
used as the relay application between GDB and the Valgrind
gdbserver. For this usage, only OPTION(s) can be given, but no
COMMAND can be given.

vgdb accepts the following
options:

--pid=<number>

Specifies the PID of
the process to which vgdb must connect to. This option is useful
in case more than one Valgrind gdbserver can be connected to. If
the --pid argument is not given and multiple
Valgrind gdbserver processes are running, vgdb will report the
list of such processes and then exit.

--vgdb-prefix

Must be given to both
Valgrind and vgdb if you want to change the default prefix for the
FIFOs (named pipes) used for communication between the Valgrind
gdbserver and vgdb.

--wait=<number>

Instructs vgdb to
search for available Valgrind gdbservers for the specified number
of seconds. This makes it possible start a vgdb process
before starting the Valgrind gdbserver with which you intend the
vgdb to communicate. This option is useful when used in
conjunction with a --vgdb-prefix that is
unique to the process you want to wait for.
Also, if you use the --wait argument in the GDB
"target remote" command, you must set the GDB remotetimeout to a
value bigger than the --wait argument value. See option
--max-invoke-ms (just below)
for an example of setting the remotetimeout value.

--max-invoke-ms=<number>

Gives the
number of milliseconds after which vgdb will force the invocation
of gdbserver embedded in Valgrind. The default value is 100
milliseconds. A value of 0 disables forced invocation. The forced
invocation is used when vgdb is connected to a Valgrind gdbserver,
and the Valgrind process has all its threads blocked in a system
call.

If you specify a large value, you might need to increase the
GDB "remotetimeout" value from its default value of 2 seconds.
You should ensure that the timeout (in seconds) is
bigger than the --max-invoke-ms value. For
example, for --max-invoke-ms=5000, the following
GDB command is suitable:

(gdb) set remotetimeout 6

--cmd-time-out=<number>

Instructs a
standalone vgdb to exit if the Valgrind gdbserver it is connected
to does not process a command in the specified number of seconds.
The default value is to never time out.

--port=<portnr>

Instructs vgdb to
use tcp/ip and listen for GDB on the specified port nr rather than
to use a pipe to communicate with GDB. Using tcp/ip allows to have
GDB running on one computer and debugging a Valgrind process
running on another target computer.
Example:

# On the target computer, start your program under valgrind using
valgrind --vgdb-error=0 prog
# and then in another shell, run:
vgdb --port=1234

On the computer which hosts GDB, execute the command:

gdb prog
(gdb) target remote targetip:1234

where targetip is the ip address or hostname of the target computer.

-c

To give more than one command to a
standalone vgdb, separate the commands by an
option -c. Example:

vgdb v.set log_output -c leak_check any

-l

Instructs a standalone vgdb to report
the list of the Valgrind gdbserver processes running and then
exit.

-D

Instructs a standalone vgdb to show the
state of the shared memory used by the Valgrind gdbserver. vgdb
will exit after having shown the Valgrind gdbserver shared memory
state.

-d

Instructs vgdb to produce debugging
output. Give multiple -d args to increase the
verbosity. When giving -d to a relay vgdb, you better
redirect the standard error (stderr) of vgdb to a file to avoid
interaction between GDB and vgdb debugging output.

The monitor commands can be sent either from a shell command line, by using a
standalone vgdb, or from GDB, by using GDB's "monitor"
command (see Monitor command handling by the Valgrind gdbserver).
They can also be launched by the client program, using the VALGRIND_MONITOR_COMMAND
client request.

help [debug] instructs Valgrind's gdbserver
to give the list of all monitor commands of the Valgrind core and
of the tool. The optional "debug" argument tells to also give help
for the monitor commands aimed at Valgrind internals debugging.

v.info all_errors shows all errors found
so far.

v.info last_error shows the last error
found.

v.info location <addr> outputs
information about the location <addr>. Possibly, the
following are described: global variables, local (stack)
variables, allocated or freed blocks, ... The information
produced depends on the tool and on the options given to valgrind.
Some tools (e.g. memcheck and helgrind) produce more detailed
information for client heap blocks. For example, these tools show
the stacktrace where the heap block was allocated. If a tool does
not replace the malloc/free/... functions, then client heap blocks
will not be described. Use the
option --read-var-info=yes to obtain more
detailed information about global or local (stack) variables.

v.info n_errs_found [msg] shows the number of
errors found so far, the nr of errors shown so far and the current
value of the --vgdb-error argument. The optional
msg (one or more words) is appended.
Typically, this can be used to insert markers in a process output
file between several tests executed in sequence by a process
started only once. This allows to associate the errors reported
by Valgrind with the specific test that produced these errors.

v.info open_fds shows the list of open file
descriptors and details related to the file descriptor.
This only works if --track-fds=yes
was given at Valgrind startup.

With mixed_output, the
Valgrind output goes to the Valgrind log (typically stderr) while
the output of the interactive GDB monitor commands (e.g.
v.info last_error)
is displayed by GDB.

With gdb_output, both the
Valgrind output and the interactive GDB monitor commands output are
displayed by GDB.

With log_output, both the
Valgrind output and the interactive GDB monitor commands output go
to the Valgrind log.

v.wait [ms (default 0)] instructs
Valgrind gdbserver to sleep "ms" milli-seconds and then
continue. When sent from a standalone vgdb, if this is the last
command, the Valgrind process will continue the execution of the
guest process. The typical usage of this is to use vgdb to send a
"no-op" command to a Valgrind gdbserver so as to continue the
execution of the guest process.

v.kill requests the gdbserver to kill
the process. This can be used from a standalone vgdb to properly
kill a Valgrind process which is currently expecting a vgdb
connection.

v.set vgdb-error <errornr>
dynamically changes the value of the
--vgdb-error argument. A
typical usage of this is to start with
--vgdb-error=0 on the
command line, then set a few breakpoints, set the vgdb-error value
to a huge value and continue execution.

The following Valgrind monitor commands are useful for
investigating the behaviour of Valgrind or its gdbserver in case of
problems or bugs.

v.do expensive_sanity_check_general
executes various sanity checks. In particular, the sanity of the
Valgrind heap is verified. This can be useful if you suspect that
your program and/or Valgrind has a bug corrupting Valgrind data
structure. It can also be used when a Valgrind tool
reports a client error to the connected GDB, in order to verify
the sanity of Valgrind before continuing the execution.

v.info gdbserver_status shows the
gdbserver status. In case of problems (e.g. of communications),
this shows the values of some relevant Valgrind gdbserver internal
variables. Note that the variables related to breakpoints and
watchpoints (e.g. the number of breakpoint addresses and the number of
watchpoints) will be zero, as GDB by default removes all
watchpoints and breakpoints when execution stops, and re-inserts
them when resuming the execution of the debugged process. You can
change this GDB behaviour by using the GDB command
set breakpoint always-inserted on.

v.info memory [aspacemgr] shows the statistics of
Valgrind's internal heap management. If
option --profile-heap=yes was given, detailed
statistics will be output. With the optional argument
aspacemgr. the segment list maintained
by valgrind address space manager will be output. Note that
this list of segments is always output on the Valgrind log.

v.info exectxt shows informations about
the "executable contexts" (i.e. the stack traces) recorded by
Valgrind. For some programs, Valgrind can record a very high
number of such stack traces, causing a high memory usage. This
monitor command shows all the recorded stack traces, followed by
some statistics. This can be used to analyse the reason for having
a big number of stack traces. Typically, you will use this command
if v.info memory has shown significant memory
usage by the "exectxt" arena.

v.info scheduler shows various
information about threads. First, it outputs the host stack trace,
i.e. the Valgrind code being executed. Then, for each thread, it
outputs the thread state. For non terminated threads, the state is
followed by the guest (client) stack trace. Finally, for each
active thread or for each terminated thread slot not yet re-used,
it shows the max usage of the valgrind stack.

Showing the client stack traces allows to compare the stack
traces produced by the Valgrind unwinder with the stack traces
produced by GDB+Valgrind gdbserver. Pay attention that GDB and
Valgrind scheduler status have their own thread numbering
scheme. To make the link between the GDB thread number and the
corresponding Valgrind scheduler thread number, use the GDB
command info threads. The output
of this command shows the GDB thread number and the valgrind
'tid'. The 'tid' is the thread number output
by v.info scheduler. When using
the callgrind tool, the callgrind monitor command
status outputs internal callgrind
information about the stack/call graph it maintains.

v.info stats shows various valgrind core and
tool statistics. With this, Valgrind and tool statistics can
be examined while running, even without option --stats=yes.

v.set debuglog <intvalue> sets the
Valgrind debug log level to <intvalue>. This allows to
dynamically change the log level of Valgrind e.g. when a problem
is detected.

v.set hostvisibility [yes*|no] The value
"yes" indicates to gdbserver that GDB can look at the Valgrind
'host' (internal) status/memory. "no" disables this access.
When hostvisibility is activated, GDB can e.g. look at Valgrind
global variables. As an example, to examine a Valgrind global
variable of the memcheck tool on an x86, do the following setup:

v.translate <address>
[<traceflags>] shows the translation of the block
containing address with the given
trace flags. The traceflags value
bit patterns have similar meaning to Valgrind's
--trace-flags option. It can be given
in hexadecimal (e.g. 0x20) or decimal (e.g. 32) or in binary 1s
and 0s bit (e.g. 0b00100000). The default value of the traceflags
is 0b00100000, corresponding to "show after instrumentation".
The output of this command always goes to the Valgrind
log.

The additional bit flag 0b100000000 (bit 8)
has no equivalent in the --trace-flags option.
It enables tracing of the gdbserver specific instrumentation. Note
that this bit 8 can only enable the addition of gdbserver
instrumentation in the trace. Setting it to 0 will not
disable the tracing of the gdbserver instrumentation if it is
active for some other reason, for example because there is a breakpoint at
this address or because gdbserver is in single stepping
mode.

3.3. Function wrapping

Valgrind allows calls to some specified functions to be intercepted and
rerouted to a different, user-supplied function. This can do whatever it
likes, typically examining the arguments, calling onwards to the original,
and possibly examining the result. Any number of functions may be
wrapped.

Function wrapping is useful for instrumenting an API in some way. For
example, Helgrind wraps functions in the POSIX pthreads API so it can know
about thread status changes, and the core is able to wrap
functions in the MPI (message-passing) API so it can know
of memory status changes associated with message arrival/departure.
Such information is usually passed to Valgrind by using client
requests in the wrapper functions, although the exact mechanism may vary.

3.3.1. A Simple Example

Supposing we want to wrap some function

int foo ( int x, int y ) { return x + y; }

A wrapper is a function of identical type, but with a special name
which identifies it as the wrapper for foo.
Wrappers need to include
supporting macros from valgrind.h.
Here is a simple wrapper which prints the arguments and return value:

To become active, the wrapper merely needs to be present in a text
section somewhere in the same process' address space as the function
it wraps, and for its ELF symbol name to be visible to Valgrind. In
practice, this means either compiling to a
.o and linking it in, or
compiling to a .so and
LD_PRELOADing it in. The latter is more
convenient in that it doesn't require relinking.

All wrappers have approximately the above form. There are three
crucial macros:

I_WRAP_SONAME_FNNAME_ZU:
this generates the real name of the wrapper.
This is an encoded name which Valgrind notices when reading symbol
table information. What it says is: I am the wrapper for any function
named foo which is found in
an ELF shared object with an empty
("NONE") soname field. The specification
mechanism is powerful in
that wildcards are allowed for both sonames and function names.
The details are discussed below.

VALGRIND_GET_ORIG_FN:
once in the wrapper, the first priority is
to get hold of the address of the original (and any other supporting
information needed). This is stored in a value of opaque
type OrigFn.
The information is acquired using
VALGRIND_GET_ORIG_FN. It is crucial
to make this macro call before calling any other wrapped function
in the same thread.

CALL_FN_W_WW: eventually we will
want to call the function being
wrapped. Calling it directly does not work, since that just gets us
back to the wrapper and leads to an infinite loop. Instead, the result
lvalue,
OrigFn and arguments are
handed to one of a family of macros of the form
CALL_FN_*. These
cause Valgrind to call the original and avoid recursion back to the
wrapper.

3.3.2. Wrapping Specifications

This scheme has the advantage of being self-contained. A library of
wrappers can be compiled to object code in the normal way, and does
not rely on an external script telling Valgrind which wrappers pertain
to which originals.

Each wrapper has a name which, in the most general case says: I am the
wrapper for any function whose name matches FNPATT and whose ELF
"soname" matches SOPATT. Both FNPATT and SOPATT may contain wildcards
(asterisks) and other characters (spaces, dots, @, etc) which are not
generally regarded as valid C identifier names.

This flexibility is needed to write robust wrappers for POSIX pthread
functions, where typically we are not completely sure of either the
function name or the soname, or alternatively we want to wrap a whole
set of functions at once.

For example, pthread_create
in GNU libpthread is usually a
versioned symbol - one whose name ends in, eg,
@GLIBC_2.3. Hence we
are not sure what its real name is. We also want to cover any soname
of the form libpthread.so*.
So the header of the wrapper will be

In order to write unusual characters as valid C function names, a
Z-encoding scheme is used. Names are written literally, except that
a capital Z acts as an escape character, with the following encoding:

Hence libpthreadZdsoZd0 is an
encoding of the soname libpthread.so.0
and pthreadZucreateZAZa is an encoding
of the function name pthread_create@*.

The macro I_WRAP_SONAME_FNNAME_ZZ
constructs a wrapper name in which
both the soname (first component) and function name (second component)
are Z-encoded. Encoding the function name can be tiresome and is
often unnecessary, so a second macro,
I_WRAP_SONAME_FNNAME_ZU, can be
used instead. The _ZU variant is
also useful for writing wrappers for
C++ functions, in which the function name is usually already mangled
using some other convention in which Z plays an important role. Having
to encode a second time quickly becomes confusing.

Since the function name field may contain wildcards, it can be
anything, including just *.
The same is true for the soname.
However, some ELF objects - specifically, main executables - do not
have sonames. Any object lacking a soname is treated as if its soname
was NONE, which is why the original
example above had a name
I_WRAP_SONAME_FNNAME_ZU(NONE,foo).

Note that the soname of an ELF object is not the same as its
file name, although it is often similar. You can find the soname of
an object libfoo.so using the command
readelf -a libfoo.so | grep soname.

3.3.3. Wrapping Semantics

The ability for a wrapper to replace an infinite family of functions
is powerful but brings complications in situations where ELF objects
appear and disappear (are dlopen'd and dlclose'd) on the fly.
Valgrind tries to maintain sensible behaviour in such situations.

For example, suppose a process has dlopened (an ELF object with
soname) object1.so, which contains
function1. It starts to use
function1 immediately.

After a while it dlopens wrappers.so,
which contains a wrapper
for function1 in (soname)
object1.so. All subsequent calls to
function1 are rerouted to the wrapper.

If wrappers.so is
later dlclose'd, calls to function1 are
naturally routed back to the original.

Alternatively, if object1.so
is dlclose'd but wrappers.so remains,
then the wrapper exported by wrappers.so
becomes inactive, since there
is no way to get to it - there is no original to call any more. However,
Valgrind remembers that the wrapper is still present. If
object1.so is
eventually dlopen'd again, the wrapper will become active again.

In short, valgrind inspects all code loading/unloading events to
ensure that the set of currently active wrappers remains consistent.

A second possible problem is that of conflicting wrappers. It is
easily possible to load two or more wrappers, both of which claim
to be wrappers for some third function. In such cases Valgrind will
complain about conflicting wrappers when the second one appears, and
will honour only the first one.

3.3.4. Debugging

Figuring out what's going on given the dynamic nature of wrapping
can be difficult. The
--trace-redir=yes option makes
this possible
by showing the complete state of the redirection subsystem after
every
mmap/munmap
event affecting code (text).

There are two central concepts:

A "redirection specification" is a binding of
a (soname pattern, fnname pattern) pair to a code address.
These bindings are created by writing functions with names
made with the
I_WRAP_SONAME_FNNAME_{ZZ,_ZU}
macros.

An "active redirection" is a code-address to
code-address binding currently in effect.

The state of the wrapping-and-redirection subsystem comprises a set of
specifications and a set of active bindings. The specifications are
acquired/discarded by watching all
mmap/munmap
events on code (text)
sections. The active binding set is (conceptually) recomputed from
the specifications, and all known symbol names, following any change
to the specification set.

--trace-redir=yes shows the contents
of both sets following any such event.

-v prints a line of text each
time an active specification is used for the first time.

Hence for maximum debugging effectiveness you will need to use both
options.

One final comment. The function-wrapping facility is closely
tied to Valgrind's ability to replace (redirect) specified
functions, for example to redirect calls to
malloc to its
own implementation. Indeed, a replacement function can be
regarded as a wrapper function which does not call the original.
However, to make the implementation more robust, the two kinds
of interception (wrapping vs replacement) are treated differently.

--trace-redir=yes shows
specifications and bindings for both
replacement and wrapper functions. To differentiate the
two, replacement bindings are printed using
R-> whereas
wraps are printed using W->.

3.3.5. Limitations - control flow

For the most part, the function wrapping implementation is robust.
The only important caveat is: in a wrapper, get hold of
the OrigFn information using
VALGRIND_GET_ORIG_FN before calling any
other wrapped function. Once you have the
OrigFn, arbitrary
calls between, recursion between, and longjumps out of wrappers
should work correctly. There is never any interaction between wrapped
functions and merely replaced functions
(eg malloc), so you can call
malloc etc safely from within wrappers.

The above comments are true for {x86,amd64,ppc32,arm,mips32,s390}-linux.
On
ppc64-linux function wrapping is more fragile due to the (arguably
poorly designed) ppc64-linux ABI. This mandates the use of a shadow
stack which tracks entries/exits of both wrapper and replacement
functions. This gives two limitations: firstly, longjumping out of
wrappers will rapidly lead to disaster, since the shadow stack will
not get correctly cleared. Secondly, since the shadow stack has
finite size, recursion between wrapper/replacement functions is only
possible to a limited depth, beyond which Valgrind has to abort the
run. This depth is currently 16 calls.

For all platforms ({x86,amd64,ppc32,ppc64,arm,mips32,s390}-linux)
all the above
comments apply on a per-thread basis. In other words, wrapping is
thread-safe: each thread must individually observe the above
restrictions, but there is no need for any kind of inter-thread
cooperation.

3.3.6. Limitations - original function signatures

As shown in the above example, to call the original you must use a
macro of the form CALL_FN_*.
For technical reasons it is impossible
to create a single macro to deal with all argument types and numbers,
so a family of macros covering the most common cases is supplied. In
what follows, 'W' denotes a machine-word-typed value (a pointer or a
C long),
and 'v' denotes C's void type.
The currently available macros are:

CALL_FN_v_v -- call an original of type void fn ( void )
CALL_FN_W_v -- call an original of type long fn ( void )
CALL_FN_v_W -- call an original of type void fn ( long )
CALL_FN_W_W -- call an original of type long fn ( long )
CALL_FN_v_WW -- call an original of type void fn ( long, long )
CALL_FN_W_WW -- call an original of type long fn ( long, long )
CALL_FN_v_WWW -- call an original of type void fn ( long, long, long )
CALL_FN_W_WWW -- call an original of type long fn ( long, long, long )
CALL_FN_W_WWWW -- call an original of type long fn ( long, long, long, long )
CALL_FN_W_5W -- call an original of type long fn ( long, long, long, long, long )
CALL_FN_W_6W -- call an original of type long fn ( long, long, long, long, long, long )
and so on, up to
CALL_FN_W_12W

The set of supported types can be expanded as needed. It is
regrettable that this limitation exists. Function wrapping has proven
difficult to implement, with a certain apparently unavoidable level of
ickiness. After several implementation attempts, the present
arrangement appears to be the least-worst tradeoff. At least it works
reliably in the presence of dynamic linking and dynamic code
loading/unloading.

You should not attempt to wrap a function of one type signature with a
wrapper of a different type signature. Such trickery will surely lead
to crashes or strange behaviour. This is not a limitation
of the function wrapping implementation, merely a reflection of the
fact that it gives you sweeping powers to shoot yourself in the foot
if you are not careful. Imagine the instant havoc you could wreak by
writing a wrapper which matched any function name in any soname - in
effect, one which claimed to be a wrapper for all functions in the
process.

3.3.7. Examples

In the source tree,
memcheck/tests/wrap[1-8].c provide a series of
examples, ranging from very simple to quite advanced.

mpi/libmpiwrap.c is an example
of wrapping a big, complex API (the MPI-2 interface). This file defines
almost 300 different wrappers.