Metadata Records: Metadata records serve many purposes. Mostly, they
capture information that may be too costly to record for each function, but
that is required to contextualize the fine-grained timings. They also are used
as markers for user-defined Event Data payloads. Metadata records have 16
bytes each.

Event Data: Free form data may be associated with events that are traced
by the binary and encode data defined by a handler function. Event data is
always preceded with a marker record which indicates how large it is.

Function Arguments: The arguments to some functions are included in the
trace. These are either pointer addresses or primitives that are read and
logged independently of their types in a high level language. To the tracer,
they are all numbers. Function Records that have attached arguments will
indicate their presence on the function entry record. We only support logging
contiguous function argument sequences starting with argument zero, which will
be the "this" pointer for member function invocations. For example, we don't
support logging the first and third argument.

A reader of the memory format must maintain a state machine. The format makes no
attempt to pad for alignment, and it is not seekable.

Function Records have an 8 byte layout. This layout encodes information to
reconstruct a call stack of instrumented function and their durations.

Field

Size (bits)

Description

discriminant

1

Indicates whether a reader should read a
Function or Metadata record. Set to 0 for
Function records.

action

3

Specifies whether the function is being
entered, exited, or is a non-standard entry
or exit produced by optimizations.

function_id

28

A numeric ID for the function. Resolved to a
name via the xray instrumentation map. The
instrumentation map is built by xray at
compile time into an object file and pairs
the function ids to addresses. It is used for
patching and as a lookup into the binary's
symbols to obtain names.

tsc_delta

32

The number of ticks of the timestamp counter
since a previous record recorded a delta or
other TSC resetting event.

On little-endian machines, the bitfields are ordered from least significant bit
bit to most significant bit. A reader can read an 8 bit value and apply the mask
0x01 for the discriminant. Similarly, they can read 32 bits and unsigned
shift right by 0x04 to obtain the function_id field.

On big-endian machine, the bitfields are written in order from most significant
bit to least significant bit. A reader would read an 8 bit value and unsigned
shift right by 7 bits for the discriminant. The function_id field could be
obtained by reading a 32 bit value and applying the mask 0x0FFFFFFF.

Function action types are as follows.

Type

Number

Description

Entry

0

Typical function entry.

Exit

1

Typical function exit.

Tail_Exit

2

An exit from a function due to tail call
optimization.

Entry_Args

3

A function entry that records arguments.

Entry_Args records do not contain the arguments themselves. Instead, metadata
records for each of the logged args follow the function record in the stream.

Interspersed throughout the buffer are 16 byte Metadata records. For typically
instrumented binaries, they will be sparser than Function records, and they
provide a fuller picture of the binary execution state.

Metadata record layout is partially record dependent, but they share a common
structure.

The same bit field rules described for function records apply to the first byte
of MetadataRecords. Within this byte, little endian machines use lsb to msb
ordering and big endian machines use msb to lsb ordering.

Field

Size

Description

discriminant

1 bit

Indicates whether a reader should read a
Function or Metadata record. Set to 1 for
Metadata records.

Since each function record uses a 32 bit value to represent the number of ticks
of the timestamp counter since the last reference, it is possible for this value
to overflow, particularly for sparsely instrumented binaries.

When this delta would not fit into a 32 bit representation, a reference absolute
timestamp counter record is written in the form of a TSCWrap record.

An EndOfBuffer record type indicates that there is no more trace data in this
buffer. The reader is expected to seek past the remaining buffer_size expressed
before the start of buffer and look for either another header or EOF.

Not all sequences of Metadata records and Function records are valid data. A
sequence should be parsed as a state machine. The expectations for a valid
format can be expressed as a context free grammar.

This is an attempt to explain the format with statements in EBNF format.

There are a few clarifications that may help understand what is expected of
Function records.

Functions with an Exit are expected to have a corresponding Entry or
Entry_Args function record precede them in the trace.

Tail_Exit Function records record the Function ID of the function whose return
address the program counter will take. In other words, the final function that
would be popped off of the call stack if tail call optimization was not used.

Not all functions marked for instrumentation are necessarily in the trace. The
tracer uses heuristics to preserve the trace for non-trivial functions.

Not every entry must have a traced Exit or Tail Exit. The buffer may run out
of space or the program may request for the tracer to finalize toreturn the
buffer before an instrumented function exits.