Optimizing Performance

When writing rules, the most common performance pitfall is to traverse or copy
data that is accumulated from dependencies. When aggregated over the whole
build, these operations can easily take O(N^2) time or space. To avoid this, it
is crucial to understand how to use depsets effectively.

This can be hard to get right, so Bazel also provides a memory profiler that
assists you in finding spots where you might have made a mistake. Be warned:
The cost of writing an inefficient rule may not be evident until it is in
widespread use.

Note that each item is only mentioned once. With lists you would get this:

a = ['a']
b = ['b', 'a']
c = ['c', 'b', 'a']
d = ['d', 'b', 'a']

Note that in this case 'a' is mentioned four times! With larger graphs this
problem will only get worse.

Here is an example of a rule implementation that uses depsets correctly to
publish transitive information. Note that it is OK to publish rule-local
information using lists if you want since this is not O(N^2).

Avoid calling depset.to_list()

You can coerce a depset to a flat list using
to_list(), but doing so usually results in O(N^2)
cost. If at all possible, avoid any flattening of depsets except for debugging
purposes.

A common misconception is that you can freely flatten depsets if you only do it
at top-level targets, such as an <xx>_binary rule, since then the cost is not
accumulated over each level of the build graph. But this is still O(N^2) when
you build a set of targets with overlapping dependencies. This happens when
building your tests //foo/tests/..., or when importing an IDE project.

Note: Today it is possible to flatten depsets implicitly by iterating over
the depset the way you would a list, tuple, or dictionary, or by taking the
depset’s size via len(). This functionality is deprecated.
and will be removed.

Avoid calling len(depset)

It is O(N) to get the number of items in a depset. It is however
O(1) to check if a depset is empty. This includes checking the truthiness
of a depset:

Use ctx.actions.args() for command lines

When building command lines you should use ctx.actions.args().
This defers expansion of any depsets to the execution phase.

Apart from being strictly faster, this will reduce the memory consumption of
your rules – sometimes by 90% or more.

Here are some tricks:

Pass depsets and lists directly as arguments, instead of flattening them
yourself. They will get expanded by ctx.actions.args() for you.
If you need any transformations on the depset contents, look at
ctx.actions.args#add to see if anything fits the bill.

Are you passing File#path as arguments? No need. Any
File is automatically turned into its
path, deferred to expansion time.

Avoid constructing strings by concatenating them together.
The best string argument is a constant as its memory will be shared between
all instances of your rule.

If the args are too long for the command line an ctx.actions.args() object
can be conditionally or unconditionally written to a param file using
ctx.actions.args#use_param_file. This is
done behind the scenes when the action is executed. If you need to explicitly
control the params file you can write it manually using
ctx.actions.write.

Performance profiling

To profile your code and analyze the performance you have two options:

use the --profile flag and the analyze-profile command, or

use the new --experimental_generate_json_trace_profile flag and load the
resulting JSON profile in chrome://tracing (recommended).

–profile and analyze-profile

This profiling method consists of two steps, first you have to execute your
build/test with the --profile flag, for example

$ bazel build --nobuild --profile=/tmp/prof //path/to:target

The file generated (in this case /tmp/prof) is a binary file, which can be
postprocessed and analyzed by the analyze-profile command:

$ bazel analyze-profile /tmp/prof

By default, it prints summary analysis information for the specified profile
datafile. This includes cummaltive statistics for different task types for each
build phase and an analysis of the critical path.

The first section of the default output is an overview of the time spent
on the different build phases:

You can use the following options to display more detailed information:

--dump=text: Print all recorded tasks in the order they occurred.

--dump=raw: Use this for automated analysis with scripts.

--html: Writes a file called <profile-file>.html in the directory of the
profile file. Open it in you browser to see a Gantt type chart that displays
time on the horizontal axis and threads of execution along the vertical axis.

The resulting profile file
(/home/johndoe/.cache/bazel/_bazel_twerth/f01bc937da326f5bb0feb15c854c110c/command.profile
in this case; can be configured by the --profile=<path> flag) can then be
loaded and viewed in Chrome. For this, open chrome://tracing in a new tab,
click Load and pick the profile file.

Example profile:

The top row (‘cpu counters’) shows the local CPU usage, which is high in this
build during the analysis phase and then gets lower during execution.
The second row (‘Critical Path’) refers to the critical path of the build, that
is even with infinite parallelism the build would not be faster than the actions
in this path.
The third row (‘grpc-command-1’) displays everything that’s happening on
Bazel’s main thread, giving a high level overview of what Bazel is doing.
The remaining rows show what the worker threads are doing.

You can interact with the profile, for example zoom in, inspect particular tasks,
filter for task descriptions and select multiple tasks to get an overview. Press
? to get an overview of what you can do.

When analyzing these kind of profiles look for the following:

Slow analysis phase (RunAnalysisPhase), especially on incremental builds. This
can be a sign of a poor rule implementation, for example one that flattens
nested sets.

Individual slow actions, especially those on the critical path. It might be
possible to split large actions into multiple smaller actions or reduce the
dependencies to speed them up.

Bottlenecks, that is a small number of threads is busy while all others are
idleing waiting for the result. Optimizing this will most likely require
touching the rule implementations to introduce more parallelism.

Note that we filter out fast tasks and certain task types completely to keep the
profile files small enough to render fast in the Chrome Trace Viewer.

Memory Profiling

Bazel comes with a built-in memory profiler that can help you check your rule’s
memory use. If there is a problem you can dump the heap to find the
exact line of code that is causing the problem.