Quickstart

This section provides a quick set of steps to try out benchmarking without
requiring you to move code into modules. Because the steps involve
disabling debugging for accurate performance results, you won't commit
the changes to your source control system, but it can still be helpful when you
want to run one-off measurements.

What to benchmark

Benchmarks are most useful for CPU work that is run many times in your app. Good
examples are RecyclerView scrolling, data conversions/processing, and pieces
of code that get used repeatedly.

Other types of code are more difficult to measure with benchmarking. Because
benchmarks run in a loop, any code that isn't run frequently, or performs
differently when called multiple times, may not be a good fit for benchmarking.

Caching

Try to avoid measuring just the cache. For example, a custom view's
layout benchmark might measure only the performance of the layout cache. To
avoid this, you can pass different layout parameters in each loop. In other
cases, such as when measuring file system performance, this may be difficult
because the OS caches the file system while in a loop.

Infrequently-run code

Code that's run once during application startup is not very likely to get JIT
compiled by Android Runtime (ART). Because of that, benchmarking this code as it
runs in a loop isn't a realistic way to measure its performance.

For this sort of code, we recommend tracing or profiling the code in
your app, instead. Note that this doesn't mean you can't benchmark code in
your startup path, but rather that you should pick code that's run in a loop,
and that is likely to get JIT compiled.

Full project setup

To set up benchmarking for regular benchmarking rather than one-off
benchmarking, you isolate benchmarks into their own module. This ensures that
their configuration, such as setting debuggable to false, is separate
from regular tests.

To do this, you need to complete these tasks:

Place code and resources you want to benchmark into a library module if they
aren't already in one.

To copy data when using AGP 3.5 or lower, you need to add a flag to your
Android manifest in the androidTest directory of your benchmarks to enable
legacy external storage behavior. See
Opt out of filtered view
for more information on opting out of scoped storage.

Clock stability

Clocks on mobile devices dynamically change from high state (for performance)
to low state (to save power, or when the device gets hot). These varying clocks
can make your benchmark numbers vary widely, so the library provides ways to
deal with this issue.

Lock clocks (requires root)

Locking clocks is the best way to get stable performance. It ensures that clocks
never get high enough to heat up the device, or low if a benchmark isn't fully
utilizing the CPU. While this is the best way to ensure stable performance, it
isn't supported on most devices, due to requiring adb root.

To lock your clocks, add the supplied helper plugin to the top level project’s
classpath in the main build.gradle file:

Apply the plugin in the build.gradle of the module you are benchmarking:

apply plugin: com.android.app
apply plugin: androidx.benchmark
...

This adds benchmarking Gradle tasks to your project, including
./gradlew lockClocks and ./gradlew unlockClocks. Use these tasks to lock and
unlock a device’s CPU using adb.

If you have multiple devices visible to adb, use the environment variable
ANDROID_SERIAL to specify which device the Gradle task should operate on:

ANDROID_SERIAL=device-id-from-adb-devices ./gradlew lockClocks

Note: Rebooting resets your device clocks.

Sustained performance mode

Window.setSustainedPerformanceMode() is a feature supported by some
devices that enables an app to opt for a lower max CPU frequency. When running
on supported devices, the Benchmark library uses a combination of this API and
launching its own activity to both prevent thermal throttling and stabilize
results.

This functionality is enabled by default by the testInstrumentationRunner set
by the Gradle plugin. If you want to use a custom runner, you can subclass the
AndroidBenchmarkRunner and use it as your testInstrumentationRunner.

The runner launches an opaque, fullscreen activity to ensure that the benchmark
runs in the foreground and without any other app drawing.

Automatic execution pausing

If neither clock-locking nor sustained performance are used, the library
performs automatic thermal throttling detection. When enabled, the internal
benchmark periodically runs to determine when the device temperature has gotten
high enough to lower CPU performance. When lowered CPU performance is detected,
the library pauses execution to let the device cool down, and retries the
current benchmark.

Configuration Errors

The library detects the following conditions to ensure your project and
environment are set up for release-accurate performance:

Debuggable is set to false.

A physical device, not an emulator, is being used.

Clocks are locked if the device is rooted.

Sufficient battery level on device.

If any of the above checks fail, the benchmark will throw an error to
discourage inaccurate measurements.

To suppress these errors as warnings, and prevent them from throwing an error
and halting the benchmark, pass the type of errors you want to suppress in a
comma-separated list to the instrumentation argument
androidx.benchmark.suppressErrors:

Note that suppressing errors allows the benchmark to run in an incorrectly
configured state, but the output of the benchmark will be intentionally
corrupted by prepending test names with the error. That is, running a
debuggable benchmark with the above suppression will prepend test names with
DEBUGGABLE_.