Fundamentals of Testing

Users interact with your app on a variety of levels, from pressing a button to
downloading information onto their device. Accordingly, you should test a
variety of use cases and interactions as you iteratively develop your app.

Organize your code for testing

As your app expands, you might find it necessary to fetch data from a server,
interact with the device's sensors, access local storage, or render complex user
interfaces. The versatility of your app demands a comprehensive testing
strategy.

Create and test code iteratively

When developing a feature iteratively, you start by either writing a new test or
by adding cases and assertions to an existing unit test. The test fails at first
because the feature isn't implemented yet.

It's important to consider the units of responsibility that emerge as you design
the new feature. For each unit, you write a corresponding unit test. Your unit
tests should nearly exhaust all possible interactions with the unit, including
standard interactions, invalid inputs, and cases where resources aren't
available. Take advantage of Jetpack libraries whenever possible;
when you use these well-tested libraries, you can focus on validating behavior
that's specific to your app.

Figure 1. The two cycles associated with iterative,
test-driven development

The full workflow, as shown in Figure 1, contains a series of nested, iterative
cycles where a long, slow, UI-driven cycle tests the integration of code units.
You test the units themselves using shorter, faster development cycles. This set
of cycles continues until your app satisfies every use case.

View your app as a series of modules

To make your code easier to test, develop your code in terms of modules, where
each module represents a specific task that users complete within your app. This
perspective contrasts the stack-based view of an app that typically contains
layers representing the UI, business logic, and data.

For example, a "task list" app might have modules for creating tasks, viewing
statistics about completed tasks, and taking photographs to associate with a
particular task. Such a modular architecture also helps you keep unrelated
classes decoupled and provides a natural structure for assigning ownership
within your development team.

It's important to set well-defined boundaries around each module, and to create
new modules as your app grows in scale and complexity. Each module should have
only one area of focus, and the APIs that allow for inter-module communication
should be consistent. To make it easier and quicker to test these inter-module
interactions, consider creating fake implementations of your modules. In your
tests, the real implementation of one module can call the fake implementation of
the other module.

As you create a new module, however, don't be too dogmatic
about making it full-featured right away. It's OK for a particular module to not
have one or more layers of the app's stack.

To learn more about how to define modules in your app, as well as platform
support for creating and publishing modules, see Android App
Bundles.

Configure your test environment

When setting up your environment and dependencies for creating tests in your
app, follow the best practices described in this section.

Organize test directories based on execution environment

A typical project in Android Studio contains two directories in which you place
tests. Organize your tests as follows:

The androidTest directory should contain the tests that run on real or
virtual devices. Such tests include integration tests, end-to-end tests, and
other tests where the JVM alone cannot validate your app's functionality.

The test directory should contain the tests that run on your local machine,
such as unit tests.

Consider tradeoffs of running tests on different types of devices

When running your tests on a device, you can choose among the following types:

Real devices offer the highest fidelity but also take the most time to run your
tests. Simulated devices, on the other hand, provide improved test speed at the
cost of lower fidelity. The platform's improvements in binary resources and
realistic loopers, however, allow simulated devices to produce more realistic
results.

Virtual devices offer a balance of fidelity and speed. When you use virtual
devices to test, use snapshots to minimize
setup time in between tests.

Consider whether to use test doubles

When creating tests, you have the option of creating real objects or test
doubles, such as fake objects or mock objects. Generally, using real objects in
your tests is better than using test doubles, especially when the object under
test satisfies one of the following conditions:

The object is a data object.

The object cannot function unless it communicates with the real object version
of a dependency. A good example is an event callback handler.

It's hard to replicate the object's communication with a dependency. A good
example is a SQL database handler, where an in-memory database provides more
robust tests than fakes of database results.

In particular, mocking instances of types that you don't own usually leads to
brittle tests that work only when you've understood the complexities of someone
else's implementation of that type. Use such mocks only as a last resort. It's
OK to mock your own objects, but keep in mind that mocks annotated using
@Spy
provide more fidelity than mocks that stub out all functionality within a class.

However, it's better to create fake or even mock objects if your tests try to
perform the following types of operations on a real object:

Long operations, such as processing a large file.

Non-hermetic actions, such as connecting to an arbitrary open port.

Hard-to-create configurations.

Tip: Check with the library authors to see if they
provide any officially-supported testing infrastructures, such as fakes, that
you can reliably depend on.

Write your tests

After you've configured your testing environment, it's time to write tests that
evaluate your app's functionality. This section describes how to write small,
medium, and large tests.

Levels of the Testing Pyramid

Figure 2. The Testing Pyramid, showing the three
categories of tests that you should include in your app's test
suite

The Testing Pyramid, shown in Figure 2, illustrates how your app should include
the three categories of tests: small, medium, and large:

Small tests are unit tests that validate your app's behavior
one class at a time.

Medium tests are integration tests that validate either
interactions between levels of the stack within a module, or interactions
between related modules.

As you work up the pyramid, from small tests to large tests, each test increases
in fidelity but also increases in execution time and effort to maintain and
debug. Therefore, you should write more unit tests than integration tests, and
more integration tests than end-to-end tests. Although the proportion of tests
for each category can vary based on your app's use cases, we generally recommend
the following split among the categories: 70 percent small, 20 percent medium,
and 10 percent large.

Write small tests

The small tests that you write should be highly-focused unit tests that
exhaustively validate the functionality and contracts of each class within your
app.

As you add and change methods within a particular class, create and run unit
tests against them. If these tests depend on the Android framework, use a
unified, device-agnostic API, such as the
androidx.test APIs. This consistency
allows you to run your test locally without a physical device or emulator.

If your tests rely on resources,
enable the includeAndroidResources option in your app's build.gradle
file. Your unit tests can then access compiled
versions of your resources, allowing the tests to run more quickly and
accurately.

Local unit tests

Use the AndroidX Test APIs whenever possible so that your unit tests can
run on a device or emulator. For tests that always run on a JVM-powered
development machine, you can use Robolectric.

Robolectric simulates the runtime for Android 4.1 (API level 16) or higher and
provides community-maintained fakes called shadows. This functionality allows
you to test code that depends on the framework without needing to use an
emulator or mock objects. Robolectric supports the following
aspects of the Android platform:

Component lifecycles

Event loops

All resources

Instrumented unit tests

You can run instrumented unit tests on a physical device or emulator. This form
of testing involves significantly slower execution times than local unit tests,
however, so it's best to rely on this method only when it's essential to
evaluate your app's behavior against actual device hardware.

When running instrumented tests, AndroidX Test makes use of the
following threads:

The main thread, also known as the "UI thread" or the "activity thread",
where UI interactions and activity lifecycle events occur.

The instrumentation thread, where most of your tests run. When your test
suite begins, the AndroidJUnitTest class starts this thread.

If you need a test to execute on the main thread, annotate it using
@UiThreadTest.

Write medium tests

In addition to testing each unit of your app by running small tests, you should
validate your app's behavior from the module level. To do so, write medium
tests, which are integration tests that validate the collaboration and
interaction of a group of units.

Use your app's structure and the following examples of medium tests (in order of
increasing scope) to define the best way to represent groups of units in your
app:

Interactions between a view and view model, such as testing a
Fragment
object, validating layout XML, or evaluating the data-binding logic of a
ViewModel object.

Vertical slices of your app, testing interactions on a particular screen.
Such a test verifies the interactions throughout the layers of your app's stack.

Multi-fragment tests that evaluate a specific area of your app. Unlike the
other types of medium tests mentioned in this list, this type of test usually
requires a real device because the interaction under test involves multiple
UI elements.

To carry out these tests, do the following:

Use methods from the Espresso Intents
library. To simplify the information that you're passing into these tests, use
fakes and stubbing.

Combine your use of
IntentSubject and
Truth-based assertions to verify the captured intents.

Use Espresso when running instrumented medium tests

Espresso helps keep tasks synchronized as you perform UI interactions similar to
the following on a device or on Robolectric:

To learn more about these interactions and how to use them in your app's tests,
see the Espresso guide.

Write large tests

Although it's important to test each class and module within your app in
isolation, it's just as important to validate end-to-end workflows that guide
users through multiple modules and features. These types of tests form
unavoidable bottlenecks in your code, but you can minimize this effect by
validating an app that's as close to the actual, finished product as possible.

Note: For each large, workflow-based test that you write, you should also write
medium tests that check the functionality of each module
included in the workflow. This testing structure makes it easier to determine
which step of a critical user journey exhibits unexpected behavior.

If your app is small enough, you might need only one suite of large tests to
evaluate your app's functionality as a whole. Otherwise, you should divide your
large test suites by team ownership, functional verticals, or user goals.

Typically, it's better to test your app on an emulated device or a cloud-based
service like Firebase Test Lab,
rather than on a physical device, as you can test multiple combinations of
screen sizes and hardware configurations more easily and quickly.

Synchronization support in Espresso

In addition to supporting medium-sized instrumentation tests, Espresso provides
support for synchronization when completing the following tasks in large tests:

To learn more about these interactions and how to use them in your app's tests,
see the Espresso guide.

Complete other testing tasks using AndroidX Test

This section describes how to use elements of AndroidX Test to further
refine your app's tests.

Create more readable assertions using Truth

The Guava team provides a fluent assertions library called
Truth. You can use this library as an
alternative to JUnit- or Hamcrest-based assertions when constructing the
validation step—or then step—of your tests.

Usually, you use Truth to express that a particular object has a specific
property using phrases that contain the conditions you're testing, such as the
following:

assertThat(object).hasFlags(FLAGS)

assertThat(object).doesNotHaveFlags(FLAGS)

assertThat(intent).hasData(URI)

assertThat(extras).string(string_key).equals(EXPECTED)

AndroidX Test supports several additional subjects for Android to make
Truth-based assertions even easier to construct:

The AndroidX Test API helps you carry out common tasks related to mobile
app testing, which the following sections discuss.

Write UI tests

Espresso allows you to programmatically locate and interact with UI elements in
your app in a thread-safe way. To learn more, see the
Espresso guide.

Run UI tests

The
AndroidJUnitRunner class
defines an instrumentation-based JUnit test runner that lets you run
JUnit 3- or JUnit 4-style test classes on Android devices. The test runner
facilitates loading your test package and the app under test onto a device or
emulator, running your tests, and reporting the results.

To further increase these tests' reliability, use Android Test Orchestrator,
which runs each UI test in its own
Instrumentation sandbox. This
architecture reduces shared state between tests and isolates app crashes on a
per-test basis. For more information about the benefits that Android Test
Orchestrator provides as you test your app, see the
Android Test Orchestrator
guide.

Interact with visible elements

The UI Automator API lets
you interact with visible elements on a device, regardless of the activity or
fragment that has focus.

Caution: We recommend testing your app using UI Automator only when
your app must interact with the system UI or another app to fulfill a critical
use case. Because UI Automator interacts with a particular system UI, you must
re-run and fix your UI Automator tests after each platform version upgrade and
after each new release of Google Play services.

As an alternative to using UI Automator, we recommend adding hermetic tests or
separating your large test into a suite of small and medium tests. In
particular, focus on testing one piece of inter-app communication at a time,
such as sending information to other apps and responding to intent results. The
Espresso-Intents tool can
help you write these smaller tests.

Add accessibility checks to validate general usability

Your app's interface should allow all users, including those with accessibility
needs, to interact with the device and complete tasks more easily in your app.

To help validate your app's accessibility, Android's testing library provides
several pieces of built-in functionality, which is discussed in the following
sections. To learn more about how to validate your app's usability for different
types of users, see the guide on testing your app's
accessibility.

Robolectric

Enable accessibility checks by including the
@AccessibilityChecks
annotation at the beginning of your test suite, as shown in the following code
snippet:

Java

Drive activity and fragment lifecycles

Use the ActivityScenario and FragmentScenario classes to test how your app's
activities and fragments respond to system-level interruptions and configuration
changes. To learn more, see the guides on how to test
activities and test
fragments.

Note: Given the device-specific conditions under which fragments behave, it's
best to work with FragmentScenario as part of your instrumented tests.

Manage service lifecycles

AndroidX Test includes code for managing the lifecycles of key services.
To learn how to define these rules, see the JUnit4
Rules guide.

Evaluate all variants of behavior that differ by SDK version

If your app's behavior depends on the device's SDK version, use the
@SdkSuppress annotation,
passing in values for minSdkVersion or maxSdkVersion depending on how you've
branched your app's logic: