Write a Sharded IRemoteTest Test Runner

When writing a test runner, it's important to think about scalability. Ask
yourself, "if my test runner had to run 200K test cases" how long would it take?

Sharding is one of the answers available in Trade Federation. It requires
splitting all the tests the runner needs into several chunks that can be
parallelized.

This page describes how to make your runner shardable for Tradefed.

Interface to implement

The single most important interface to implement to be considered shardable by
TF is
IShardableTest,
which contains two methods: split(int numShard) and split().

If your sharding is going to depend on the number of shards requested, you
should implement split(int numShard). Otherwise, implement split().

When a TF test command is executed with sharding parameters --shard-count and
--shard-index, TF iterates through all IRemoteTest to look for ones
implementing IShardableTest. If found, it will call split to
get a new IRemoteTest object to run a subset of test cases for a specific
shard.

What should I know about the split implementation?

You runner may shard upon some conditions only; in that case return null
when you did not shard.

Try to split as much as it makes sense: split your runner into unit of
execution that makes sense for it. It really depends of your runner. For
example:
HostTest
is sharded at the Class level, each test class is put in a separate shard.

If it makes sense, add some options to control the sharding a little bit.
For example:
AndroidJUnitTest
has a ajur-max-shard to specify the maximum number of shards it could
split in, regardless of the number requested.

Detailed example implementation

Here is an example code snippet implementing IShardableTest you can
reference. The full code is avaiable at
(https://android.googlesource.com/platform/tools/tradefederation/+/refs/heads/master/test_framework/com/android/tradefed/testtype/InstalledInstrumentationsTest.java)

This example simply creates a new instance of itself and sets shard
parameters to it. However, the splitting logic can be totally different from
test to test; and as long as it is deterministic and yields collectively
exhaustive subsets, it is okay.

Independence

Shards need to be independent! Two shards created by your implementation of
split in your runner should not have dependencies on each other or share
resources.

Shards splitting needs to be deterministic! This is also mandatory, given the
same conditions, your split method should always return the exact same list of
shards in the same order.

NOTE: Since each shard can run on different TF instances, it is critical to
ensure the split logic yields subsets that are mutually exclusive and
collectively exhaustive in a deterministic manner.

How to shard a test locally

To shard a test on a local TF, you can simply add the --shard-count option to
the command line.

tf >run host --class com.android.tradefed.UnitTests --shard-count 3

Then TF will automatically spawn commands for each shard and run them.