Test Requirements

Since bug 915156 landed on December 6th 2013, make test-perf requires Node.js on the host to run the tests. The relevant modules should be installed automatically with npm.

Prior to running the test, you need to configure a runner host. The runner host is a module that will either run the test in B2G desktop or on a device (real or virtual — like an emulator). By default it runs in B2G desktop, which is not very relevant for performance. To configure the runner just edit the file local.mk in the Gaia top level directory (create it if it doesn't already exist) and put the following line:

MARIONETTE_RUNNER_HOST=marionette-device-host

This will use the device host runner. The default value is marionette-b2gdesktop-host.

Output

Note: This is available in Gaia 1.4, but you should be able to run it against Gaia 1.3 on a real device.

The test output the data in JSON format. By default it is output to stdout and might be mixed with error message from other commands like npm. This is not a very good idea for automation. So you can redirect this JSON output to a file. Just define MOZPERFOUT for the host runner, either on the command line as an option or in the local.mk file as shown above.

MOZPERFOUT=myfile.json

Running all tests

You must make sure that you run the performance test using the same version of Gaia that has been installed on the target device. There is no guarantee that it will work otherwise as some subtle changes my have been make.

make test-perf

Running tests for a specific app

APP=browser make test-perf

Running tests for a set of apps

APPS="browser communications/contacts" make test-perf

Setting the number of runs

By default, each test is run five times. You can change that by setting the value of RUNS before running the tests. For example, to run each test three times you'd use this option:

RUNS=3 make test-perf

Writing new tests

With the details of running the test suite out the way, let's now look at how you can write your own performance tests for Gaia.

Startup tests

If you only want to record events that happen during the startup of an app, there's a generic test written to help you do so. To record the events, all you have to do is dispatch them.