Running Tests

Marionette-based tests can be run in-tree with a locally built Firefox through mach marionette-test, as well as with a downloaded Firefox binary against in- and out-of-tree tests with runtests.py. The mach command calls runtests.py on the binary built from your mozilla-central checkout. All the same options should be available in both cases. (Aside: Are you looking to run unit tests for the Marionette Test Runner? That's described over in Developer setup under "Marionette Harness Unit Tests")

Running in-tree tests, mach will automatically manage the Python virtual environment in which your tests are run. The Marionette client that is picked up is the one that is in-tree at testing/marionette/client.

If you want to run tests from a downloaded tests.zip file, you will first need to set up the Python Marionette client.Second, you will need to download the tests from the tests.zip package found attached to Treeherder jobs for your system. If you are using a tests.zip package, extract it, and find the tests under marionette/tests/testing/marionette/client/marionette/tests.

You can launch the tests using one of the methods below. If desired, you can replace the path to unit-tests.ini with the path to a particular test. See the Marionette Python TestRunner for more options.

For Desktop Firefox

With a local build

Assuming you've run ./mach build successfully, you can use mach (German for “do”) to run all the tests:

For Firefox for Android (Fennec)

With a local build

Assuming you've run ./mach build, ./mach package and ./mach install successfully (see Simple Firefox for Android build; you may need to add 'ac_add_options --enable-marionette' to your mozconfig), you can use mach (German for “do”) to run all the tests. As a result of setting up a Marionette-enabled Fennec build, you will have Android SDK set up, likely somewhere under ~/.mozbuild -- this will be useful shortly.

The Marionette Test Runner can take care of launching an Android emulator and a Marionette-enabled Fennec for you, but it needs to know where to find an emulator and an AVD, which AVD to use, etc. You can use mach commands to set up these pieces.

# This command will offer to download and set up an AVD, and use it to launch an emulator
# The AVD is likely set up at ~/.mozbuild/android-device/avd
./mach android-emulator
# Install Fennec on the emulator
./mach install
# Run Fennec at least once to set up a profile
./mach run

If you set an AVD_HOME environment variable and if emulator command is in your PATH, you may omit the --avd-home and --emulator-binary arguments. See ./mach marionette-test -h for additional options that allow to you specify x86 vs arm and so on.

To connect to an already-running Fennec instance either in an emulator or on a device, you will need to enable Marionette manually. You can do this by building with ENABLE_MARIONETTE=1 and adding a boolean preference named marionette.defaultPrefs.enabled with a value of true in your profile. Once this preference has been set you will need to restart Fennec for it to take effect.