We are using Jest as our test runner for our JavaScript (React and Selenium) tests. We have one or two flaky tests (as TeamCity marks them flaky and we process them as technical-debt), although we will fix them, for the continuously-delivery process it would be very handy if we can retry the tests automatically once or twice.

Currently Jest does not support re-running of tests out-of-the-box.

I have been thinking about parsing the output for failed tests and then passing them into a second run with the -t parameter.

1 Answer
1

We recently switched from mocha to jest and in the process came across a flake which appears to due to a bug in react. Given the difficulty we had in investigating this issue and that it should be fixed in react 16, we too decided to take the approach of simply retrying the failing tests (the flake rate was ~3% and only when running multiple workers, so the likelihood of it appearing a second time when running a single test should be zero).

Since there was no integrated option in jest to retry on fail, we had to work with what we had. We use jest-cli in gulp, which provides a result object passed to a callback upon completion of the test suite:

jest.runCLI(config, __dirname, (result) => { })

result contains a testResults object, which is an array of objects containing details on each test file including number of failing tests (within the file), error messages, and file path.

The flake we saw (debugID may not be empty) had a consistent error message which never appeared for any other test failure, so we could be confident that a flake had occurred. This knowledge, combined with the full path of the file in which the flake occurred, gave us enough information to retry flaky tests: