Introducing Semaphore Insights: Find Test Files That Slow You Down

Today we’re happy to announce Semaphore Insights, a new part of Semaphore
with the goal to give you useful, actionable feedback on your continuous
integration process.

As developers we all want our builds to finish fast, but sometimes we introduce
bottlenecks to the test suite without realizing it. This can be by writing
tests that make excessive database calls or similar inefficient use of
resources. Just like we understand the need for refactoring the application
code, we need to find time to refactor and improve our test code too.

So the first available feature in Semaphore Insights is a report on your
slowest test files. It displays all test files which take more than a minute
to run, and if none such are found then the slowest five by default.

We have found that a minute is a sensible threshold in large projects — more
than that is usually a smell and we recommend looking into the code or
considering to split the test file. The benefits are both a faster feedback
loop your development workflow and parallel execution in CI. For example,
if you have a test file that takes 4 minutes to complete, then your whole build
cannot complete sooner than 4 minutes, plus some setup and teardown operations
that may exist for your project.

How to enable Semaphore Insights

On your project page, click on "Find slowest test files" link and then enable
the feature for your test framework. That’s it — the report will be ready
after your next successful build. To see the chart of historical performance,
just click on a test file to uncover it.

In order to gather the necessary information, Semaphore inserts some additional
formatting options to the corresponding test framework’s configuration file,
such as .rspec. That should not affect your build in any way, but if you
think it does, report it through a support message and we’ll investigate.

We will be rolling out this feature to all users over the next couple of days.

Semaphore Insights currently supports RSpec 3 and Cucumber. If your testing
framework is not supported but you would like it to be, we want to hear from
you.

We hope that you will find Semaphore Insights useful. If you have any feedback
on it, or even have ideas where it could go next,
please get in touch.