Smoke Testing

Semaphore

Introduction

Smoke testing is a technique responsible for discovering major flaws in software
early in your continuous delivery pipeline. It consists of very shallow, but
broad tests that verify the crucial parts of the application. This makes smoke
testing a very cost-effective operation.

In this article, we will explore the origins of smoke testing and how it can be
incorporated into your continuous integration or continuos delivery pipeline.

Origins of Smoke in Testing

The term smoke test originally comes from electrical engineering. Once the
circuit of an electrical device is ready for testing, it is attached to the
power for the first time. If there are some major defects, smoke will appear
almost instantly. It’s a clear sign to the engineers that no further tests are
needed (or even possible) before the circuit is fixed.

The same concept was adopted in software development, together with the original
term.

What and How We Should Test

Smoke tests should determine if the system is stable enough for it to make sense
to proceed to the next phase. This is achieved by testing the main
functionalities of the system at a very basic level.

For example, if we are working on an e-commerce web application, we would need
to check if a customer can purchase a product. We don't need to verify all the
details and edge cases of the purchasing process. Passed smoke tests are usually
followed by a comprehensive test suite.

Although it is common to smoke test the main user-facing features of an
application, you can use them at many different abstractions levels in your
entire system. The value of this multi-level smoke testing approach is that you
can discover defects on a more controlled and granular level. When working on
distributed systems that involve many teams, this can be a big time saver.

Here are a couple more examples of how you can apply smoke testing:

Dependency check - verifies that your application has all dependencies met
in order to boot. These could be libraries, network and database connections,
etc.

Server probe - after server software upgrades, which can break your production
environment in unforeseen ways. With a good smoke test you can easily and
cheaply save yourself from introducing broken servers into production.

Post-deploy verification - after a deploy to a server, a smoke test can
verify if the deploy has been successful, and initiate automatic rollback if
the deploy turns out to be broken.

Depending on what you are testing, actions that you will take based on the
result can vary a lot. However, it is essential that they are fast in order to
ensure a fast feedback loop.

Smoke tests should evolve as your application evolves. In the beginning,
they could check for one or two essential functions, and over time they may
need to grow to satisfy their purpose.

Tools that can serve well for writing smoke tests can be very simple. For
instance, even a simple curl api.myapp.com/is_alive is a good start. Checking
versions of installed dependencies with a simple Bash script could also be
enough.

For more complex checks, you can rely on a testing library in your favourite
programming language. If you are using Ruby and want to test some of the main
features of your web application, you can run a few Cucumber scenarios against
your production instance.

Conclusion

Smoke testing is a practice that allows you to catch errors that would be
detected anyway, but do so much faster and cheaper. You can start simple by
adding one basic test as the first step in your continuous integration pipeline.
It takes little and can save a lot of time.

P.S. Would you like to learn how to build sustainable Rails apps and
ship more often? We've recently published an ebook
covering just that — "Rails Testing Handbook".
Learn more and download a free copy.

Darko Fabijan

Rendered Text co-founder. Enjoys breaking new ground and exploring tools and ideas that make lives of developers interesting and productive.

Set up continuous integration and delivery for your project in a minute.