Author: gosukenator

This article introduces an application of Walter, a deployment pipeline, to software tests.
Specifically, how Walter is applied to the integration tests of my personal products, Serverspec and Specinfra are covered.

TL;DR

Application of Walter to the integration tests of Serverspec and Specinfra are described.

Combination of Walter and Wercker is nice. Especially parallel executions by Walter in Wercker is useful.

We can run the tests in my laptop without pushing the changes to the GitHub repository. It is handy 🙂
When the processes are different between local and Wercker, we can use the only_if feature which allows us to change the behaviors by the environments.

Of course, we can execute the tests with Wercker v2 locally. Unfortunately Wercker v2 is a bit hassle, since it needs Docker nevertheless the integration tests of Serverspec and Specinfra do not need Docker.. Therefore I feel Walter is handy since Walter does not need any VM systems.

Paralell Execution

The tests are run in each VM which is created for the OS on DigitalOcean. The tests are executed in parallel with the following Walter configuration.

In the above settings, each Walter stage starts VMs (CoreOS, CentOS 6.5, CentOS 7.0, Ubuntu 14.04 and FreeBSD 10.1) in DigitalOcean in parallell,
and then executes the provisioning by Itamae and run tests with Serverspec.

When the test run in sequence, the tests are not finished within the limit (25 minitutes). By contrast, Walter with the above settings finish the all tests within the limit.

We can see the Wercker log here. The following is the image of the log. As wee see, it is a bit difficult to understand the results.
Specifically when some tests are failed in a environment (OS), understanding that the failed tests were run in which OS is not intuitive.

I overcome the problem by the Slack notification feature provided by Walter.

Slack Notification

Walter provide messenger notification. The results of the stages are sent to a Slack or HipChat channel.

When tests of a stage failed, the stage is shown with red as follows

In the above case, we can see the tests of CoreOS are failed.
Clicking the bottom link and move the the build page, we can see the failure is from CoreOS tests.

Future

Currently I am satisfied with the combination of Wercker and Walter. As a next step I will use Walter with Jenkins or other CI systems and add the features to Walter if needed. In addition, I have a plan to provide the Walter server with which users can build CI system without Jenkins or other CI services.