A simple unit testing framework for testing buildpacks based on shUnit2.

Full Description

Buildpack Testrunner

A simple unit testing framework for testing buildpacks based onshUnit2. It provides utilities forloading buildpacks and capturing and asserting their behavior. It can be runlocally, as part of a continuous integration system, or even directly on Herokuas a buildpack itself.

Requirements

This buildpack requires that the following directories/files exist:

bin/detect
bin/release
bin/compile
test/

Running Buildpack Tests on Heroku

The testrunner is itself a buildpack and can be used to run tests for yourbuildpack on Heroku. This can be very helpful for testing your buildpack on areal Heroku dyno before pushing it to a public repo. To do this, create aCedar app out of your buildpack and set the testrunner as its buildpack:

If you would like caching to be enabled run tests-with-caching instead.Note, the cache will only live for the life of the dyno (i.e. one test run).

heroku run tests-with-caching

When running tests on a dyno, the exit code is not returned correctly to thelocal shell. To workaround this limitation, pipe the output to the reportscript, which will parse the output and return the correct exit code. Forexample:

heroku run tests | bin/report

Local Setup

To use the testrunner locally, first clone this repository:

git clone https://github.com/heroku/heroku-buildpack-testrunner

If you do not already have shUnit2 installed, eitherdownload it or check itout from Github:

git clone https://github.com/kward/shunit2

Do not use apt-get for obtaining shUnit2 because it is the wrong version.

Once you have shUnit2, set an SHUNIT_HOME environment variable to the rootof the version you wish to use. For example:

export SHUNIT_HOME=/usr/local/bin/shunit/source/2.1

Local Usage

where buildpack_n can either be a local directory or a remote Git repositoryending in .git. Each buildpack must have a test directory and filesmatching the *_test.sh pattern to be run. The -s flag sets a single testsuite to run in the test directories of the buildpacks. The -c flagenables persistent caching of files downloaded with cUrl. Seelib/magic_curl/README.md for more info.

Would first run the tests in the buildpack at ~/a_local_buildpack and thenclone the Git repository at git@github.com:rbrainard/heroku-buildpack-
gradle.git into a temp directory and run the tests there too.

Docker Usage

The testrunner can be packaged into a Docker image that can be run byindividual Buildpacks. To create the image, run ./build.sh from theroot directory of the project.

Writing Unit Tests for a Buildpack

Writingtests for a buildpack is similar to any other xUnit framework, but the stepsbelow summarize what you need to get started testing a buildpack. In addition,to the steps below, its advised to familarize yourself with the shUnit2 documentationbefore starting.

Create a test directory in the root of the buildpack.

Create test scripts in the test directory ending in _test.sh. They can be grouped any way you like, but creating a test script for each buildpack script is recommended. For example the detect script should have a corresponding detect_test.sh test script.

It is recommended (but not required) to source in the test_utils.sh script at the beginning of your test script.This contains common functions for setup, teardown, and asserting buildpack behavior.

. ${BUILDPACK_TEST_RUNNER_HOME}/lib/test_utils.sh

Each test case in the script should be contained a function starting with test.Like testing with other xUnit frameworks, the test cases should be fairly granularand try not to depend on outside factors or upon each other.

If you are using test_util.sh, at the beginning of each test case, you will be provided empty ${BUILD_DIR} and ${CACHE_DIR}directories for use with buildpack scripts. These directories are deleted after each test case completes. You will also be provided a${BUILDPACK_HOME} value to deterministically find the root of your buildpack.

When running buildpack scripts, it is recommended to use the detect,compile, and release functions from test_utils.sh, which will providethe correct parameters and capture the stdout, stderr, and return values ofthe scripts. If you need to manually capture a command, the capture functionis also available to you by just calling capture before your command, butuse the pre-defined functions whenever possible. Either way you capture, youwill then have access to the ${STD_OUT} file, ${STD_ERR} file, and${RETURN} value after the capture completes. To inspect these files andvalues, there are a few helpful assertions:

assertCapturedSuccess: captured command exited with 0 and stderr is empty

Metatesting

The tests for the testrunner itself work just like any other buildpack. To test the testrunner itself, just run:

bin/run .

This can be helpful to make sure all the testrunner libraries work on your platform before testing any real buildpacks.

One caveat about negative tests for assertions is that they need to be captured and wrapped in paraenthesis to supressthe assertion failure from causing the metatest to fail. For example, if you want to test that assertContains prints outthe proper failure message, capture, wrap, and then assert on the captured output.