Emacs Lisp: Adding Tests: ert-runner and overseer

In the previous Adding Tests post I found that running M-x ert on its own did not always pick up added or removed tests, but that a test running tool called ert-runner might fix this. It does, and adding another tool called overseer makes it easy to run tests or subsets of tests without leaving Emacs.

Here’s how I got there:

Install Cask to manage project dependencies, as ert-runner uses it to run the tests from the command-line. I found that the default brew install cask didn’t set up ~/.cask/cask.el properly, but the longer curl -fsSL https://raw.githubusercontent.com/cask/cask/master/go | python did.

With ert-runner installed by Cask, from the command line, run cask exec ert-runner init. This creates a test directory and an empty test/test-helper.el file. Previously, our tests for the begin/end functionality were in the same file as the code: for ert-runner to be able to find them, they need to be in the test directory, in a file ending -test.el. So we move the tests from mods-org.el to test/mods-org-test.el.

The newbie / coming from Ruby mistake that I made at this point was to go to the command line and run cask exec ert-runner, expecting it to pick up the files automatically. This of course fails. We need to put a (provide 'mods-org) at the bottom of the mods-org.el file, and a (load-file "mods-org.el") at the top of test/mods-org-test.el.

Having done that, we can run the tests, and they all pass. Further, if some of them fail, we can do the usual thing: comment out all but one of them and re-run and, unlike with M-x ert, it picks up the changes and just runs a single test.

When going out to the command line or popping open a shell to run cask exec ert-runner every time proves irksome, we can take advantage of the fact that we also added overseer, which gives us some handy shortcuts. With a single test file so far, I have been using C-c , b to run all the tests from the buffer.

Thanks to Sacha Chua for unblocking me at one point, and sharing a draft post which takes things further into continuous integration and test coverage tools.