As I have previously written, I have recently been spending some time experimenting with various aspects of Scala, including some of the frameworks which have become available. One of the frameworks I have had the privilege of using is the somewhat unassumingly-titled Specs, and implementation of the behavior-driven development methodology in Scala.

Specs takes full advantage of Scala’s flexible syntax, offering a very natural format for structuring tests. For example, we could write a simple specification for a hypothetical add method in the following way:

We could go on, of course, but you get the picture. This code will lead to the execution of four separate assertions in three tests (to put things into JUnit terminology). Fundamentally, this isn’t too much different than a standard series of unit tests, just with a slightly nicer syntax.

Specs defines a domain-specific language for structuring test assertions in a simple and intuitive way. However, this is hardly the only framework for BDD. Perhaps the most well-known such framework is RSpec, which answers a similar use-case in the Ruby programming language. Our previous specification could be rewritten using RSpec as follows:

The end-result is basically the same: the add method will be tested against the given assertions (all four of them) and the results printed in some sort of report form. In this area, RSpec is significantly more mature than Specs, generating very slick HTML reports and nicely formatted console output. This isn’t really a fundamental weakness of the Specs framework however, just indicative of the fact that RSpec has been around for a lot longer.

These two frameworks are interesting of course, but they are merely implementations of a much larger concept: behavior-driven development. I’ve never been much of a fan of unit testing. It’s always seemed to be incredibly dull and a very nearly fruitless waste of effort. As much as I hate it though, I have to bow to the benefits of a self-contained test suite; and so I press on, cursing JUnit every step of the way.

BDD provides a nice alternative to unit testing. At its core, it is not much different in that test groupings and primitive assertions are used to check all aspects of a test unit against predefined data. However, there is something about the “flow” of a behavioral spec that is considerably easier to deal with. For some reason, it is far less painful to devise a comprehensive test suite using BDD principles than conventional unit testing. It seems a little far-fetched, but BDD actually makes it easier to write (and more importantly, formulate) exactly the same tests.

It’s an odd phenomenon, one which can only be caused by the storyboard flow of the code itself. It is very natural to think of distinct requirements for a test unit when each of these requirements are being labeled and entered in a logical sequence. Moreover, the syntax of both Specs and RSpec is such that there is very little boiler-plate required to setup an additional test. Compare the previous BDD specs with the following JUnit4 example:

JUnit just requires that much more syntax. It breaks up the logical flow of the tests and (more importantly) the developer train of thought. What can be worse is this syntax bloat makes it very tempting to just group all of the assertions into a single test – to save typing if nothing else. This is problematic because one assertion may shadow all the others in the case of a failure, preventing them from ever being executed. This can make certain problems much more difficult to isolate.

Logical flow is extremely important to test structure. BDD frameworks provide a very nice syntax for painlessly defining comprehensive test suites. The really wonderful thing about all of this is that BDD is available on the JVM, right now. There’s nothing stopping you from writing your code in Java as you normally would, then creating your test suite in Scala using Specs rather than JUnit. Alternatively, you could use RSpec on top of JRuby, or Gspec with Groovy. All of these are seamless replacements for a test framework like JUnit, and requiring of far less syntactic overhead.

The growing move toward polyglot programming encourages the use of a separate language when it is best suited to a particular task. In this case, several languages are available which offer far more powerful test frameworks than those which can be found in Java. Why not take advantage of them?

As an aside, it’s worth mentioning that the should/in syntax is one of the things that I really liked about Specs as opposed to RSpec and similar. I’m sure there’s a way to replicate the syntax somehow with rspec, but it just falls out naturally with Specs. Very nice job!

I’m a bit skeptical. Is it that much of an improvement to group your tests this way?

Eventually, you want to be pointed to a Java method so you can inspect it and debug it, so the information ’should handle mixed signs’ seems to be one extra layer of noise over “shouldHandleMixedSigns()”…

Yes, I’ve looked Reductio, but only briefly. It’s about on the same level as ScalaCheck: a testing solution which is quite obviously superior in coverage and minimalism, but also way beyond my limited intellect. I’m planning on really digging into combinitorial testing one of these days, but at the moment it’s easier for me to stick with the conventional canned-data methodology.

@Cedric

To be honest, I was quite skeptical too when I first started playing with Specs. It’s a very difficult phenomenon to describe, one which almost requires you to experience it for yourself. Obviously the reduced syntax cruft is one benefit of DSLs like Specs, but the overall flow is really the big win. Somehow – and I’m honestly not sure how – BDD specifications are psychologically easier to formulate, easier to code and (ultimately) easier to verify correct.

Oh, and for the record: I have looked at TestNG, and I do find it to be a nicer framework than JUnit. However, it’s still a straight-up unit test framework. TestNG is to JUnit what SVN is to CVS: vastly improved, but still fundamentally the same.

While I agree completely with your example on here, I think that the Java example is perhaps a little harsh; in JUnit 4 you now have the Hamcrest matchers which do a surprisingly good job (not nearly as good as Specs or RSpec, of course) considering that Java is the language.

It at least reads a little more sensibly, and you can specify more complex Matchers which can be used like “is”. I love the stuff that you can do with the simple-but-powerful syntax within Scala; Specs is just one example but there are many more.

Hamcrest doesn’t really solve everything. It is a more “fluid” syntax, but it’s also significantly more verbose. It’s not too bad, but it really feels like strapping a saddle onto a dinosaur: you just can’t “fix” JUnit. It’s a nice tool because it’s so ubiquitous, but you can’t say much more for it.

Post a Comment

Comments are automatically formatted. Markup are either
stripped or will cause large blocks of text to be eaten, depending
on the phase of the moon. Code snippets should be wrapped in
<pre>...</pre> tags. Indentation within
pre tags will be preserved, and most instances of
"<" and ">" will work without a problem.

Daniel Spiewak is a software developer based out of Wisconsin, USA.
Over the years, he has worked with Java, Scala, Ruby, C/C++, ML,
Clojure and several experimental languages. He currently spends most
of his free time researching parser theory and methodologies,
particularly areas where the field intersects with functional
language design, domain-specific languages and type theory. He can
be reached by email.

If you're feeling particularly masochistic, you can follow
Daniel on Twitter @djspiewak.