Summary
Today I released a new ScalaTest-1.5 snapshot release that contains several enhancements, including formatted output for TDD-style traits, indented output for nested style traits, and two new style traits, PropSpec and FreeSpec.

Advertisement

Today I released an updated ScalaTest-1.5-SNAPSHOT that includes several enhancements, including two new style traits, PropSpec and FreeSpec. The snapshot works with Scala 2.8. You can download the snapshot release via the scala-tools.org Maven repository with:

The PropSpec trait

PropSpec allows you to create a suite of property-based tests. It works the same as a FunSuite, but instead of test you write property, and instead of testsFor you write propertiesFor.

PropSpec supports both ScalaCheck and ScalaTest property styles (as well as any other property style you may wish to use it with.) If you want to write properties in the ScalaCheck style, mix Checkers into your PropSpec. If you want to write them in the ScalaTest style, mix in PropertyChecks. Here's an example that uses both generator- and table-driven property checks:

For more information, see the Scaladoc documentation for PropSpec and my earlier post, ScalaTest Property Checks Preview. Trait PropSpec is the last piece of ScalaTest's new support for property-based testing described in that previous post.

Note: Trait PropSpec is in part inspired by class org.scalacheck.Properties, designed by Rickard Nilsson for the ScalaCheck test framework.

The FreeSpec trait

The other new style trait introduced in today's snapshot release is FreeSpec. Whereas ScalaTest's other specification-style traits facilitate writing text with certain grammatical structures using words like "when," "should," and "can," FreeSpec allows you to structure the text of your specification however you wish. You write a test in a FreeSpec with a string followed by in and a block of test code, just as you do in WordSpec and FlatSpec:

"should pop values in last-in-first-out order" in {
// ...
}

You can surround tests with description clauses composed of a string, a dash character (-), and a block. Here's an example:

The FreeSpec concept first saw light of day way back in ScalaTest 0.9.4 under the name SpecDasher. I released it already deprecated with a warning that I would remove it in 0.9.5, which I did. I only released it because I'd shown it in Programming in Scala, which had already gone to the printer. I wanted all the code in the book to work for the version of ScalaTest mentioned in the book. I removed it in 0.9.5 because I wasn't convinced I had figured out the best way to do it, and even if I figured that out, I only planned to add it if users actually convinced me it would be useful to them.

Eric Torreborre's specs framework always had a >> operator that provided a similar nesting ability to FreeSpec's dash character, but in the specs case >> was an alias for in, which meant tests (called "examples" in specs) were being nested not specification text (and you still could put a should on top). The specsy project by Esko Luontola came closer to the concept, and in fact Esko wrote a blog post about the troubles with pre-defined words, Choice of Words in Testing Frameworks. Over time I did get input from users that they would find value in this kind of trait, for example, in this scalatest-users discussion started by Sukant Hajra. And I got the opposite feedback, in a way, of seeing someone use trait Spec and completely ignoring the guiding structure. So now FreeSpec is here.

Formatting and indentation

Another user feedback that I got over time is that users really liked the formatted output of the BDD-style (Behavior-Driven-Development-style) traits, and wanted to see the same thing in the TDD-style (Test-Driven-Development-style) traits. I did things that way originally because making the output of running a test suite a more useful artifact was a push from the BDD folks. But the users gave me this feedback, so now even when you run a Suite, FunSuite, JUnitSuite, JUnit3Suite, or a TestNGSuite, you get nicely formatted output. For example, given this FunSuite:

Another bit of user feedback was that people preferred to see the nested levels in the test class echoed in indentation levels in the output. ScalaTest has supported arbitrary levels of indentation since 1.0 in its event hierarchy, but didn't fire indentation of more than one level from any ScalaTest trait, not even from Spec, which allowed arbitrarily deep levels of nesting. My thought was that most often people wouldn't actually nest more deeply than two levels, and I felt that it was more readable to flatten two levels to one. Also I observated Ruby's RSpec tool, which inspired Spec's describe/it syntax, at the time flattened everything to one level. I wasn't sure which way to go, so I followed RSpec initially. Meanwhile Eric Torreborre indented specs output to match the input, and I observed that led some specs users to nest text more deeply, and even RSpec now indents its output this way. The ScalaTest users I asked about this for the most part indicated they would like the indented output. So now given this Spec:

scala> (new MySpec).execute()A Stack (when empty)- should be empty (pending)
- should complain on peek (pending)
- should complain on pop (pending)A Stack (when full)- should be full (pending)
- should complain on a push (pending)

As of the latest ScalaTest 1.5 snapshot, you will get:

scala> (new MySpec).execute()MySpec:
A Stack
(when empty)- should be empty (pending)
- should complain on peek (pending)
- should complain on pop (pending)(when full)- should be full (pending)
- should complain on a push (pending)

For two levels of indentation, it isn't necessarily an improvement in readability, but at more levels it is more clearly a readability win. For example, given this WordSpec:

import org.scalatest.WordSpec
classScalaTestGUISpecextendsWordSpec {
def theUser = afterWord("the user")
def display = afterWord("display")
def is = afterWord("is")
"The ScalaTest GUI" when theUser {
"clicks on an event report in the list box" should display {
"a blue background in the clicked-on row in the list box" in {}
"the details for the event in the details area" in {}
"a rerun button" that is {
"enabled if the clicked-on event is rerunnable" in {}
"disabled if the clicked-on event is not rerunnable" in {}
}
}
}
}

Instead of the ScalaTest 1.3 output of:

scala> (new ScalaTestGUISpec).execute()The ScalaTest GUI (when the user clicks on an event report in the list box)
- should display a blue background in the clicked-on row in the list box
- should display the details for the event in the details area
- should display a rerun button that is enabled if the clicked-on event is rerunnable
- should display a rerun button that is disabled if the clicked-on event is not rerunnable

As of the latest ScalaTest 1.5 snapshot, you'll get:

scala> (new ScalaTestGUISpec).execute()ScalaTestGUISpec:
The ScalaTest GUI
when the user clicks on an event report in the list box
should display
- a blue background in the clicked-on row in the list box
- the details for the event in the details area
a rerun button that is
- enabled if the clicked-on event is rerunnable
- disabled if the clicked-on event is not rerunnable

Give it a try

These enhancements will be released as part of ScalaTest 1.5 within the next few weeks. I'm posting this preview now because I want to get feedback in general on the API and find if there are any bugs to fix or any code-breakages. (I expect no source code to break with any of these enhancements, so let me know if you have a problem.) So please give it a try and either post feedback to the discussion for for this blog post, or email the scalatest-users mailing list.