A Cucumber Experiment

Having used the GildedRose recently as the subject of a coding dojo, I thought it would also make an interesting subject for some experimentation with Cucumber. Cucumber is a tool that allows you to use natural language to specify executable Acceptance Tests. The GildedRose exercise, originally created by Bobby Johnson, can be found on github, in both C# (the original), and Java (my copy). This code kata is a refactoring assignment, where the requirements are given together with a piece of existing code and the programmer is expected to add some functionality. If you take one look at that existing code, though, you’ll see that you really need to clean it up before adding that extra feature:

Having gone through the exercise a few times, I already had a version lying around that had the requirements implemented as a bunch of junit regression tests, and some real unit tests for my implementation, of course. A good starting point to get going with Cuke, though in many real-life situations I’ve found that those regression tests are not available…

Adding Cucumber to the maven build

To prepare for the use of Cucumber, I first had to set it up so that I had all the dependencies for cucumber, and have it run in the maven integration-test phase. The complete Maven pom.xml is also on GitHub.

Now, let’s add a scenario, let’s say a scenario for the basic situation of quality decreasing by one if the sell-in date decreases by one:

Feature: Quality changes with sell-in date Feature
In order to keep track of quality of items in stock
As a ShopKeeper
I want quality to change as the sell-in date decreases
Scenario: Decreasing Quality of a Basic Item
Given a Store Keeping Item with name "+5 Dexterity Vest"
And a sellIn date of 5
And a quality of 7
When the Item is updated
Then the sellIn is 4
And the quality is 6

There’s a few things to notice here. First of all, this is readable. This scenario can be read by people that have no programming experience, and understood well if they have some knowledge of the domain of the application. That means that scenarios like this one can be used to talk about the requirements/expected behaviour of the application. Second is that this is a Real Life example of the workings of the application. Making it a specific example, instead of a generic rule, makes the scenario easier to understand, and thus more useful as a communication device.

Glue

So what happens when we try to run the integration-test phase again? Will we magically see this scenario work? Well, no, there’s still some glue to provide, but we do get some help with that:

Just what a Java developer likes: Ruby stackstraces! Luckily, the first entry is fairly clear: TODO (Cucumber::Pending).
And indeed, now that we take another look at it, the methods we just created all have a @Pending annotation. If we remove that, the scenario ‘runs’ successfully:

[INFO] Feature: Quality changes with sell-in date Feature
[INFO] In order to keep track of quality of items in stock
[INFO] As a ShopKeeper
[INFO] I want quality to change as the sell-in date decreases
[INFO]
[INFO] Scenario: Decreasing Quality of a Basic Item # features/quality_decrease.feature:6
[INFO] Given a Store Keeping Item with name "+5 Dexterity Vest" # BasicFeature.aStoreKeepingItemWithNamePlus5DexterityVest_(String)
[INFO] And a sellIn date of 5 # BasicFeature.aSellInDateOf5()
[INFO] And a quality of 7 # BasicFeature.aQualityOf7()
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the sellIn is 4 # BasicFeature.theSellInIs4()
[INFO] And the quality is 6 # BasicFeature.theQualityIs6()
[INFO]
[INFO] 1 scenario (1 passed)
[INFO] 6 steps (6 passed)
[INFO] 0m0.269s

Adding code

Of course, it doesn’t actually test anything yet! But we can take a look at the way the text of the scenario is translated into executable code. For instance:

@Given ("^a sellIn date of 5$")
public void aSellInDateOf5() {
}

This looks straightforward. The @Given annotation gets a regular expression passed to it, which matches a particular condition. If the method actually set a sell-in value on some object, this would be a perfectly valid step in performing some test.

So let’s see where we can create such a sellable item. This other method looks promising:

For the non-initiated into the esoteric realm of regular expressions, the regexp here looks a little more scary. It really isn’t all that bad, though. The backslashes (\) are there to escape out the quotes (“). The brackets (()) are there to capture a specific region to be passed into the method: anything within those brackets is passed in as the first parameter of the method (out arg1 String parameter). And to specify what we want to capture, the [^\”]* simply means any character that is not a quote. So this captures everything within the quotes in our scenario, which happens to be the name of the item.

The test still passes, so this seems to work! For the rest of the scenarios, see the checked-in feature file on GitHub. Take a look at that file, and compare it to the README. Which one is clearer to you? Do they both contain all the information you need?

The odd one out

Note that there is one scenario there that is not covered by the methods that we have, so once you try to run this particular scenario, things fall apart:

Scenario: Quality and SellIn of a Sulfuras item does not change
Given a Store Keeping Item with name "Sulfuras, Hand of Ragnaros" with a sellIn date of 5, a quality of 7
When the Item is updated
Then the sellIn is 5
And the quality is 7

Since this item is supposed to be immutable, I can’t really go and set the sellIn or quality after it’s been created. So I made a separate method to pass-in the sell-in and quality at initialisation time:

There could be a nicer way to do this, either by creating all items in this way, or by changing the way the immutable items work, but this was easy enough to do that I didn’t look any further.

So when we run all the scenarios, we get:

[INFO] Feature: Quality changes with sell-in date Feature
[INFO] In order to keep track of quality of items in stock
[INFO] As a ShopKeeper
[INFO] I want quality to change as the sell-in date decreases
[INFO]
[INFO] Scenario: Decreasing Quality of a Basic Item # features/basic.feature:6
[INFO] Given a Store Keeping Item with name "+5 Dexterity Vest" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 5 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 7 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the sellIn is 4 # BasicFeature.theSellInIs(int)
[INFO] And the quality is 6 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality decrease doubles after sell-in has passed # features/basic.feature:14
[INFO] Given a Store Keeping Item with name "+5 Dexterity Vest" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 0 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 10 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the sellIn is -1 # BasicFeature.theSellInIs(int)
[INFO] And the quality is 8 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality never becomes negative # features/basic.feature:22
[INFO] Given a Store Keeping Item with name "+5 Dexterity Vest" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 0 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 0 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the sellIn is -1 # BasicFeature.theSellInIs(int)
[INFO] And the quality is 0 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality of Aged Brie increases with age # features/basic.feature:30
[INFO] Given a Store Keeping Item with name "Aged Brie" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 5 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 1 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the sellIn is 4 # BasicFeature.theSellInIs(int)
[INFO] And the quality is 2 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality of Aged Brie never increases past 50 # features/basic.feature:38
[INFO] Given a Store Keeping Item with name "Aged Brie" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 5 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 50 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the sellIn is 4 # BasicFeature.theSellInIs(int)
[INFO] And the quality is 50 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality of Backstage Passes increases by 1 if sell-in is greater than 10 # features/basic.feature:46
[INFO] Given a Store Keeping Item with name "Backstage passes to a TAFKAL80ETC concert" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 11 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 20 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the quality is 21 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality of Backstage Passes increases by 2 if sell-in is less than 10 but more than 5 # features/basic.feature:53
[INFO] Given a Store Keeping Item with name "Backstage passes to a TAFKAL80ETC concert" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 6 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 20 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the quality is 22 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality of Backstage Passes increases by 3 if sell-in is 5 or less but more than 0 # features/basic.feature:60
[INFO] Given a Store Keeping Item with name "Backstage passes to a TAFKAL80ETC concert" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 5 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 20 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the quality is 23 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality of Backstage Passes is 0 after the concert (sell-in) passes # features/basic.feature:67
[INFO] Given a Store Keeping Item with name "Backstage passes to a TAFKAL80ETC concert" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 0 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 20 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the quality is 0 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality and SellIn of a Sulfuras item does not change # features/basic.feature:74
[INFO] Given a Store Keeping Item with name "Sulfuras, Hand of Ragnaros" with a sellIn date of 5, a quality of 7 # BasicFeature.aStoreKeepingItemWithNameAndSellInDateAndQualityOf(String,int,int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the sellIn is 5 # BasicFeature.theSellInIs(int)
[INFO] And the quality is 7 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality of Conjured items goes down twice as fast as a normal item before sell-in # features/basic.feature:80
[INFO] Given a Store Keeping Item with name "Conjured Mana Cake" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 5 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 20 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the quality is 18 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] Scenario: Quality of Conjured items goes down twice as fast as a normal item after sell-in # features/basic.feature:87
[INFO] Given a Store Keeping Item with name "Conjured Mana Cake" # BasicFeature.aStoreKeepingItemWithName(String)
[INFO] And a sellIn date of 0 # BasicFeature.aSellInDateOf(int)
[INFO] And a quality of 20 # BasicFeature.aQualityOf(int)
[INFO] When the Item is updated # BasicFeature.theItemIsUpdated()
[INFO] Then the quality is 16 # BasicFeature.theQualityIs(int)
[INFO]
[INFO] 12 scenarios (12 passed)
[INFO] 64 steps (64 passed)
[INFO] 0m1.617s
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.861s
[INFO] Finished at: Fri Sep 16 15:57:23 CEST 2011
[INFO] Final Memory: 5M/106M
[INFO] ------------------------------------------------------------------------

So now we have a full set of acceptance tests, covering the whole of the requirements, and with only very limited amount of code needed. Of course, the small amount of code needed is for a large part because I already refactored the original to something more managable. I would be a nice next experiment to start with these scenarios, and grow the code from there. If you do this, let me know, and send a pull request!

Parameterization

For some type of tests, it makes sense to have a scenario where you put in different types of data, and expect different results. By separating the scenario from the input and output data, you can make thes kind of tests much more readable. In the case of our example, you can find the same tests in the parameterised.feature file, but it’s small enough to simply include here:

As you can see, this is much shorter. It does skip on the detailed text for each scenario, though, which I thought to be somewhat of a loss in this particular case. For tests with larger sets of data, this is probably a great feature, though. For this example I though the feature was clearer with the more verbose scenarios.