Thursday, January 27, 2011

My cucumber tests usually have two sections, .manual and .feature, files. I use cucumber to write my automated test cases as well as the manual test cases. This way all of my test cases can be accessed from one project and you can easily differentiate which features are automable and which do not.

The good thing with cucumber is that it will only run the files that have .feature on it and ignoring the rest.

The way I run my test is usually by running the whole folder which is something like this:

cucumber features\regression

inside that folder I will have a mix of .manual and .feature files and folders.

Recently, I have been getting this error when I run my cucumber tests.

I check the feature file and everything looks ok. It actually took me a while to figure out that the mistake was just because I accidentally named a .manual file into a .feature file and thus the error as cucumber could not understand how to run a manual test. What an annoying mistake to make!

Monday, January 24, 2011

After finding out how to run my cucumber test through RubyMine, I try to enable the ruby-debug from RubyMine as well. It turns out that I need to install another ruby gem called Ruby-debug-ide which for some reason I could not download it straight from the RubyMine ide.

Wednesday, January 19, 2011

I have been running my cucumber test using a command line console. I am so used to running it from the console that I did not bother to figure out how to run the cucumber test through the RubyMine ide. However, I was shown by my colleque on how to set it up:

From RubyMine you would need to do the following:1. Click on File->Settings2. Click on Ruby SDK and Gems3. Click on Attach Gems4. Locate your Cucumber gems and attach it.5. Click Ok

Next is to configure your cucumber test environment variables.

1. Click on Run->Edit Configurations2. Add the environment variables on the Environment Variables field

You can also specify which test you want to run on the "Feature file" field.

After that, all you need to do is to click the Run (Shift + F10) button to run your cucumber test from RubyMine.

Friday, January 7, 2011

We found it useful to have a cucumber writing standard document where everyone can use it as a reference to write the scenarios. Without the standard we will have inconsistency in the features files.

1. The first cucumber writing standard that we have is to use a third person narrative when we write Given/When steps.

Given the user is on Google homepage And the user clicks on the Google Search button

We find it more relevant to use "the user" instead of using the first person narrative "I"

2. We use "is/are" in the Given steps for assigning a variable.

Given price is "5" dollar And quantity is "10"

And we use "should" in the Then steps for asserting result

Then the total price should be "50" dollar

This is so that we can differentiate quickly which one is the pre condition and which one is the assertion.

The above examples are the two things that are most argued in our team. Some people prefer to use "I" and some don't like to use the word 'should'. I guess there is no right or wrong answer. However, since we are working in a team, it is important that we come up with a cucumber writing standard so that we can have a consistent writing.

3. Condition statements (e.g if statements) are to be avoided in the scenario. This might sound obvious but for a tester who is new to cucumber might not find it that obvious.

Instead of

Given the user is on Yahoo.com

Then the user should see his name if the user has signed in

We separate the scenarios into two lines

Scenario: User has signed in Given the user is on Yahoo.com

When the user signs in into his account as "bananta" Then the user should see his name as "bananta"

Scenario: User has not signed in Given the user is on Yahoo.com Then the user should not see "bananta"

4. Another standard that we have is that testers own/maintain the cucumber scenarios and the developers maintains the steps defintions. In other words, testers should not modify the steps definition and developers should not modify the test scenarios. This might sound obvious but in our case, the devs are modifying the scenarios without the testers' knowledge which cause the test cases to behave differently.

We have more cucumber writing standards that are specific to our projects but the above examples are the main one that we have.