We are employing BDD and using SpecFlow to drive our development (ATDD).

Our QA team would like to define their own 'end-to-end regression tests (in Gherkin/SpecFlow) and re-use the Steps we have already defined.

(Please note - I know that this is not a great example but it should provide enough details)

A test may include..

Log in

Search for a product

Select a product to buy

Create an order

Select delivery option.

Submit the order.

Cancel the order.

This would suggests a scenario like..

Given I am logged in
When I Search for a product
And I Select a product to buy
And I Create an order
And I Select delivery option
And I Submit the order
And I Cancel the order
Then ??!!

Which is clearly wrong as we are not checking the output at each step.

So this may be resolved as a sequence of scenarios:

Scenario 1:
Given I am logged in
When I Search for a product
Then I see a list of products

Scenario 2:
When I select a product to buy
Then I can create an order

Scenario 3:
When I create the order
And I Select delivery option
Then I can submit the order

etc etc

The main issue with this is that there seems no way to specify the order/sequence that the scenarios are run in (a characteristic of nUnit?). Because there are dependencies between scenarios (they are not set to a know starting point) they must be run in sequence.

My questions are:

a) Are we trying to fit a square peg in a round hole?!

b) Does anyone know if there is a way to use SpecFlow/Gherkin in this way?

1 Answer
1

I would say that you are writing your scenarios on the wrong abstraction level. But that depends on what you want to use them for;

If you want to write test-scripts then you are on the right track... but it will be a nightmare to maintain as it, in the first case (long script) will be very brittle and the second case (several scenarios) need to ensure a certain execution order. Both of them are discouraged and considered anti-patterns.

I would suggest that you merge the ATDD-tests you are writing and talk to the test department to get their view on the matter and include the test-cases they need to ensure that the system is thoroughly tested. Who know? You might even learn something from each other :P

And when you write those "specifications" (as I rather call them) you write them on a higher level. So instead of writing:

Given I am logged in
When I Search for a product
And I Select a product to buy
And I Create an order
And I Select delivery option
And I Submit the order

you write something like

When I submit an order for product 'Canned beans'

In the step-definitions behind that step you perform all that automation (login, browsing to the product page, select the delivery options, submit the order).

All of this can be read about in these great articles on how to write maintainable UI Automation tests:

Many thanks for your answer Marcus and the articles you list - very helpful! I am a little clearer now on the quesion from test. That is 'do we have is the ability to chain already-written scenarios together to run in sequence to perform end-to-end system tests?' e.g. log in then browse products then submit order then register account then edit account then search orders then view order then cancel order then etc etc. It'd be useful therefore to be able to chain scenarios as When Then When Then When Then.. Unfortunately there does not seem to be a way to run scenarios in specific order.
–
Mark ChidlowDec 15 '11 at 8:49

Update: if the scenarios are named '001 Do this', '002 Do that' etc then they are run in that order. Feels a little hacky but for our purposes this will do until a better solution can be found.
–
Mark ChidlowDec 15 '11 at 14:04

I would definitively recommend against writing scenarios that are dependent on order. That will get hard to maintain...
–
Marcus HammarbergDec 15 '11 at 14:45