Friday, September 30, 2011

The default cucumber options is to not expand the scenario outline table in the reporting. So when you run bundle exec features, it will print out the report like the one below:

Scenario Outline: Add two numbers
Given I have entered "<num_1>" into the calculator
And I press the add button
And I have entered "<num_2>" into the calculator
And I press the equal button
Then the result should be "<result>" on the screen
Examples:
|num_1 | num_2 | result |
| 1 | 1 | 2 |
| 2 | 2 | 4 |
| 3 | 5 | 8 |

In one way this is good because the report is concise and easily readable. However, one issue that I have is when there is an error in the scenario. Let say the error is on the "And I press the equal button", then you would not know it because it will not show which one is the error.

One way to overcome this is to use the "--expand" options from cucumber --help which will expand the scenario outline. I found this very useful when you have thousands of scripts running. This --expand option will give a much more verbose reporting.

Here is the result of running bundle exec --expand cucumber features:
As you can see, if the error occurs in the "And I press the add button" then cucumber will highlight it with red.

Scenario Outline: Add two numbers
Given I have entered "<num_1>" into the calculator
And I press the add button
And I have entered "<num_2>" into the calculator
And I press the equal button
Then the result should be "<result>" on the screen

Examples:

Scenario: | 1 | 1 | 2 |
Given I have entered "1" into the calculator
And I press the add button
And I have entered "1" into the calculator
And I press the equal button
Then the result should be "2" on the screen

Scenario: | 2 | 2 | 4 |
Given I have entered "2" into the calculator
And I press the add button
And I have entered "2" into the calculator
And I press the equal button
Then the result should be "4" on the screen

Scenario: | 3 | 5 | 8 |
Given I have entered "3" into the calculator
And I press the add button
And I have entered "5" into the calculator
And I press the equal button
Then the result should be "8" on the screen

-r, --require LIBRARY|DIR Require files before executing the features. If this
option is not specified, all *.rb files that are
siblings or below the features will be loaded auto-
matically. Automatic loading is disabled when this
option is specified, and all loading becomes explicit.
Files under directories named "support" are always
loaded first.
This option can be specified multiple times.
--i18n LANG List keywords for in a particular language
Run with "--i18n help" to see all languages
-f, --format FORMAT How to format features (Default: pretty). Available formats:
debug : For developing formatters - prints the calls made to the listeners.
html : Generates a nice looking HTML report.
json : Prints the feature as JSON
json_pretty : Prints the feature as prettified JSON
junit : Generates a report similar to Ant+JUnit.
pdf : Generates a PDF report. You need to have the
prawn gem installed. Will pick up logo from
features/support/logo.png or
features/support/logo.jpg if present.
pretty : Prints the feature as is - in colours.
progress : Prints one character per scenario.
rerun : Prints failing files with line numbers.
stepdefs : Prints All step definitions with their locations. Same as
the usage formatter, except that steps are not printed.
usage : Prints where step definitions are used.
The slowest step definitions (with duration) are
listed first. If --dry-run is used the duration
is not shown, and step definitions are sorted by
filename instead.
Use --format rerun --out features.txt to write out failing
features. You can rerun them with cucumber @rerun.txt.
FORMAT can also be the fully qualified class name of
your own custom formatter. If the class isn't loaded,
Cucumber will attempt to require a file with a relative
file name that is the underscore name of the class name.
Example: --format Foo::BarZap -> Cucumber will look for
foo/bar_zap.rb. You can place the file with this relative
path underneath your features/support directory or anywhere
on Ruby's LOAD_PATH, for example in a Ruby gem.
-o, --out [FILE|DIR] Write output to a file/directory instead of STDOUT. This option
applies to the previously specified --format, or the
default format if no format is specified. Check the specific
formatter's docs to see whether to pass a file or a dir.
-t, --tags TAG_EXPRESSION Only execute the features or scenarios with tags matching TAG_EXPRESSION.
Scenarios inherit tags declared on the Feature level. The simplest
TAG_EXPRESSION is simply a tag. Example: --tags @dev. When a tag in a tag
expression starts with a ~, this represents boolean NOT. Example: --tags ~@dev.
A tag expression can have several tags separated by a comma, which represents
logical OR. Example: --tags @dev,@wip. The --tags option can be specified
several times, and this represents logical AND. Example: --tags @foo,~@bar --tags @zap.
This represents the boolean expression (@foo || !@bar) && @zap.

Beware that if you want to use several negative tags to exclude several tags
you have to use logical AND: --tags ~@fixme --tags ~@buggy.

Positive tags can be given a threshold to limit the number of occurrences.
Example: --tags @qa:3 will fail if there are more than 3 occurrences of the @qa tag.
This can be practical if you are practicing Kanban or CONWIP.
-n, --name NAME Only execute the feature elements which match part of the given name.
If this option is given more than once, it will match against all the
given names.
-e, --exclude PATTERN Don't run feature files or require ruby files matching PATTERN
-p, --profile PROFILE Pull commandline arguments from cucumber.yml which can be defined as
strings or arrays. When a 'default' profile is defined and no profile
is specified it is always used. (Unless disabled, see -P below.)
When feature files are defined in a profile and on the command line
then only the ones from the command line are used.
-P, --no-profile Disables all profile loading to avoid using the 'default' profile.
-c, --[no-]color Whether or not to use ANSI color in the output. Cucumber decides
based on your platform and the output destination if not specified.
-d, --dry-run Invokes formatters without executing the steps.
This also omits the loading of your support/env.rb file if it exists.
-a, --autoformat DIR Reformats (pretty prints) feature files and write them to DIRECTORY.
Be careful if you choose to overwrite the originals.
Implies --dry-run --formatter pretty.
-m, --no-multiline Don't print multiline strings and tables under steps.
-s, --no-source Don't print the file and line of the step definition with the steps.
-i, --no-snippets Don't print snippets for pending steps.
-q, --quiet Alias for --no-snippets --no-source.
-b, --backtrace Show full backtrace for all errors.
-S, --strict Fail if there are any undefined or pending steps.
-w, --wip Fail if there are any passing scenarios.
-v, --verbose Show the files and features loaded.
-g, --guess Guess best match for Ambiguous steps.
-l, --lines LINES Run given line numbers. Equivalent to FILE:LINE syntax
-x, --expand Expand Scenario Outline Tables in output.
--drb Run features against a DRb server. (i.e. with the spork gem)
--port PORT Specify DRb port. Ignored without --drb
--version Show version.
-h, --help You're looking at it.