The Swifty iOS DevOps and Continuous Delivery Blog 

Behat with Sauce Labs and Jenkins

Note : This post may not work with recent Behat. Please use Official Docs for Behat OR my Github repo for to get it working without any issue https://github.com/Shashikant86/BehatDemo

Abstract:

The key benefits of Behavior-Driven-Development (BDD) practices are communication enhancement and customer satisfaction. You can read more about that by Dan North and Gojko Adzic. Perhaps the biggest practical challenge in the way of reaping those benefits is the burden of provisioning, installation and maintenance of requisite complex and fussy infrastructure. The recent availability of latest CI servers like Jenkins & cloud based testing services like Sauce Labs carries the potential to remove that barrier. This post discusses and shows how to integrate Behat an emerging BDD framework for PHP with continuous integration server like Jenkins and cloud based testing services like Sauce Labs.

What is Behat?

Behat is a BDD framework for PHP. There are some tools available for BDD like Cucumber for Ruby, SpecFlow for .NET and Lettuce for Python. Behat is the first BDD tool for PHP applications. Developers can also use the PHPSpec framework to implement classes within the Behat projects. Behat is written in PHP by Konstantin Kudryashov. With Behat, you can write human readable stories which turns as tests to run against your application. Behat can be used for API testing, functional testing and data-driven testing. Developers will do API testing and we will carry on with functional testing (web acceptance testing) with Behat.

Functional Testing with Behat and Mink

Behat is used for acceptance testing (any tests) by executing a Gherkin scenario. Developers can implement integrated classes. Testers start thinking of more workflow level and technical level steps (actions) which turns as scenarios for features. Once tester started to think of Web Acceptance Testing (functional testing) with browser interaction then another tool called “Mink” comes into the picture.

Mink is used for browser emulation (functional testing) where browser interaction takes place. As of now, there are following Selenium drivers available for browser emulation.

This file is a default config file that Behat uses to execute features. Example of behat.yml is shown below:

YAML

1

2

3

4

5

6

7

default:

context:

parameters:

javascript_session: selenium

base_url: http://en.wikipedia.org/wiki/Main_Page

browser: firefox

show_cmd: open %s

You can change the drivers by changing the “javascript_session” parameter. It can be “selenium” or “webdriver”.

sauce.yml

This file is used for running features on Sauce Labs. The code for this file is explained in “behat and Sauce Labs” section below.

build.xml

This file is used for running features with ANT. We can use this ANT file to plug into Jenkins. Simple ANT file should look like this:

XHTML

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

<project name="behat"default="behat"basedir="">

<delete dir="${basedir}/report" />

<mkdir dir="${basedir}/report"/>

<target name="behat">

<exec dir="${basedir}"executable="behat"failonerror="true">

<arg line="-f junit --out ${basedir}/report"/>

</exec>

</target>

<target name="create-test-report"

description="Generate reports for executed JUnit tests.">

<junitreport todir="./report">

<fileset dir="${basedir}/report">

<include name="TEST-*.xml"/>

</fileset>

<report format="frames"todir="./report/html"/>

</junitreport>

</target>

</project>

report

This directory is used to store *.xml reports generated by Behat’s “junit” formatter. We can use this reports to plug into Jenkins.

Start your Engine

Remember, you have to download latest version of selenium server. Now navigate to directory where you saved selenium server .jar file. You have to launch it using command shown below:

Shell

1

java-jar selenium-server-standalone-2.15.0.jar

Behat & Sauce Labs

Sauce Labs is a cloud testing service which allows selenium tests to run in the cloud. Sauce Labs allocates machines and browsers for your tests, capture screenshot for every step and record video of all jobs(tests). You don’t need to setup separate machines to run tests. Sauce labs helps us to write tests without complex infrastructure.

In order to integrate Behat with Sauce Labs, you need to have an account with Sauce Labs. You need “Username” and “API Key” to plug them into a config file.

Behat executes features with “behat.yml” file by default, but we can run features with any other configuration file. We can create another configuration file like “sauce.yml” to run features on Sauce Labs. Example “sauce.yml” should look like this:

YAML

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

default:

context:

parameters:

default_session: goutte

javascript_session: selenium

base_url: http://en.wikipedia.org/wiki/Main_Page

browser: firefox

selenium:

host: ondemand.saucelabs.com

port: 80

browser: >

{

"username": "your username",

"access-key": "your API key",

"browser": "firefox",

"browser-version": "7",

"os": "Windows 2003",

"name": "Testing Selenium with Behat"

}

We will use “sauce.yml” as a config file to run features on Sauce Labs. If you wish to run all features from “features” directory on Sauce labs, you can use this command:

Shell

1

behat-csauce.yml

When to implement step definitions?

If you can speak fluent Gherkin, then you don’t need to write code. Behat/Mink will understand Gherkin and run your features without suggesting step definitions.

If features written by someone else, you can take full advantage of Mink API’s in order to implement step definitions suggested by Behat/Mink.

It’s very important to write good Gherkin to write minimum code.

It’s very easy to access Mink API’s by writing simple code.

Now we will see how will you “click” particular element on page. You need to use Xpath as a locator for that element. Mink will suggest some step definitions, you need to complete it like this:

Example : Feature wikiSearch

We will write simple feature to add product into shopping cart. Feature will look like this:

Vim

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

Feature:wikiSearch

Inorder to search information on wiki

AsaWiki user

Iwant to get sensible results from site

@javascript

Scenario Outline:Search Keywords on Google

GivenIam on"/"

AndIfill insearchBox with"<input>"

WhenIpress search button

ThenIshould see"<output>"

Examples:

|input|output|

|London|lʌndən/|

|NewYork|nɪuˈjɔək|

|Sydney|sɪdni/|

|Mumbai|मुंबई|

|Bejing|北京|

|Tokyo|東京|

|Lahore|لاہور|

|Paris|paʁi|

Feature Explained

Gherkin Language

A feature file mentioned above is a good example of data-driven testing. This feature will execute our scenario for 8 different data set mentioned in the example section. This feature will have following steps

You can implement step definitions forundefined steps with these snippets:

/**

*@Given/^Ifill insearchBox with"([^"]*)"$/

*/

public functioniFillInSearchboxWith($argument1)

{

thrownewPendingException();

}

/**

*@When/^Ipress search button$/

*/

public functioniPressSearchButton()

{

thrownewPendingException();

}

You can see above, Behat/Mink has suggested some step definitions for undefined steps. We can implement these step definitions using Mink in “bootstrap/FeatureContext.php” file. We can implement first step definition “iFillInSearchboxWith($argument1)” like this:

Shell

1

2

3

4

5

6

7

8

/**

*@Given/^Ifill insearchBox with"([^"]*)"$/

*/

/*

public functioniFillInSearchboxWith($input)

{

$this->fillField("searchInput",$input);

}

Now we will run “behat”command again, you can see eight step is passed. Terminal window output should be like this:

You can see detail steps, screenshots and video of this job on Sauce Labs
See terminal output as shown below:

Shell

1

behat-csauce.yml

terminal output

Building Features with Jenkins

Continuous Integration

Continuous Integration (CI) is one of the best practice in agile projects to detect bugs early. Each integration is verified by an automated build to detect integration errors as quickly as possible. Now we will see how we can integrate Jenkins to our behat project.

Don’t forget to start your engine before building project in Jenkins. You have start Selenium/WebDriver before starting Jenkins.

There is “/report” directory to save JUnit reports in xml format generated by Behat. You can generate reports using this command:

Shell

1

behat-fjunit--out~/pathto/report

To continue, you need Jenkins installed on your machine. Now start Jenkins by executing *.war file from terminal on port 8080. You also need to visit “http://localhost:8080” to see Jenkins GUI.

Shell

1

$java-jar jenkins.war

Visit http://localhost:8080/ and you should see Jenkins Dashboard.

Create “New Job” for behat project.

Specify your SCM to create Jenkin’s workspace on your system.

You can specify build file as per your project structure. In here I have configured in “Acne” project by specifying build file path and test reports like this.

Save your configuration. Go back to project and click”Build Now”, you will see:

build now

You can see features running in browser.

Sit back and enjoy your features running on Jenkins till it finishes job.

Few minutes later, you will see your build is “Green”:

Build Success

Watch your test results:

Jenkins Test Result

Behat, Sauce Labs and Jenkins

You can run all your features on Sauce Labs just by updating ANT file like:

We can write web acceptance tests with Behat and Mink combination. We can plug them into Sauce Labs with config file (“sauce.yml”) and run them on a CI server. Now you can implement BDD practices for PHP applications by taking benefit from Bahat and Sauce Labs.