At the bottom are regular unit tests. There are a lot of these because we aim to cover as much of our code as possible.

But unit testing only tests parts of our code in isolation - it can’t see how things play together. For that, we have those two layers above:

Service / integration testing - making sure components of our software interact correctly with each other. For example, that your application can successfully connect to the database.

UI / acceptance testing - making sure a feature in the software actually works from the user’s perspective. For example, your user can enter a known username and successfully log in.

Why are these layers important? Imagine you need to test a tax calculator on an eCommerce site. This is a crucial feature because if it fails, people can’t buy and the company loses money.

So you write lots of unit tests to make sure every possible input to the tax calculator produces the right output. With these tests in place, you feel confident that no matter which country, region, currency and purchase amount are entered, the tax amount will be correct.

After some tweaks, all the unit tests pass. Build is green! Ready to go!

But then you get a call from management (for dramatic effect, let’s say it’s 2am) saying the shopping cart interface is broken on the site.

You start a browser and go through the purchase process, and then at the last stage, when users need to view the tax amount,the amount isn’t shown at all. A browser issue caused another DIV to move out of place, and the tax amount is hidden by an ugly rectangle.

(We realize this example is a bit over the top. But things like this do actually happen.)

So even with all those unit tests, the user experience is broken. 100% code coverage is important - but just on it’s own, it can’t make sure that critical software functions actually work. That’s why integration and unit tests are needed.

Most of us know this in the back of our minds. Only a few of us actually venture out to write and execute these tests. But in fact,it’s much easier than you think.

Unit tests and Selenium tests - happy neighbors?

Here’s a JUnit test folder of a small Jenkins plugin we have written especially for this post. On first glance it looks like just any other test folder.

But take a closer look:

In this JUnit folder, living peacefully side-by-side, are tests from all three tiers of the testing pyramid. The fourth test listed is a regular unit test, but the top three tests in the list are actually based on Selenium 2.

And the acceptance tests for this Jenkins plugin have Selenium code that drives a real browser and simulates user actions.

For example, here’s a bit of code that uses a real browser (e.g. Chrome) to click the link “Manage Jenkins”, then click “Manage Plugins”, then verify that the plugin is installed:

public void pluginIsListed() {
//Find and click on the 'Manage Jenkins' link
webDriver.findElement(By.linkText("Manage Jenkins")).click();
//It takes a few seconds for the page to load, so instead of running Thread.sleep(), we use the WebDriverWait construct
WebDriverWait wait = new WebDriverWait(webDriver, 30);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("h1")));
//assert that we are on the Manage Jenkins page
assertEquals("Header not found", "Manage Jenkins", webDriver.findElement(By.cssSelector("h1")).getText());
//Find and click on the 'Manage Plugins' link
webDriver.findElement(By.linkText("Manage Plugins")).click();
wait.until(ExpectedConditions.visibilityOfElementLocated(By.linkText("Installed")));
//Find and click on the 'Installed' link
webDriver.findElement(By.linkText("Installed")).click();
assertEquals("Plugin link not found", "Jenkins Sauce Health Check plugin", webDriver.findElement(By.linkText("Jenkins Sauce Health Check plugin")).getText());
}

Cool huh? Let’s dive in a bit deeper and see how simple it is to cover all three tiers of the testing pyramid in one JUnit test suite.

Code example with all three testing tiers: Sauce Labs Health Check

Don’t worry - we’re not pushing our product (just yet ;) - we just based our example on a Jenkins plugin that interacts with our Sauce cloud service. But it could just as well be any Java code in any application.

(Oh, in case we haven’t met, we’re the team at Sauce Labs. We help thousands of organizations do Selenium testing on the cloud.)

The simple plugin we have written is called Sauce Health Check - see it on Github. Here’s what it does:

Adds a line to the bottom of the Jenkins interface, like this:

Performs a REST call to the Sauce Labs cloud service to see its status - up or down.

Depending on the status, the message will be green and will say “Basic service status checks passed”, like in the screenshot; otherwise red and will say “Sauce Labs down.”

The “Check now” link is an AJAX call. When clicked, it shows a little “working” icon, and re-checks the status of the service. We purposely made this an AJAX call because that’s something that can only be fully tested with browser/UI automation.

In case you’re wondering - why would you need to see the Sauce Labs status at the bottom of your Jenkins dashboard? No reason. It’s just the simplest piece of software we could think of that could help us demonstrate all three tiers of Java testing.

The code is simple and we won’t run you through it - suffice it to say that there’s a descriptor that tells Jenkins this is a plugin, a SauceStatusHelper which calls the Sauce Labs REST API to see if the service is up or down, and a PageDecorator which defines how the result is displayed on the Jenkins interface.

It’s showtime! See the tests

As promised, here are all three tiers of testing within one JUnit test suite, testing the Sauce Health Check Plugin.

1. Unit Tests

The unit tests reside in SauceStatusPageDecoratorTest.java. Here we have 3 functions that test which message is returned by PageDecorator for three possible statuses of the Sauce Cloud.

The REST API is mocked out to create a self-contained test of the code. If these unit tests pass, it means that the basic logic of the plugin is coded correctly.

This preceding test is important because, in reality, the service is very rarely down. So it’s difficult to test this possibility in a UI automation test, but at least we make sure it’s supported in the code.

2. Integration Test

Next up we want to check that the “integration” of our plugin has succeeded - that the plugin is properly installed on Jenkins. We do this using a bit of Selenium 2 (A.K.A Selenium Webdriver) code. The integration tests are in IntegrationIT.java.

First off we define the WebDriver instance variable (WebDriver is the piece of Selenium that automates the browser), and define a JUnit rule that launches a local Jenkins instance with our plugin installed.

Testing plugin message while navigating through Jenkins pages

Now we’ll test that the plugin’s message is displayed as we navigate through the Jenkins user interface. We expect to always see a message like this at the bottom of the screen:

We find and click the Manage Jenkins link using the WebDriver findElement function, wait for the page to load, and make sure that the Sauce Status message appears at the bottom of the screen. This is done by selecting an element with ID equal tosauce_status:

Now we use an XPath expression to click the Jenkins link in the navigation bar, and verify the Sauce Status message is still displayed:

webDriver.findElement(By.xpath("//ul[@id=\"breadcrumbs\"]//a[1]")).click();
//wait until the 'Welcome to Jenkins' div is visible wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//td[@id='main-panel']/div[2][contains(text(), 'Welcome to Jenkins!')]")));
assertNotNull("Status not found", webDriver.findElement(By.id("sauce_status")));

Now we click through to a fourth page, UI Samples, this time using a CSS selector, and again check for the message at the bottom:

And lastly, cleaning up after ourselves - closing down the Firefox browser we used to perform the tests:

@After public void tearDown() throws Exception {
webDriver.quit();
}

Now let’s do the same thing on multiple browsers with Sauce

In the Selenium tests shown above, we only tested the plugin on one browser, Firefox. In a real application, you’d want to test the user interface on multiple browsers and operating systems.

Of course, this adds the complexity of having to install all those platforms on the computer that is running your tests - you might want to test on dozens or hundreds of different platforms. This complexity is part of the reason why many organizations are hesitant to do acceptance testing on a significant scale.

An easy way to run your Selenium tests on a large number of platforms is the Sauce Labs cloud service. We support JUnit, and maintain over 700 browser/OS combinations, including Android and iOS. So you can just specify which platform you want to run your tests on, within your JUnit test files, and we’ll handle it for you.

Keep in mind - Sauce is a commercial service. We have a 14 day Free Trial which you can use to run automated or manual tests.

However, we’re completely free for open source projects - in fact, we are used by Mozilla, YUI, Travis, JQuery, and many other popular open source projects.

Running the same integration and acceptance tests on the Sauce cloud

In the JUnit test folder, there is another file called SauceIT.java. These are exactly the same Acceptance Tests as we described above, but they run on multiple browsers in the Sauce Cloud. Here’s how it works -

The following variables define the Sauce Connect tunnel (assuming you are testing behind a firewall), and the operating system, version and browser you want to test on.

Summary

It’s been a long post. But we hope we’ve made our point that JUnit users should be doing more than just unit testing. And that really, it’s quite easy to add integration and acceptance tests with UI automation to your JUnit testing repertoire.

Our code example of a simple Jenkins plugin demonstrated how within one JUnit test suite, you can run regular unit tests, integration tests, and acceptance tests with full UI automation. We showed some pretty advanced browser automation flows, achieved in only a few lines of code.

Finally, we took the liberty of showing how the Sauce Labs cloud service fits in - helping you easily run the same Selenium tests on a large number of browser/OS platforms.