testing a evolving codebase

We are all fans of automated testing. In our adoption of this principle over the last three years using selenium, we have come across a few pertinent challenges and a few ways to get around them. This blog narrates some of these experiences, starting with the challenges and then our solutions.

Problems

Common Steps

Every test has some common steps, for example all our tests start with login. Sadly any change to these common steps break all our tests. In a evolving code base, these changes are more common than we would want them to be.

Rich UI

A lot of code is generated especially with frameworks like GWT, it is imperative for automation test authors to understand which locators they can count on, is it xpath, id, css or some other strategy. In our experience, css is the best bet, as all buttons would uniformly belong to the same style.

Pure Client Side approach

Some things can not be validated or completed with a pure client side approach. For example we want to search a order, the assumption for this test is that the order exists, an independent test would like to create the actual order and then search for it. Searching is the only part that we are actually interested in from the testing perspective, having a server side support for quickly creating this order from the test system is really handy.

Language choice

It is far easier to debug a test in selenese with the IDE, however selenese only supports javascript eval which at times comes short for some operations. We can use any of the drivers, but that trades off some of the readability and the reverse syncing. For example once I have written a java test, if any of the locators change, I can not rely on the IDE to make this change easily.

Solution

DSL

A simple DSL that is built on using composite commands solves the common step issue. Selenium has a very extensible command execution model, a few hooks into the selenium-executionloop.js and this can be achieved. For example we can create a login command, which is made of opening a site, typing in the uid and pwd and clicking the submit button. Since the common steps do not need to be repeated in each test now, any change to them is restricted to one place.

Smart Locator

Selenium locators already understand that for recording clicks on link, it is enough to just record the actual text associated with the link. Once we externalize the styles, for example that all buttons have a class “btn” and tie it up to the locators, we are able to find a button based on the style class and the text which is actually recorded in the test. If we externalize this text and the style class, any changes to the class, can now be restricted to one place.

Java Javascript Collaboration

We have a simple java layer that can execute selenese in headless mode (converting them into the selenium java driver calls) and hooks in selenium IDE (using LiveConnect) which can execute java commands. This helps us get around pure client side limitations and language issues. One of the keys in this approach is to make writing a custom java command as simple as possible, in our implementation one plain class with a annotation in a magic package does it.

A technology enthusiast who was lucky enough to work on multiple products spread across different business verticals with various technical needs. Saw some smart solutions which could handle millions of transactions per second and some products which had their own custom languages.
Interested in anything which is technically fun. An open source enthusiast, who is yet to make his mark :(