Mah blogginess

Maven, TestNG, and SeleniumRC

Today, I set up a Maven module to run SeleniumRC (Java) tests via TestNG. It wasn’t a straightforward or easy task, so I thought I’d share some of what I found with you.

We opt to have all of our modules stay in lock-step version-wise, so we keep a pretty large structure in our SVN trunk. We include/exclude modules for compilation by using Maven profiles, so we don’t have to build the whole structure if we are only changing one small component. Our top-level pom.xml file looks similar to this:

As you can see, running “mvn clean install” at this top level would only end up building the baseModule. To get all the modules, we run “mvn clean install -Pall” which builds baseModule along with otherModule and web-test and anything else as represented by the ellipse. The web-test module is the one we will be interested in.

For our heavy-weight tests (non-unit tests, like functional or web testing), we like to have the module compile, but not run unless we specifically trigger the tests. To achieve this, we create a pom.xml (for the web-test module) that looks something like this:

It sets up src/it/java as the location holding the source code for the tests

It sets the surefire plugin (which runs your tests) to only run at the integration-test part of the Maven lifecycle

It says to ignore any tests in the TestNG “web-test” group

Note the packaging says this module is of the type “pom”. I found that if this is set to “jar”, the tests run twice for some unknown reason

Now, if you had tests in src/it/java (that were all annotated as “web-test”) and you ran “mvn clean install”, you’d see that zero tests ran. This is what we want! Now, we have to get the tests running if we pass a specific profile in. To achieve this, I added the following profile definition to the web-testpom.xml.

This profile is invoked when you run “mvn clean install -Prun-web-test”. There are several things I’d like to point out here:

At runtime, this overrides the previously-declared surefire plugin configuration, but only when invoking the profile.

Originally, I was mostly getting stuff to work without the <suiteXmlFiles /> section, but with the <groups /> and <forkMode /> sections… But it was running every single test class as it’s own TestSuite.

To fix the multiple TestSuite (and @Before*, @After* problems that I’ll discuss later), I ended creating my own testng.xml file. Having this file and using the <suiteXmlFiles /> element made things a whole lot easier for configuring my TestNG executions than trying to configure the tests through the clumsy interference of the surefire plugin.

As much as I like Maven, I have to say that the lack of documentation and general obfuscation that happens when using their plugins can really drive a person BATTY!

Now, I need to start integrating the stuff I need for using SeleniumRC. I found some decent help on how to start up and shut down the selenium-server, however I disagree with using a post-integration-test phase to shut down the server, because if the tests fail in the integration-test, Maven won’t progress to the next phase and you’ll be left with a spuriously running selenium-server. The client API’s do have access to the command that the was being run to shut the server down, so it is just a matter of having some TestNG @AfterSuite or @AfterGroup method issue the Selenium.shutDownSeleniumServer() command.

This brings me to some of the TestNG wonkiness I saw. I decided to use a Singleton Data Provider (named SeleniumFactory) to deliver the Selenium objects to the individual tests. (Selenium objects are time-expensive to create, so I wanted to only create 1 for the duration of the entire test suite.) I wanted to initialize the SeleniumFactory object in a @BeforeGroup method, deliver the Selenium object as a Data Provider, and then tear the SeleniumFactory down in a @AfterGroup method. (At first, I was fighting Maven’s handling of each test as its own test suite which was giving me sequence issues, which is why I ended up defining my own testng.xml file, as I stated above.) Let me start with some code to show you what I mean:

I would have assumed that an @BeforeGroup method got called before a data provider method, but I guess my assumption is wrong, eh? After I figured out this order-of-method-calls business, I was able to adapt my SeleniumFactory accordingly.

… And that is my story. To summarize: each of these tools could be better documented, but at the end of the day I was able to get a Maven module up and running SeleniumRC tests via TestNG. Now, on to the test-writing!