Integration Tests with Arquillian

Lead technology consultant working for msg systems ag in Germany, Markus Eisele, examines the Arquillian tool for testing application code.

Since the early days of Java Enterprise Edition, testing of
container components and their interaction has been a pain-point.
Their cooperation with other services has been traditionally
difficult to simulate. With the increasing functionality of the
container (dependency injection, etc.) cross-links and dependencies
have also grown. A fully automated integration test suite
previously consisted of many JUnit tests. Now, Arquillian
aims to make this task easier by offering the first complete and
integrated test suite for integration testing.

The mission of the JBoss project Arquillian is easy. It’s about
providing a simple test framework. The aim is to remove all
container lifecycle and deployment tasks from the actual test logic
and enable developers to build a broad range of integration tests
for their enterprise Java applications. Integration tests should be
a breeze with Arquillian. Since the early days of Java EE, testing
enterprise applications has been a major challenge for any project.
The isolated testing of technical components was impossible in some
cases, since simple unit tests were frequently insufficient. One
main reason for this, was the design of the components. They are
rarely completely isolated and often depended on both their runtime
and other components.

In the overall context, these aspects are as important as the
technical work performed by a component. Even with a clean layer
separation, a combination of so-called Mocks (interface simulation)
and unit testing is required to integration test the interfaces and
the implemented logic of the components. And despite all the
potential problems, the actual behaviour of the target container
remains completely untested. Neither transaction control,
dependency injection, or other declarative services can be tested
adequately. It is usually one of those final, open questions
whether the right services are bound or the transferred data is
actually in the proper format. Ensuring this, however, is an
essential part of development processes in companies, and it is
known as integration testing. The art of designing an automated
integration test and regularly carrying them out, is something only
a few experienced developers are able to do. This results in many
stability problems for projects. The JBoss project Arquillian tries
to eliminate this deficiency. It allows developers to easily run
integration tests for container components. It does not matter
whether these are deployed on a local (embedded) or remote
container. The idea behind Arquillian is to make integration tests
as simple as unit tests.

In essence it’s a custom test runner for JUnit and TestNG, which
transfers control of the test lifecycle of the respective test
implementation to Arquillian. From there Arquillian delegates to
the respective environments for testing within the container or
against the runtime. An Arquillian test case still looks almost
exactly like a normal JUnit or TestNG test case. The testcases
basically cover three different aspects. The testcase itself, the
deployment, which has to be packaged into an archive and, last but
not least, the deployment to the runtime.

Neither the deployment process nor the actual container control
is very magical. The packaging of the testcases into the right
format for the target runtime is done by ShrinkWrap. This is also
the true start of an Arquillian test lifecycle. It defines the
basis of the test case, a suitable archive at runtime and deploys
it on the target container. After that the actual execution of the
test case is initiated and results are collected. Finally, the
archive will be removed from the runtime (undeployed). Up to now,
there are only a couple of open-source containers supported. JBoss
(> = 5), GlassFish (> = 3), Jetty (>= 6.1), Tomcat 6, Weld
SE and EE 1.0 or 1.1, 1.0, IronJacamar 1.0, OpenWebBeans Apache 1.0
and Apache OpenEJB 3.1 are among them. According to the
manufacturer future support will also cover commercial platforms
such as Oracle’s WebLogic or IBM’s WebSphere.

First steps

If you want to start with Arquillian today, you should be aware
that this is still an alpha release. You can download a complete
archive from git or build it from sources. The following examples
use the Arquillian Examples project, which is based on Arquillian
1.0.0.Alpha4. If you are trying out this alpha release you are not
an early follower but a leader. You need the complete stack of
brand new technologies.

In addition to Maven 3.0 a current 1.6 Java Development Kit
(JDK) is also required. The best way to install the necessary parts
is to install the complete development environment and extract the
ZIP file with the examples into any directory. The most interesting
use case certainly is the testing of Enterprise Java Beans (EJBs).
In conjunction with the GlassFish Embedded Server 3.1 this is
already included as a pre-configured example. Find it in the folder
ejb31-gfembedded. All examples run with Maven. To use the embedded
GlassFish as a container, create the required profile called
“glassfish-embedded-3″ in the projects pom.xml. The example is
executed using the “mvn test-Pglassfish-embedded-3″ command. A
simple one-liner responds any runs or errors. If everything runs
smoothly you will see: Tests run: 1, Failures: 0, Errors: 0,
Skipped: 0, Time elapsed: 9828 seconds. If this does not work right
away, you have to check for some potential problems and solve them.
The number one reason for any errors are unknown or missing Maven
repository entries. The Arquillian libraries are in the Public JBoss Repository. This should be referenced
from your pom.xml or even settings.xml. If you are experiencing any
java.net.ConnectExceptions the reason could be a known bug in
Embedded GlassFish. It prevents builds greater than 3.1 b04 from
working offline because it is checking for required xml schema
files. The same will happen, if you sit behind a proxy or firewall.
In this case you need to add some configuration parameters via the
Surefire plugin (SystemProperties, http.proxyHost and
http.proxyPort). But there are other circumstances, that prevent
the most recent Arquillian Alpha from running with the latest
GlassFish bits.

During development of GlassFish 3.1 the embedded API changed.
This should be reflected with the next Arquillian release. So if
you are in need of more recent builds, you should wait a little bit
longer for Arquillian to provide an updated version.

If you experience other errors, you are basically helpless at
first. Arquillian log messages are generally rare. So you need to
dig a little deeper into the configuration. By default, the root
directory of GlassFish stays in the target folder which was created
by Maven. Each test run creates a new directory there. You can
change this behavior by making changes to the arquillian.xml
configuration file. It belongs to the srctestresources folder and
should look like this:

Using this configuration, the GlassFish directory used each time
a test is run, is always in the same place. You are now free to
make any additional entries to the embedded GlassFish configuration
by changing the domain.xml. But are you still missing the verbose
logging? Then you have to get your hands on the logging.properties
file. Simply copy it from an existing GlassFish domain and add it
to your development project (For example srctestgf). Parse it on
as a system property (java.util.logging.config.file) to the
Surefire plugin. If you now rerun your tests you can see the
container log messages.

Another very helpful artifact would be to have a version of the
packaged archive that Arquillian is deploying. This is simple, too.
Add the following entry to the arquillian.xml.

Besides satisfying your curiosity, this can be especially
helpful in future debugging. Particularly if the generated archive
can be successfully deployed to a standalone glassfish, it ensures
that you are most likely stumbling over some problems that aren’t
the project developer’s fault.

First Simple Testcase

When all preparations are complete, it is time to devote time to
the actual test functionality. To test a component in the
container, it must be present. A simple Stateless Session Bean
HelloEJB should be enough as the first example:

The world’s best known example EJB is now being reviewed with a
corresponding test case:

The @RunWith JUnit annotation ensures that the Arquillian test
runner is used. After that you simply inject the HelloEJB. The
@Deployment annotation declares the method that wraps the actual
EJB archive. The actual test case is initiated by the annotation
@Test in JUnit by known style. The assertEquals checks the return
value against a fixed input string. If the two strings match, the
test is a success. All other cases produce an error. This simple
error doesn’t say much about the error in general at a first
glance. So you have to check the targetsurefire-reports folder for
details. Each testcase produces a result in text and xml files.
This isn’t very handy, but you have to blame Surefire for that, not
Arquillian. If you change the EJB call into anything to force an
error (e.g. sayHelloEJB (“Markus”) you can find the specific
error:

More Container Functions

In a similar manner as described above you can test almost any
container functionality with Arquillian, with the Stateless EJBs
being the simplest case. It becomes more complicated when the
runtime with the appropriate resources (eg a database or interface)
must be supplied or if Context and Dependency Injection (CDI) is
required. But even for such requirements Arquillian has a solution.
The simplest case is the use of CDI. To enable CDI with your
testcase you simply have to add an empty beans.xml file to the test
archive. This is achieved by:

From now on CDI functions are accessible because test.jar
contains the required beans.xml in the META-INF folder. Another
option is the use of an appropriate beans.xml with content. For
this purpose, it has to be put into the project’s resources
directory and referenced accordingly:

Up to now everything we tested was done using the GlassFish
Embedded API. If you are hoping to test your Java Persistence API
(JPA) applications you have to switch the container because these
are only supported with the remote GlassFish container. This
requires the entire Arquillian configuration to be changed. The
remote deployment is backed by the JSR-88 API. If you are looking
for some good examples you can consult the Arquillian User
Guide.

Arquillian basically manages to test everything which could be
directly injected or by facilitating the CDI features. In addition
to the previous example, you can also use it to test JMS
connections. For this it’s as simple as adding the appropriate
queues, topics and connection factories:

Arquillian as a Client

Having tests for the external system representations, is as
important as focusing on application testing from the inside. You
should spend as much time on the interfaces which people or
programs use to interact with your application. Typically it should
be ensured that each application and its execution path is fully
tested. Third parties can interact with the application through
various interfaces, such as Web Services, EJBs, or remote via http.
If you are taking care of this you have to consider a lot of
things, like object serialization and network connections. In order
to test this view onto the environment Arquillian provides two
modes: IN_CONTAINER and AS_CLIENT. The IN_CONTAINER mode
corresponds to the examples shown above. The AS_CLIENT allows
checking the outside view of an application. Running the AS_CLIENT
mode is as simple as adding another Annotation to your
testcase:

In a direct comparison with the fashion IN_CONTAINER the client
tries to fashion as few tasks to perform in the container and
natural to act as a client. The @Deployment method simply controls
the container’s lifecycle.

Debugging, and Eclipse Integration

If you don’t like the painful cmd-line maven execution of your
Arquillian tests, you are free to integrate it into your favorite
IDE as long as it is Eclipse or NetBeans or has any Maven
integration at all. Eclipse requires you to use the m2eclipse
plugin. You need to activate the relevant Maven profile, and you
can start using Run As | JUnit Test to run individual tests from
the IDE. NetBeans doesn’t require a separate plugin. It has Maven
support since version 6.8. Here too, the active Maven Profile needs
to be added. The configuration is already done. Debugging basically
follows the same characteristics as with other deployed
applications. Test-runner and tests run in a different JVM than the
container itself. Therefore, the debugger has to be attached to the
correct JVM. Details about this can be found in the container
documentation of your choice.

Conclusion

It’s lots of fun to use Arquillian. There is nothing comparable
on the market in terms of seamless container integration. It’s
simple enough to use and not too hard to configure. Simple test
cases can be created in minutes. Tradeoff is, that it quickly
becomes complex. The configuration of the container is only one
area. The more complex your deployments get (e.g. dependent
libraries, special classpath configuration) the harder the setup.
And Arquillian is still an Alpha version which is not finished at
the time of writing. Some things simply don’t work at the moment.
If you are running into trouble it could consume your time very
quickly.

To find out where exactly the problem is, you need time and
knowledge. The core of Arquillian doesn’t log too much. To find a
real fault is not easy. But the project is not complete yet. A
couple of features are already announced. In addition to Method
Interceptors the so-called Arquillian Resources should become
available with the next release. That will make any
container-specific objects (such as the initial context) inject
able via a generic @ArquillianResource annotation. Support for
other containers is also on the list of things to come.

It will be exciting and worth further observation. The project
is slightly behind schedule and it should come up with the next
development milestone soon.

Markus Eisele is a lead technology consultant working for msg systems ag in Germany. Markus is a software architect, developer and consultant. He also writes for German IT magazines. He works daily with customers and projects dealing with Enterprise Java and other new technologies, on a variety of platforms using different vendors. His main area of expertise are Java EE Middleware Servers. You can reach him via twitter.com/myfear or check his blog at http://blog.eisele.net.