Tuesday, 3 May 2016

Data Driven Testing with JUnit Theories and Parametrization

Theories are part of the J Unit test suite and run the same way that parametrization tests work. The function of a theory is to test a large amount of data with a relatively small number of tests.

The main difference, as I see it, between theories and parametrized tests is that with a theory you are testing essentially data that has been generated prior to the test and parametrized tests are better suited to running a large amount of variables through tests to validate them. You see this when you run it, a parametrized test will generate a test for every iteration. This makes it easy to validate a wide range of data. Where a Theory will generate a single test and fail if all of the data is not correct, so its less of a validation more just ensuring what's expected is there.

Theories take data from a parameter marked as a @Datapoint. and uses all the data it in the following methods, when 2 or more methods are used along with data points it works in the same way a 2D array would with a for each loop iterating one over the next until it has used every possible combination of the data points.

Theory's are handy over parametrised tests in that they are less "busy", that is... they are short and easy to read/understand and give a clear (concise) output. Parameters require a little more work and a little more effort to understand but are very limited in there use cases.

Walk through

Lets take a step back from what its for and see how we can use theories and parametrized tests.

Theories

First declare that the class will be using Theories above the class declaration:

@RunWith(Theories.class)publicclassTheoriesClass{

This lets our class know its to run using the Theories test runner.

Next inside the Class declare what data you will be rerunning through the method this is done using the annotation @DataPoints/@DataPoint (Datapoints not available in earlier versions)

and that's it, fairly straightforward way to assert that an object or variable contains something that it must. I find one big differences in using this method is that you don't assert that something is true or false or works in a certain way. The theory will assert a object has a matcher in it or certain value easily enough but not process that data once it has been passed to the theory...that's what a parametrized test is for.

Observations with Theories...

...! if you put in 2 or more variables into the test it iterates over them recursively

with the data points [1,2,3] will show us an output like : 1, 2, 3, 2, 4, 6, 3, 6, 9

..! Theories should be kept to assertions about data that is supposed to relate, trying to do extra processing isn't what its for and will result in frustrations. i.e. modify data to make it readable as use asserts such as assumeThat()

Parametrized Testing

Very similar to theories ( although more of a precursor to theories)

Parametrized tests have more straight forward use cases

To start same as theories we declare the testrunner

@RunWith(Parameterized.class)

publicclassParameterClass{

Next the Parametrized tests use global variables and a constructor so add a something similar to the snippet below

This is one of the big differences with Parametrised tests they use constructors and global in order to achieve repeating the same test over and over.

The variables in the constructors are defined by the @Parameter method/declaration.

In this example you will notice 2 inputs for the tests, adding more inputs is a little trickier using this type of test, since each variable must have a object to bind to, unlike above it wont just replicate and iterate through each scenario.

From that snippet we can see a few things. First notice the additional tag for the name, it i an optional part but allows you to name each of the tests and is very useful when running a lot of data or putting the test into a build where there may be failed components. If you don't add the name annotation they will just be called by the number of the test which can cause pain debugging later.

Next notice how the object is returned in the return statement so it can be bound to the global variables. the Array has 2 objects in it and they are bound in the order of the class constructor. So for the example the object at Array[1] will be inputNumber and Array[2] will be expectedResult.

Finally we get too the class itself, as you can see this one is the same as above except since it is created for each iteration of the test, we use the current global variables to test against and don't pass anything into the test itself.