JUnit Parameterized Tests

While testing, it’s common to execute a series of tests which differ only by input values and expected results. As an example, if you are testing a method that validates email IDs, you should test it with different email ID formats to check whether the validations are getting correctly done. But testing each email ID format separately, will result in duplicate or boilerplate code. It is better to abstract the email ID test into a single test method and provide it a list of all input values and expected results. JUnit supports this functionality through parameterized tests.

To see how parameterized test works, we’ll start with a class with two methods which we will put under test.

The EmailIdUtility class above has two utility methods. The createEmailID() method accepts two String parameters and generates an email ID in a specific format. The format is simple – If you pass mark and doe as parameters to this method, it returns [email protected]. The second isValid() method accepts an email ID as a String, uses regular expression to validate it’s format, and returns the validation result.

We will first test the isValid() method with a parameterized test. JUnit runs a parameterized test with a special runner, Parameterized and we need to declare it with the @RuntWith annotation. In a parameterized test class, we declare instance variables corresponding to the number of inputs to the test and the output. As the isValid() method under test takes a single String parameter and returns a boolean, we declare two corresponding variables. For a parameterized test, we need to provide a constructor, which will initialize the variables.

EmailIdValidatorTest.class

1

2

3

4

5

6

7

8

9

10

11

12

...

@RunWith(value=Parameterized.class)

publicclassEmailIdValidatorTest{

privateStringemailId;

privatebooleanexpected;

publicEmailIdValidatorTest(StringemailId,booleanexpected){

this.emailId=emailId;

this.expected=expected;

}

...

We also need to provide a public static method annotated with @Parameters annotation. This method will be used by the test runner to feed data into our tests.

The @Parameters annotated method above returns a collection of test data elements (which in turn are stored in an array). Test data elements are the different variations of the data, including the input as well as expected output needed by the test. The number of test data elements in each array must be the same with the number of parameters we declared in the constructor.

When the test runs, the runner instantiates the test class once for each set of parameters, passing the parameters to the constructor that we wrote. The constructor then initializes the instance variables we declared.

Notice the optional name attribute we wrote in the @Parameters annotation to identify the parameters being used in the test run. This attribute contains placeholders that are replaced at run time.

JUnit Theories

In a parameterized test, the test data elements are statically defined and you as the programmer are responsible for figuring out what data is needed for a particular range of tests. At times, you will likely want to make tests more generalized. Say, instead of testing for specific values, you might require to test for some wider range of acceptable input values. For this scenario, JUnit provides theories.

A theory is a special test method that a special JUnit runner (Theories) executes. To use the runner, annotate your test class with the @RunWith(Theories.class) annotation. The Theories runner executes a theory against several data inputs called data points. A theory is annotated with @Theory, but unlike normal @Test methods, a @Theory method has parameters. In order to fill these parameters with values, the Theories runner uses values of the data points having the same type.

There are two types of data points. You use them through the following two annotations:

@DataPoint: Annotates a field or method as a single data point. The value of the field or that the method returns will be used as a potential parameter for theories having the same type.

@DataPoints: Annotates an array or iterable-type field or method as a full array of data points. The values in the array or iterable will be used as potential parameters for theories having the same type. Use this annotation to avoid single data point fields cluttering your code.

Note: All data point fields and methods must be declared as public and static.

1

2

3

4

5

6

7

8

9

...

@DataPoint

publicstaticStringname="mary";

@DataPoints

publicstaticString[]names(){

returnnewString[]{"first","second","abc","123"};

}

...

In the code example above, we annotated a String field with the @DataPoint annotation and a names() method that returns a String[] with the @DataPoints annotation.

Creating a JUnit Theory

Recall the createEmailID() method that we wrote earlier on this post – “The createEmailID() method accepts two String parameters and generates an email ID in a specific format.” A test theory that we can establish is “Provided stringA and stringB passed to createEmailID() are non-null, it will return an email ID containing both stringA and stringB ”. This is how we can represent the theory.

The testCreateEmailID() theory we wrote accepts two String parameters. At run time, the Theories runner will call testCreateEmailID() passing every possible combination of the data points we defined of type String. For example (mary,mary), (mary,first), (mary,second), and so on.

Assumptions

It’s very common for theories NOT to be valid for certain cases. You can exclude these from a test using assumptions, which basically means “don’t run this test if these conditions don’t apply“. In our theory, an assumption is that the parameters passed to the createEmailID() method under test are non-null values.

If an assumption fails, the data point is silently ignored. Programmatically, we add assumptions to theories through one of the many methods of the Assume class.
Here is our modified theory with assumptions.

In the code above, we used assumeNotNull because we assume that the parameters passed to createEmailID() are non-null values. Therefore, even if a null data point exists and the test runner passes it to our theory, the assumption will fail and the data point will be ignored.
The two assumeThat we wrote together performs exactly the same function as assumeNotNull. I have included them only for demonstrating the usage of assumeThat, which you can see is very similar to assertThat we covered in the earlier post.

The following is the complete code using a theory to test the createEmailID() method.

EmailIDCreatorTest.java

Java

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

packageguru.springframework.unittest.parameterized;

importorg.junit.Test;

importorg.junit.experimental.theories.DataPoint;

importorg.junit.experimental.theories.DataPoints;

importorg.junit.experimental.theories.Theories;

importorg.junit.experimental.theories.Theory;

importorg.junit.runner.RunWith;

importorg.junit.runners.Parameterized;

importjava.util.Arrays;

importstaticorg.hamcrest.CoreMatchers.*;

importstaticorg.hamcrest.CoreMatchers.containsString;

importstaticorg.junit.Assert.*;

importstaticorg.junit.Assume.assumeNotNull;

importstaticorg.junit.Assume.assumeThat;

@RunWith(Theories.class)

publicclassEmailIDCreatorTest{

@DataPoints

publicstaticString[]names(){

returnnewString[]{"first","second","abc","123",null};

}

@DataPoint

publicstaticStringname="mary";

/*Generated Email ID returned by EmailIdUtility.createEmailID must contain first part and second part passed to it*/

In the test class above, I have included null as a data point in the return statement of Line 23 for our assumptions and couple of System.out.println() statements to trace how parameters are passed to theories at run time.

Here is the output of the test in IntelliJ:

Also, here is the output I got while running the test with Maven for your review:

In the output above, notice that whenever a null value is being passed to the theory, the remaining part of the theory after assumeNotNull does not execute.

Summary

Parameterized tests in JUnit helps remove boiler plate test code and that saves time while writing test code. This is particularly useful during Enterprise Application Development with the Spring Framework. However, a common complaint is that when a parameterized test fails it’s very hard to see the parameters which caused it to fail. By properly naming the @Parameters annotation and great unit testing support that modern IDEs provide, such complains are quickly failing to hold grounds. Although theories are less commonly used, they are powerful instruments in any programmers test toolkit. Theories not only makes your tests more expressive but you will see how your test data becomes more independent of the code you’re testing. This will improve the quality of your code, since you’re more likely to hit edge cases, which you may have previously overlooked.