In the usual case, where the array elements are in fact exactly equal, the potentially expensive reflection-based loop to compare them is avoided by using Arrays.deepEquals() first. The exact comparison is only executed when deepEquals() returns false.

Command-line options

When running JUnit from the command line, a command-line parameter can be supplied using --filter, which supplies a filter that will restrict which tests and subtests from the rest of the command will be run. For example, this will run only the tests in ExampleTestSuite that are in categories Cat1 or Cat2:

In general, the argument to --filter should be ClassName=param, where ClassName names an implementation of FilterFactory, whose createFilter method will be called with an instance of FilterFactoryParams that contains "param", in order to return the filter to be applied.

Test Runners

Pull request #763: Allow custom test runners to create their own TestClasses and customize the scanning of annotations.

This introduces some extension points to ParentRunner to allow subclasses to control creation
of the TestClass instance and to scan for annotations.

The AnnotatedBuilder is a strategy for constructing runners for test classes that have been annotated with the @RunWith annotation. All tests within such a class will be executed using the runner that was specified within the annotation.

Prior to JUnit 4.12, this covered only the tests within the annotated test class. With 4.12, the AnnotationBuilder will also support inner member classes. If a custom test runner supports inner member classes (which JUnit does not support out-of-the-box), the member classes will inherit the runner from the enclosing class, e.g.:

When running tests from multiple threads, JUnit will now call RunListener methods from multiple threads if the listener class is annotated with @RunListener.ThreadSafe. In addition, the code in RunNotifier has been modified to not use locks.

This allows for validation to be added to annotations. Validators should extend AnnotationValidator and be attached to annotations with the @ValidateWith annotation. CategoryValidator extends AnnotationValidator and ensures that incompatible annotations (@BeforeClass, @AfterClass, @Before, @After) are not used in conjunction with @Category.

Exception Testing

ExpectedException didn't handle AssertionErrors and AssumptionViolatedException well. This has been fixed. The new documentation explains the usage of ExpectedException for testing these exceptions. The two methods handleAssertionErrors() and handleAssumptionViolatedExceptions() are not needed anymore. If you have used them, just remove it and read ExpectedException's documentation.

In JUnit 4.11 and earlier, if you wanted to write a custom runner that handled
AssumptionViolatedException or you needed to create an instance of AssumptionViolatedException
directly, you needed to import an internal class (org.junit.internal.AssumptionViolatedException).
Now you can import org.junit.AssumptionViolatedException (which extends
org.junit.internal.AssumptionViolatedException).

The classes in Assume have been modified to throw org.junit.AssumptionViolatedException.

The constructors in the external AssumptionViolatedException are also
simpler than the ones in the internal version. That being said,
it's recommended that you create AssumptionViolatedException via the methods in Assume.

Pull request #985: Change AssumptionViolatedException to not set the cause to null; fixes issue #494

Previously, the AssumptionViolatedException constructors would explicitly set the cause to null
(unless you use a constructor where you provide a Throwable, in which case it would set that as
the cause). This prevented code directly creating the exception from setting a cause.

With this change, the cause is only set if you pass in a Throwable.

It's recommended that you create AssumptionViolatedException via the methods in Assume.

The method ErrorCollector.checkSucceeds() is now generic. Previously, you could only pass
in a Callable<Object> and it returned Object. You can now pass any Callable and the
return type will match the type of the callable.

Timeout exceptions try to determine if there is a child thread causing the problem, and if so its stack trace is included in the exception in addition to the one of the main thread. This feature must be enabled with the timeout rule by creating it through the new Timeout.builder() method:

Timeout deprecated the old constructor Timeout(int millis).
A new constructor is available: Timeout(long timeout, TimeUnit unit). It enables you to use different granularities of time units like NANOSECONDS, MICROSECONDS, MILLISECONDS, and SECONDS. Examples:

Each test is run in a new daemon thread. If the specified timeout elapses before the test completes, its execution is interrupted via Thread#interrupt(). This happens in interruptable I/O (operations throwing java.io.InterruptedIOException and java.nio.channels.ClosedByInterruptException), locks (package java.util.concurrent) and methods in java.lang.Object and java.lang.Thread throwing java.lang.InterruptedException.

A specified timeout of 0 will be interpreted as not set, however tests still launch from separate threads. This can be useful for disabling timeouts in environments where they are dynamically set based on some property.

Parameterized Tests

Pull request #702: Support more return types for the @Parameters method of the Parameterized runner

The return types Iterator<? extends Object>, Object[] and Object[][] are now supported on methods annotated with @Parameters. You don't have to wrap arrays with Iterables and single parameters with Object arrays.

Rules

The Stopwatch Rule notifies one of its own protected methods of the time spent by a test. Override them to get the time in nanoseconds. For example, this class will keep logging the time spent by each passed, failed, skipped, and finished test:

JUnit 4.11 introduced restrictions requiring @Rule members to be non-static and @ClassRule members to be static. These restrictions have been relaxed slightly, in that a static member annotated with both @Rule and @ClassRule is now considered valid. This means a single rule may be used to perform actions both before/after a class (e.g. setup/tear down an external resource) and between tests (e.g. reset the external resource), without the need for any workarounds mentioned in issue #793.

Note that a non-static @ClassRule annotated member is still considered invalid, even if annotated with @Rule.

The DisableOnDebug rule allows users to disable other rules when the JVM is launched in debug mode. Prior to this feature the common approach to disable rules that make debugging difficult was to comment them out and remember to revert the change. When using this feature users no longer have to modify their test code nor do they need to remember to revert changes.

This rule is particularly useful in combination with the Timeout rule.

Pull request #974: Updated TemporaryFolder.newFolder() to give an error message if a path contains a slash.

If you call TemporaryFolder.newFolder("foo/bar") in JUnit 4.10 the method returns a File object for the new folder but actually fails to create it. That is contrary to the expected behaviour of the method which is to actually create the folder. In JUnit 4.11 the same call throws an exception. Nowhere in the documentation does it explain that the String(s) passed to that method can only be single path components.

With this fix, folder names are validated to contain single path name. If the folder name consists of multiple path names, an exception is thrown stating that usage of multiple path components in a string containing folder name is disallowed.

Methods annotated with @Rule can now return either a TestRule (or subclass) or a
MethodRule (or subclass).

Prior to this change, all public methods annotated with @Rule were called, but the
return value was ignored if it could not be assigned to a TestRule. After this change,
the method is only called if the return type could be assigned to TestRule or
MethodRule. For methods annotated with @Rule that return other values, see the notes
for pull request #1020.

Pull request #1020: Added validation that @ClassRule should only be implementation of TestRule.

Prior to this change, fields annotated with @ClassRule that did not have a type of TestRule
(or a class that implements TestRule) were ignored. With this change, the test will fail
with a validation error.

Prior to this change, methods annotated with @ClassRule that did specify a return type
of TestRule(or a class that implements TestRule) were ignored. With this change, the test
will fail with a validation error.

Theories

Up until JUnit 4.11 a @DataPoints-annotated array field could contain null values, but the array returned by a @DataPoints-annotated method could not. This asymmetry has been resolved: both can now provide a null data point.

The Theories runner now disallows Theory methods with parameters that have "unresolved" generic type parameters (e.g. List<T> where T is a type variable). It is exceedingly difficult for the DataPoint(s) scraper or other ParameterSuppliers to correctly decide values that can legitimately be assigned to such parameters in a type-safe way, so JUnit now disallows them altogether. Theory parameters such as List<String> and Iterable<? extends Number> are still allowed.

The machinery to perform this validation was in the code base for some time, but not used. It now is used.

Theory failure messages previously were of the form: ParameterizedAssertionError: theoryTest(badDatapoint, allValues[1], otherVar), where allValues, badDatapoint and otherVar were the variables the datapoints was sourced from. These messages are now of the form:

If, in a theory, all parameters were "assumed" away, the Theories runner would properly fail, informing you that no parameters were found to actually test something. However, if you had another method in that same class, that was not a theory (annotated with @Test only,) you could not use Assume in that test. Now, the Theories runner will verify the method is annotated with @Theory before failing due to no parameters being found.

Previously if a data points array field or method was non-static or non-public it would be silently ignored and the data points not used. Now the Theories runner verifies that all @DataPoint or @DataPoints annotated fields or methods in classes are both public and static, and such classes will fail to run with InitializationErrors if they are not.

@DataPoints fields or methods can now be given (one or more) names in the annotation, and @Theory method parameters can be annotated with @FromDataPoints(name), to limit the data points considered for that parameter to only the data points with that name:

@DataPointspublicstaticString[] unnamed =newString[] { ... };
@DataPoints("regexes")
publicstaticString[] regexStrings =newString[] { ... };
@DataPoints({"forMatching", "alphanumeric"})
publicstaticString[] testStrings =newString[] { ... };
@Theorypublicvoid stringTheory(String param) {
// This will be called with every value in 'regexStrings',// 'testStrings' and 'unnamed'.
}
@Theorypublicvoid regexTheory(@FromDataPoints("regexes") String regex,
@FromDataPoints("forMatching") String value) {
// This will be called with only the values in 'regexStrings' as // regex, only the values in 'testStrings' as value, and none // of the values in 'unnamed'.
}

Any theory method parameters with boolean or enum types that cannot be supplied with values by any other sources will be automatically supplied with default values: true and false, or every value of the given enum. If other explicitly defined values are available (e.g. from a specified ParameterSupplier or some DataPoints method in the theory class), only those explicitly defined values will be used.

Validity of DataPoints for theory parameters for all field data points and multi-valued method data points (i.e. not single-valued method data points) is now done on runtime type, not field/method return type (previously this was the case for multi-valued array methods only).

Validity of DataPoints for theory parameters for all data points now correctly handles boxing and unboxing for primitive and wrapper types; e.g. int values will be considered for theory parameters that are Integer assignable, and vice versa.

Previously @DataPoint(s) methods that threw exceptions were quietly ignored and if another DataPoint source was available then those values alone were used, leaving the theory passing using only a subset of the (presumably) intended input values. Now, any data point method failures during invocation of a theory will cause the theory being tested to fail immediately.

This is a non-backward-compatible change, and could potentially break theory tests that depended on failing methods. If that was desired behavior, then the expected exceptions can instead be specifically ignored using the new ignoredExceptions array attribute on @DataPoint and @DataPoints methods. Adding an exception to this ignoredExceptions array will stop theory methods from failing if the given exception, or subclasses of it, are thrown in the annotated method. This attribute has no effect on data point fields.

Previously, when building sets of data points for theory parameters, the only valid multi-valued @DataPoints types were arrays. This has now been extended to also take parameters from Iterable@DataPoints methods and fields.

Categories

@interface Category now is annotated with @Inherited itself. This enables inheritance of categories from ancestors (e.g. abstract test-classes). Note that you are able to "overwrite" @Category on inheritors and that this has no effect on method-level categories (see @Inherited).

From a given set of test classes, the Categories runner runs only the classes and methods
that are annotated with either the category given with the @IncludeCategory annotation, or a subtype of that category. Either classes or interfaces can be used as categories. Subtyping works, so if you say @IncludeCategory(SuperClass.class), a test marked @Category({SubClass.class}) will be run.

You can also exclude categories by using the @ExcludeCategory annotation; see SlowTestSuiteWithoutFast.

The suite FastOrSmokeTestSuite is an example to run multiple categories.

To execute tests which match all categories, use matchAny = false in annotations. See FastAndSmokeTestSuite.

How to install Maven

If you are in the project root, extract the archive (tar xvzf apache-maven-3.0.4-bin.tar.gz).
Create directory .m2 in your user home. Then the artifacts and plugins are stored in ~/.m2/repository.
( ~ stands for user home)

How to launch the build from the command line

Clone the project (git clone https://github.com/junit-team/junit4.git) and navigate to the project root on your local system (cd junit).
Clean the previous build in target directory, build the project, and install new artifacts in your local repository:

apache-maven-3.0.4/bin/mvn clean install

On Windows type the command apache-maven-3.0.4\bin\mvn clean install.

Set the environment variables M2_HOME and PATH when frequently building via command line mvn clean install.

How to install and build the Maven project in Eclipse

I made a clone of JUnit project from GitHub to local folder C:\cygwin\usr\local\etc\junit.

In menu go to File -> Import...

In the popup menu open section Maven, click on Existing Maven Projects and click on Next. In Import Maven Projects specify the project root, and next proceed further with installing maven support plugin in Eclipse.

You have created the Maven project, and now build the project.

In the Package Explorer click on pom.xml. In the menu Run -> Run As -> 2 Maven build... open the popup Edit Configuration and specify the build phase clean install in section Goals. Click on Run and build the project.

How to install and build the Maven project in IntelliJ IDEA

In IDEA menu create a new project File -> New Project....

Select Create Java project from existing sources, then click on Next and specify Project file location.

On the right-hand side is the Maven Projects tab. Click on + and add pom.xml into the project. Then click on the icon Maven Settings, and set Maven home directory as the location of extracted Maven archive on your system. Click on the green triangle and launch the build.

Miscellaneous

Travis CI is a free CI server for public Github repositories. Every pull request is run by Travis CI and Github's web interface shows the CI result for each pull request. Every user can use Travis CI for testing her branches, too.

The time it takes to start a thread can be surprisingly large.
Especially in virtualized cloud environments where noisy neighbours.
This change reduces the probability of non-deterministic failures of
tests with timeouts (@Test(timeout=…)) by not beginning the timeout
clock until we have observed the starting of the task thread – the
thread that runs the actual test. This should make tests with small
timeout values more reliable in general, and especially in cloud CI
environments.

Fixes to issues introduced in JUnit 4.12

The following section lists fixes to problems introduced in the first
release candidates for JUnit 4.12. You can ignore this section if you are
trying to understand the changes between 4.11 and 4.12.

In order to make the JUnit code more consistent with current coding practices, we changed
a number of field names to not start with the prefix "f". Unfortunately, at least one IDE
referenced a private field via reflection. This change reverts the field names for fields
known to be read via reflection.

In 745ca05
we removed classes that were deprecated for many releases. There was some concern that people
might not expect classes to be removed in a 4.x release. Even though we are not aware of any
problems from the deletion, we decided to add them back.

JUnit 4.12 RC1 introduced serilization incompatibilities with some of the classes. For example,
these pre-release versions of JUnit could not read instances of Result that were serialized
in JUnit 4.11 and earlier. These changes fix that problem.