UnitTests

Generic Support for Unit Tests in the NetBeans IDE

Lukáš Jungmann, Marián Petráš

Overview

The main goal of the generic support for unit testing is to provide an API that would allow to easily add support for various specific types of unit test frameworks (JUnit, PHPUnit, J2MEUnit, Ruby tests, …) into various project types (J2SE projects, NB modules, WebApps, C/C++, Ruby, …) into the IDE.

The most important changes from the current state:

support for unit testing is extracted from the core projects infrastructure to a separate module (generic support for unit tests)

it is not required for a project type to provide support for any particular unit test framework

each project type may be extended with a support for one or more unit testing frameworks by external modules

One external module can add support for multiple specific testing frameworks (but will usually add support for only one).

It is not necessary for one external module to add support to all project types – it can add support only to a small number of project types, while another module adds support for the same testing framework to other project types.

Motivation (Problems To Be Solved)

So far, the NetBeans support for unit testing is spread across several
modules as follows:

the JUnit module can create simple JUnit tests in Java projects

– but it does not know how to create J2MEUnit tests for J2ME projects.There is an unofficial (friend) API for this – JUnitPlugin.java

the JUnit module knows how to navigate between a source class

and the corresponding test class
– but it does not know how to navigate between Ruby source code and
the corresponding testsThere is another unofficial (friend) API for this, in a separate
module ("Navigate To Test", directory "gototest")

the JUnit module knows how to parse output that is generated when

JUnit tests are executed via Ant (JUnitAntLogger, JUnitOutputReader)
– but it does not know how to parse and display Ruby unit tests, TestNG tests, Maven tests, …There is no API for this, either official or unofficial;
the respective modules simply copy the code from the JUnit
module and accomodate it according to their needs. Some
80 % of the code remains unchanged, but in several copies.

the JUnit module does not initiate execution of tests (it is handled

by the projects infrastructure and its actions) so it does not know
that tests have been started until it detects some output from Ant's
<junit> task

One of the consequences is that every time the test execution mechanism
needs to be changed (e.g. make Ant-based projects generate XML report
files), the developer who wants this change must make the change not
only in the JUnit module, but also in modules defining web projects,
NetBeans module projects, several sub-types of J2SE projects, J2ME
projects, etc. For each type of project, the change requires a change of
a (typically) quite large build file and possibly also some changes in
the Java code. As more and more project types are supported, this is
becoming a maintenance nightmare.

Goals

These are the expected results of the transition to a generic support for unit tests:

pluggable support for unit testing frameworks

shared actions for creation and running of unit tests and for navigation between tests and the tested code

shared implementation of a test results window with a public API

(this may be, and probably will be, done later)

Architecture

In the description below, the term "testing framework" refers to the generic unit testing framework being described by this document.

Module Projects UI does not know anything about unit testing.
Module Generic Unit Testing Framework provides all the tests-related actions currently provided by module Projects UI. It depends on the projects
Modules providing support for project types may not know anything about unit testing either.

API for queries about test roots for various individual testing frameworks

(e.g. Give me test roots for TestNG tests in this project.)

API for incremental display of test results (in future)

API changes required in the projects infrastructure:

It must be possible to query a project about its type, such that the plugin providing support for a specific testign framework knows whether it should plug into it or not.

SPI of modules providing support for individual unit testing frameworks

provide information about test roots, or even create the test roots (TBD)

Open Issues

The description so far only assumed that all projects are Ant-based. But this was a wrong assumption. So we should either make it more general (such that it works with projects not based not Ant, for example with C/C++ projects) or declare that it is only for Ant-based projects.

Other Related Changes

Change of Shortcuts

Shortcuts for execution of unit tests should be changed as follows:

Ctrl+F6

Executes a unit test corresponding to the currently selected class, using the appropriate unit testing framework. The change is that if the currently selected class represents a unit test, it is executed itself.

Shift+F6

Executes the currently selected class using the project's main execution mechanism, i. e. not using any testing framework.