SignatureTest

Signature tests verify that public and protected members of non-private classes in each module do not change or remain compatible with previous version. These tests are part of the build or the continuous build process. The testing infrastructure is currently based on APITest, an extension and temporary fork of sigtest tool which is used as part of the JCP test compatibility kits to verify that the API exposed by a JSR implementation does not diverge from the reference implementation.

Use from Maven

Consistency Check

Turned on Nov 7, 2012:

For a while the VersioningPolicy prescribes how to properly version modules and their API changes. However the policy has for a long time only been applied to modules with stable API. From time to time this leads to no such method or class errors like Issue 186233 which need to be evaluated by platform/module system component owner.

Moreover the introduction of incremental binary build AutoUpdateTask requires more proper attitude to versioning. Only then one can be sure that incremental binary build really links properly.

For the above reasons ita consistency check that verifies minimal versioning consistency has been introduced. The HgParallelProjectIntegration runs:

Check on all modules exposing non-private packages

Compares the spec version.

If the new spec is the same as previous, the check fails if there is some API change (method or class)

If the new spec is higher, the check will pass regardless of any API changes.

The check is integral part of push-* jobs and if it detects a failure, the push job fails. To recover you are supposed to increment spec version of your module whenever you change its API. Increment spec versions of modules that depend on your API change as well (not enforced right now). Run following command to simulate the check locally:

# first of all make sure your system is built OK# Then either:# generate appropriate JUnit testing report for all modules:
$ ant check-sigtests-version
## or run the 'version' check for a single module
$ ant -f your.module/build.xml check-sigtest -Dsigtest.check.type=versioncheck

Continuous Check for Changes

The continuous build process runs a sigtest build that compares APIs in a new build with APIs of the one of the previous build. Whenever there is a change (either compatible or incompatible) a mail is sent:

A module can configure sigtest behavior by modifying its project.properties:

# redefine the email where reports are sent
sigtest.mail=your.name@your.org
# don't fail if the generation of the report fails
sigtest.gen.fail.on.error=false

At the time of release

When doing a release it is important to record the actual signatures of stable modules (and to some extend also under development ones), so we know what to keep backward compatibility with. To generate these signatures do:

Check for Incompatible Changes Since Previous Version

There is another check for API signature compliance: A check that the APIs are compatible with a previous released version. This check is available to every module, which can make it part of its build by generating its API snapshot by:

ant netbeans gen-sigtest

This creates a file nbproject/org-netbeans-module-name.sig which can be checked into the version control system as a reference point. This check is part of continuous build as well: every module with its golden file in nbproject/org-netbeans-module-name.sig is checked for backward compatibility and failures are included in continuous build test results. To run the same test for one module on local computer just use:

$ ant netbeans check-sigtest

in your module directory and API of your module will be verified. Of course, only if the signature file exists.

Who needs the Golden Signature File

Not every module with an API needs this golden file; currently only those modules that produced a stable API in release 6.1 have it. The plan is to update these golden files with a new version as soon as it is released, so future checks are done against latest released version.

If an incompatible change is made (discouraged and subject to API review process) the golden file needs to be edited by hand to remove the element that is no longer present in the API.

Checking for all changes from previous release

In case you have the sig file in nbproject directory generated at the time of previour release (all public APIs have it), you can see all changes since that time by:

apitest tool

This check for incompatibilities is done by NetBeans' own version of an OpenJDK tools. This adds a standard Ant task to check for binary backward compatibility, mutual signature compatibility. The binary compatibility check ignores generic types. Its sources are distributed together with its Mercurial repository and are available under GPL version 2. Builds are available from our hudson builder.