PDE/API Tools

API tooling will assist developers in API maintenance by reporting API defects such as binary incompatibilities, incorrect plug-in version numbers, missing or incorrect @since tags, and usage of non-API code between plug-ins. The tooling will be integrated in the Eclipse SDK and will be used in the automated build process. Specifically, the tooling is designed to do the following:

Identify binary compatibility issues between two versions of a software component or product.

Update version numbers for plug-ins (bundles) based on the Eclipse versioning scheme.

Update @since tags for newly added classes, interfaces, methods, etc.

Provide new javadoc tags and code assist to annotate types with special restrictions.

Leverage existing information (in MANIFEST.MF) to define the visibility of packages between bundles.

Present state

API tooling is currently under construction in the PDE project and will be integrated into the Eclipse SDK during milestone 6. We intend to ship API tooling function in the Eclipse 3.4 release and our milestone development timeline corresponds to that of Eclipse 3.4. The API Tooling Milestone Plan can be found here. The API tooling code base graduated from the PDE incubator project into the Eclipse PDE project during milestone 5. The code is in HEAD but the API tooling plug-ins have not yet been added to the automated builds.

Getting Started

User Guide

API tooling is under construction. Here's a link to the bleeding edge user guide to help get you started.

Getting the Source Code

The source code for this implementation is available from the dev.eclipse.org CVS server in the /cvsroot/eclipse respository. You need to check out three projects from HEAD. There is a team project set file to assist you with this.

-Dpluginfiles and -Dcontributors are required to provide contributed javadoc tags. Since our tests do not run as a plugin test, we have no extension registry to search for contributions. As such, we have to create our own registry and populate it with all of the contributors ourselves (the listing of plugin.xml files). It should be noted that the number of plugin.xml files and number of contributors must match (if you specify 3 plugin.xml files you must specify 3 contributors).

The required bundles are used as a pool when resolving required bundles for test bundles in the suite. The .ee file describes the execution environment required by API profiles in the test suite. Our tests require a J2SE-1.5 profile. A sample .ee file has the following contents:

A zip file containing pre-configured ee files is available here. Each of the ee files in the archive are designed to be placed in the install directory of the corresponding VM.

For example the sun-1-6u3.ee file should be placed in the directory where the sun 1.6.0 update 3 VM is installed:

C:
/VMs
/sun16u3
/jre
/bin
...
sun-1-6u3.ee

The tests can be run individually, or you can run them all from the APIToolsTestSuite class.

Bugs and Enhancement Requests

The API tooling project uses Bugzilla for tracking bugs and enhancement requests. Active API tooling bugs can be found with this query. All bugs should be filed in with the Eclipse project, PDE product, under the API Tools component.

Use Cases

These are the use cases API tooling is designed to support.

Binary Compatibility Reporting (Batch Mode)

Two versions of the same API profile are compared for binary compatibility. An xml file is produced summarizing any incompatibilities. The comparison tool can be invoked from the command line as a stand alone Java application specifying the profiles to compare and which parts of the profiles to consider (for example, only compare portions of the profile that are deemed to be API).

The report includes errors regarding component version identifiers that have not been incremented properly. As well, if source code is available for the "newer" API profile, the report includes missing @since Javadoc tags.

An exclude list should be added to filter out the cases where the binary incompatibility is "under control", i.e. approved by the PMC.
The best way to maintain the exclude list would be to have a javadoc tag in the source code that mentions why this is a breakage. Something like:
@breakage-addition ......
@breakage-remove Type#member .....

The removals would be located on the parent of the removed member.

Updating the source code improves the traceability of a breakage and allows readers of the source code to get a better picture without the need to check another document.

Binary Compatibility Reporting (IDE Mode)

Workspace projects are compared to a baseline API profile for binary compatibility. Incompatibilities are flagged in source files using markers (that also appear in the Problems view). The user configures the severity of errors produced. A set of external API profiles are managed in the workspace and one is specfied as the baseline against which workspace projects are compared. The user defines the workspace API profile as combination of workspace projects and external API components.

Compatibility errors are updated incrementally or in full depending on the build setting in the workspace (i.e. auto-build vs. manual/full build). Error markers are also produced for components with incorrect version numbers and missing @since Javadoc tags. Quick fixes are available to address the problems.

An exclude list should be added to filter out the cases where the binary incompatibility is "under control", i.e. approved by the PMC.
The best way to maintain the exclude list would be to have a javadoc tag in the source code that mentions why this is a breakage. Something like:
@breakage-addition ......
@breakage-remove Type#member .....

The removals would be located on the parent of the removed member.

Updating the source code improves the traceability of a breakage and allows readers of the source code to get a better picture without the need to check another document.

API Usage Reporting (Batch Mode)

The most common API usage report locates illegal use of APIs among components in a single API profile - i.e. access to non-API types, methods, and fields; and illegal extension, implementing, or instantiating. The API usage scanner can be invoked as a stand alone Java application to examine all or specific portions of an API profile for illegal API use. An xml file is produced as output.

The API scanner should also support scanning for use of a specific component. For example, rather than scanning component X to determine what use it makes of other APIs, scan a profile to find all uses of the API in X.

Another interesting scan would be to report what parts of a profile or component would be broken when migrating to another version of a required component. For example, the internals of a component often change or can be removed in a newer release of the component.

API Usage Reporting (IDE Mode)

The Eclipse SDK already provides compiler warnings for discouraged accesses between bundles - which is the same as referencing non-API code. Rather than duplicate this effort, the integrated tooling could just report illegal implementing, subclassing, and instantiation. Problem markers would be created incrementally, similar to the support for binary compatibility.

API Usage Searching (IDE Mode)

Similar to the extensive search facility provided by JDT for searching projects in the workspace, API tooling could support searching of API profiles. This would allow to search for all uses of a component, type, method, etc., from an API profile or component.

Version Management

In addition to reporting missing @since tags and incorrect bundle version numbers (based on the Eclipse bundle versionion scheme), the tooling will provide quick fixes to correct these problems.

As well, the tooling will assist developers on determining compatible version ranges of required bundles (plug-ins). Developers often increment the lower bound of version ranges of required bundles in each major release. Usually this makes sense (for example, the debug platform's UI bundle usually requires the latest debug core bundle). However, sometimes this is uneccessary, and a bundle may run perfectly fine with an older version of a required bundle. Given a range of versions of a required bundle, API tooling will be able to determine which versions of the bundle satisfy API (and non-API) accesses.

Building API Components & Profies

Provide a mechanism to export API components. This could be used in a build process or from the IDE.

Extensible Javadoc Tags, API Visibilities & Restrictions

Clients can provide their own javadoc tags via the apiJavadocTags extension point.

Extensible javadoc tags allow clients to specify what kind of Java member the tag will apply to. The Java members supported in extensible javadoc tags are:

Interfaces

Classes / Enums

Methods

Fields

The platform provides a default set of tags. The tags and the Java members they apply to are: