Latest revision as of 11:37, 24 August 2010

Eclipse Quality means extensible frameworks and exemplary tools
developed in an open, inclusive, and predictable process involving the entire
community. From the "consumption perspective," Eclipse Quality means
good for users (exemplary tools - cool/compelling to use, indicative of what is
possible) and ready for plug-in developers (deliver usable building blocks -
with APIs). From the "creation perspective," Eclipse Quality means
working with a transparent and open process, open and welcoming to participation
from technical leaders, regardless of affiliation.

Eclipse Quality is not a binary statement. Rather, quality is a
spectrum and Eclipse projects are expected to, release-by-release, improve their
quality. Eclipse Quality is an evolution over the life of the project and the
evaluation of a project's quality must be appropriate to the project's maturity
level. For example, a project in the Incubation Phase will
have more Provisional APIs and fewer Platform APIs than a project
in the Mature Phase. Similarly, a 1.0 Release
will have fewer API clients than a 2.0 Release; and a 3.0 Release will have a
better release-to-release migration plan than a 2.0 Release; etc.

Note the Eclipse Quality is about both extensible frameworks and exemplary tools - great tools are important for attracting the users,
who then attract the ecosystem, that then provide members, who then contribute
resources, who then create additional valuable frameworks and tools. Neither
frameworks without users nor tools without frameworks are interesting points
along the software development spectrum.

GEclipse Quality goals:

Platform quality frameworks. (note that "Platform quality" here
refers to high quality APIs [1]
(such as a module API in NetBeans [2] or a Java platform API [3] or the Windows
API [4]) and not to the Eclipse Platform project).

Platform quality tools.1

All exemplary tools are built on platform APIs.

Performance and scalability to enterprise class use.

Automated tests and a quality plan that can be used by any Eclipse
developer.
We want any Eclipse user to be able to run all the tests and verify that
they have a working and correct installation of the frameworks and tools.

A release-to-release migration plan potentially including automated tools
for conversions. Release-to-release migration includes not only the APIs,
but also the artifacts generated by the tools (e.g., configuration files,
persisted tables, etc).

API stability.

Predictable behavior and a predictable rate of change.

Demonstrated community involvement.

RIP due diligence as
per the Eclipse IP Policy and other Eclipse processes [5],[6], [7], [8].
(This is the only required Quality element, the rest are goals.)

APIs, Provisional or Not

In an earlier draft document, Eclipse Quality APIs [9], a platform API was defined to have:

A specification. A description of the cover story and the necessary
details. Specifications should be clear about what is defined and what is
not defined. Furthermore, specifications should indicate what areas are
likely to change in the future (e.g., this two-valued parameter will
probably have N-values in the future). A specification is a difficult
document to write, and it is more than just a paragraph or two about the
component.

A test suite.

An implementation.

One or more clients. Usually, just having “a client” is not
sufficient – an Eclipse quality API client is ideally developed by a
separate team. A client (API user) that is written by the same team that
also creates the API implementation usually suffers from information leakage
and thus is not a good test of the API. Even better is two or more clients,
each developed by a separate team, all of whom communicate only through the
API documents.

A support promise. An implicit or explicit promise about how stable
the API will be from release to release.

Note that the API (Application
Programming Interface) is the entire public programming surface of the
framework, thus it includes not only Java classes, packages, and interfaces, but
also extension points, file formats, generated meta-data, and even objects that
are passed through to internal packages.

The question arises of what to do about code that is intended to become API
in a future release, but is not up to these standards in the current release. We
can consider the following cases:

Specification

Test Suite

Implementation

Clients

Support Promise

Package

RPlatform API

yes

yes

yes

yes

yes

public

Provisional

yes

yes

yes

yes

not quite

public

Incomplete

incomplete

incomplete

yes

yes

yes

public

Experimental

yes

yes

yes

yes

?

Gprovisional

Non-API

-

-

yes

-

none

internal

Provisional, Incomplete, and Experiment APIs are collectively referred to as Transient
APIs. The interesting cases are Provisional and Incomplete. Under
this definition, Provisional API is real API in all senses (good specification,
good Javadoc, good tests, etc) but the project does not feel that it has had
sufficient community feedback to completely freeze the APIs yet. The TPTP
project uses Provisional APIs for all new API introductions: the new API is
released as provisional in release X and then hardened into platform APIs in
release X+1.

Incomplete APIs are those where the project wrote the code before defining
the APIs and thus the documentation and/or test suite is incomplete, but the
project intends to support the API. Incomplete APIs may appear in milestone
releases (although the lack of documentation will make it difficult for the
plug-in developer community to adopt and verify those APIs), but Incomplete APIs
must not appear in releases.

1 Admittedly, there is not a good definition for "platform
quality tools" yet. A future version of this document will try to clarify
this.