The Eclipse Foundation is proposing a new Top-Level Project to develop
a Test and Performance Tools Platform: an open development platform supplying
frameworks and services for test and performance tools that are used throughout
the software lifecycle (e.g., testing, tracing/profiling, tuning, logging,
monitoring, analysis, autonomics, administration, etc.)

Intel Corporation has also offered to lead the project and is making a
major resource contribution to the Eclipse Foundation and to the development
of software within the project

Where did the Test & Performance Tools Platform Project come from?

The Test & Performance Tools Platform Project is based on and extends
the Eclipse community's Hyades Project that was created in late 2002.

How is the project managed?

The Test & Performance Tools Platform Project is managed by a Project
Management Committee (PMC). The work of the Top Level Project is organized
into Projects, each with its own Project Lead who is responsible for its
success.

Does the Hyades project still exist as a separate project within the
Test & Performance Tools Platform Project?

The scope of the original Hyades project has been partitioned into four
new projects: The Hyades Core Platform, provides core elements and interoperability,
and three others which leverage that infrastructure and relate to specific
types of activity: Hyades Testing, Hyades Tracing and Profiling, and Hyades
Monitoring. Additional subprojects may be added, and new features and requirements
are being added to each of the four projects. Test & Performance was
chosen as a descriptive name for the top-level project, and its scope leaves
open the possibility of additional projects being created which do not directly
leverage the Hyades Core Platform.

What is the overall vision of the Test and Performance Tools Platform
Project and what are its benefits?

The project provides an open development platform supplying frameworks
and services for test and performance tools that are used throughout the
software lifecycle (e.g., testing, tracing/profiling, tuning, logging, monitoring,
analysis, autonomics, administration, etc.) Included in this platform are
data collection services and user interfaces for tracing, test case execution,
logging and statistical behavior in local or remote execution environments.

In most cases such tools, even if they are sold by the same company, were
developed independently. This means that the interoperability is retro-fitted,
and tends not to operate at a particularly fine-grained level either for
user interface or for the data assets the tools work with (tests, traces
etc.). By providing a common infrastructure upon which tools can be built,
where possible leveraging existing standards, the Test and Performance Project
can drive interoperability into existing tooling, and allow new tooling
to interoperate from inception, significantly enhancing the experience of
users of the tools. In addition, by allowing flexible combinations of asset
type and user interface, the project may offer the possibility of vendors
(or even the project itself) providing brand new types of tools function.

The integration with the Eclipse IDE allows test and performance to be
driven early in the software lifecycle, enhancing the quality of software
being developed inside Eclipse, and as Eclipse expands into Rich Clients
and additional new projects, the Test & Performance project will ensure
these applications can easily be subjected to effective testing and performance
analysis.

Why is this type of activity being pursued in an Open Source project?

In an open source model, the cost of building the common infrastructure
is shared amongst the participants in the project, and the diversity of
intellectual inputs amongst Open Source Committers in open collaboration
with their peers ensures that it is naturally based on Open Standards and
can be best of breed. The focus on a shared infrastructure means the incremental
cost of adding new features or supporting new platforms and environments
is small, so the project itself can be nimble, and vendors leveraging the
framework can accelerate the rate at which they add value to their own tools,
thereby moving the overall Eclipse Ecosystem forward.

Which organizations are offering to be involved in the Test & Performance
Tools Platform Project?

The following organizations are currently committed to participating in
the development of the project:

Intel

IBM

Scapa Technologies

FOKUS

SAP

Compuware

Is this project concerned with test and performance of Eclipse itself,
of applications built with Eclipse, or of systems in general?

The Test and Performance Tools Platform is concerned with the performance
of systems in general, although some special support is provided for systems
built in Eclipse and for the Eclipse workbench itself. For example, the
tests used to validate the Hyades infrastructure and tooling are stored
in the Eclipse.org CVS repository in ways that can be managed and executed
by tools that Hyades itself supplies. At this stage, however, the definition
and testing of Eclipse builds, and certification of plug-ins against those
builds, is not being addressed by the project.

Does this mean that the project tests, monitors, traces systems on which
the Eclipse Workbench is not installed?

Yes. The project leverages core Eclipse subsystems such as SWT for user
interfaces and EMF for data persistence, but there are elements of the Hyades
infrastructure which operate in target environments which can be remote
to the workbench, some of which are written in native code, rather than
Java. User interfaces can be built as workbench clients, rich clients, and
in some cases the infrastructure can operate without a user interface.

Does the project only build infrastructure, or does it provide tools?

As is true in general for Eclipse, the project builds both infrastructure
and Exemplary Exensible Tools. The open source distribution contains tools
that will cover much of the basic activities of tracing, profiling, monitoring
and testing, although this will probably primarily apply to small-scale
systems, open source systems, and development environments rather than production
systems. Individual vendors (both contributing Eclipse members, non-contributing
members and non-members) may provide additional functionality through interfacing
to the infrastructure, extending sample tools etc. They will also tend to
provide support for the combined offering under commercial licensing arrangements.

What platforms and environments does the Test & Performance Tools
Platform
support ?

The user interface components of the Hyades Infrastructure can run on any
Eclipse-supported platform. The various test execution, monitoring and tracing
components typically run on a broad range of infrastructure and interface
with a broad range of systems. One notable deficiency at present is that
the various trace facilities are currently Java-specific, although the project
has a goal to extend additional language support in due course. For the
other agents there is a reasonable coverage implemented or planned for popular
operating systems, application servers, web servers etc, with a particular
focus on providing support for other open source products.

The Test and Performance Tools Platform Project Requirements, Architecture,
and Planning Groups guides our project. Each of these groups are aligned
with their respective Eclipse Foundation Committees.

Requirements are gathered from Eclipse member companies, research organizations,
independent projects and projects hosted by the Eclipse Foundation. Feedback
from these groups and investment in our meritocracy help prioritize implementation
of the new technology in the Test and Performance Tools Platform as we take
a generic view of how the frameworks could be reused. Ultimately, however,
contributing organizations and individuals need to make long-term commitments
to developing, documenting, testing and enhancing elements of the infrastructure,
and new contributors are always welcome.

Can I use the Test & Performance Tools Platform Project distributions
in my commercial activities?

The projects that are hosted by Eclipse make all distributions available
under licenses that are certified by the Open
Software Initiative. Test and Performance Tools Platform Project distributions
are made available under the Eclipse
Public License. It makes the Test and Performance Tools Platform available
for commercial re-distribution without royalty or the obligation to donate
software back to Eclipse. See the license for more detail.

Please tell me more about the projects within the Test & Performance
Tools Platform Top-Level Project?

The Hyades Core Platform project covers the common infrastructure in the
areas of user interface; EMF based data models, data collection and communications
control, as well as remote execution environments. In addition, the platform
provides the extension points for leveraging or extending the common infrastructure
in solution specific tooling or runtime. This includes Eclipse workbench
plugins as well as runtime plugins on a target and optionally remote system.

The Hyades Tracing and Profiling project extends the platform with specific
data collection for Java and distributed applications that populate the
common trace model, additional language and protocol support is anticipated.
There are also viewers and analysis services that draw data from the common
trace model. Capabilities are provided to collect and analyze both heap
and stack information as well as generic toolkits for instrumenting running
applications.

The Hyades Monitoring project collects, analyzes, aggregates and visualizes
data that can be captured in the log and statistical models. The typical
examples are the collection of system or application resources such as CPU
or memory utilization and support for the viewing ,aggregation and analysis
of that data. Logs can also be transformed into the common format and model
allowing for symptom and pattern analysis. The correlation of the data in
these models is of particular interest when it is associated with other
model instances of statistical or log data as well as traces and tests.

You've used the term Data Models. I'm not familiar with that term in
this context?

The Data Models are abstract descriptions in UML of the types of assets
(tests, traces, logs etc.) that the project deals with, and they are provided
with a concrete implementation through the Eclipse Modeling Framework (EMF).
This means that tools inside the Eclipse Workbench can manipulate objects
which correspond to those abstract descriptions, which are managed and persisted
inside the eclipse asset model and can be stored in projects, subjected
to version control, etc. When they actually hit a filesystem they are in
an open XMI format, but individual tools vendors don't have to go through
the effort of manually constructing code to map that XMI into and out of
an object format they can usefully manipulate, they merely speak to the
EMF-generated API. Furthermore if two tools both speak to that API they
can interoperate through shared use of the stored assets inside the asset
management model provided by Eclipse.

Are these data models in any way standard?

The Test data model is an implementation of the UML 2 Test Profile (U2TP)
defined by the Object Management Group. The Log Model is derived directly
from a standard currently proposed at OASIS known as Common Base Event.
The Trace Model is ad-hoc but maps quite closely to the data formats of
JVMPI and JVMTI. The statistical model is also ad-hoc, but maps well onto
the formats used by JMX and the Microsoft PerfMon counters.

Where can I learn more about the Test & Performance Tools Platform
Project (white papers, etc.)?

The Test and Performance Top-Level Project Proposal and associated documents
are at /proposals/eclipse-tptp/index.html
The Hyades Project Website is at /Hyades
until it merges into the proposed new structure.