Using a Java benchmark to break bottlenecks

[PROGRAMMERS REPORT -Sept 24, 2002] - The Standard Performance Evaluation Corp. (SPEC) has released
SPECjAppServer2001, a benchmark for measuring the performance of Java 2
Enterprise Edition (J2EE) application servers connected to databases. The
benchmark enables users to test and compare hardware and Java software
platforms, and to check the performance of their Java server applications.

The benchmark focuses on testing the EJB container in a J2EE 1.2 server,
using a workload that models a supply-chain, order/inventory and manufacturing
environment.

The SPEC benchmark is derived from the ECperf 1.1 (JSR-131) developed as part
of the Java Community Process (JCP). The group, which has a long history in
building and maintaining hardware benchmarks, has made some changes to the
original Java server benchmark to keep it in line with its general benchmark
policies.

SPEC's new stewardship of a Java standard should hearten those who distrust
vendor-administered benchmarks. Of course, there is a body, which includes many
developers, that looks askance at all benchmarks. Programmers Report asked
Kaivalya Dixit, SPEC president, what such a benchmark means to a developer
today.

'Application developers that use J2EE environments can benefit from this
benchmark,' said Dixit. 'They can improve the performance of their application
by analyzing performance bottlenecks and applying appropriate solutions to their
software.'

Using a benchmark like this can help users to uncover architecture issues
that may not usually be uncovered until an application is deep in the deployment
process. Said Dixit: 'You can find out if your app server software is not fast
enough, or if your database server's response time is too slow.

'A benchmark like SPECjAppServer provides 'a harness' to try different
hardware and software configurations to optimize a solution,' he added.

As for the objectivity of the benchmark, Dixit is clear. 'SPEC's
infrastructure of common code, comprehensive run and reporting rules, review
process and tools provides a level playing field for the benchmark,' he said,
noting that SPEC invites comment on all its benchmark methods.