Gambit-C vs. Bigloo vs. Chicken vs. MzScheme vs. Scheme48

The following tables contain the execution time of the Gambit
benchmarks on various implementations of Scheme in different
situations. For a given benchmark and situation, the entry in green
indicates which Scheme system has the fastest execution and the number
given is the CPU time in milliseconds. Other entries give the execution
time relative to the green entry. Blank entries indicate that this
benchmark was not executed (possibly because the system did not accept
to compile the program, or because a given situation could not
be simulated with that implementation of Scheme).

The benchmarks were run in four different situations.
This was done by using compiler options and/or declarations.

Safe with generic arithmetic and mutable bindings.
This situation approximates the R5RS semantics:

generic arithmetic operations

mutable bindings for the definitions in the benchmark

mutable predefined bindings (for +, car, ...)

safe execution (i.e. an exception must be signalled on errors)

Safe with generic arithmetic.
This situation corresponds to the proposed R6RS semantics:

generic arithmetic operations

overflow detection on fixnum arithmetic (either produce a
bignum or signal an exception)

immutable bindings for the definitions in the benchmark

immutable predefined bindings (for +, car, ...)

safe execution (i.e. an exception must be signalled on errors)

Safe with fixnum/flonum specializations.
This situation corresponds to the proposed R6RS semantics, with the
use of fixnum and flonum specific functions for arithmetic:

arithmetic operations are specialized to fixnum or flonum
arguments as appropriate (for example turning + into
fl+) and the fixnum operations may wrap on overflow

immutable bindings for the definitions in the benchmark

immutable predefined bindings (for +, car, ...)

safe execution (i.e. an exception must be signalled on errors)

Unsafe with fixnum/flonum specializations.
This situation corresponds to the proposed R6RS semantics, with the
use of fixnum and flonum specific functions for arithmetic and the use
of the declaration ``(declare (unsafe))'':

like ``Safe with fixnum/flonum specializations'' but
errors are not checked

The following program options were used for all situations: -h 5000000

Most of the benchmarks take parameters that affect the computation
that is performed (number of iterations, initial arguments, etc). The
benchmarks were carefully written to defeat some compiler
optimizations such as partial or total evaluation of the benchmark at
compile time when the benchmark parameters are known. Our position
is that each benchmark should be thought of as a function exported
from a separately compiled module, and the function is called from
another module with the benchmark parameters (so the compilation of
the benchmark module should not take into account the specific values
that are passed to the function or even the type of value passed).
The benchmark parameters are passed to the function using a call to
``apply''. For the systems tested this seems to achieve
the desired effect (at least for the versions used). It should be
noted that some of the systems could generate faster code if this
approach was not used. In particular, for the Bigloo compiler, if the
types of the benchmark parameters are known, better code can often be
generated. Moreover it is common in Bigloo to specify the type of
parameters on exported functions, so the performance obtained with
this experimental setup may not be representative of the performance
commonly achieved with Bigloo.