Memory Management (Garbage Collection)

The generational nature of the SAP JVM's memory system provides
the flexibility to use specific garbage collection algorithms suited to the
needs of a diverse set of applications. The SAP JVM supports several different
garbage collection algorithms designed to serve different pause time and throughput
requirements.

Background

A major attraction of the Java programming language for programmers
is that it is the first mainstream programming language to provide built-in
automatic memory management, or garbage collection (GC). In traditional languages,
dynamic memory is allocated using an explicit allocate/free model. In practice,
this turns out to be not only a major source of memory leaks, program bugs,
and crashes in programs written in traditional languages, but also a performance
bottleneck and a major impediment to modular, reusable code. (Determining
free points across module boundaries is almost impossible without explicit
and hard-to-understand cooperation between modules.) In the Java programming
language, garbage collection is also an important part of the "safe" execution
semantics required to support the security model.

A garbage collector automatically handles freeing of unused object
memory behind the scenes by reclaiming an object only when it can prove that
the object is no longer accessible to the running program. Automation of this
process completely eliminates not only the memory leaks caused by freeing
too little memory, but also the program crashes and hard-to-find reference
bugs caused by freeing too much memory.

Traditionally, garbage collection has been considered an inefficient
process that impedes performance, relative to an explicit-free model. In fact,
with modern garbage collection technology, performance has improved so much
that overall performance is actually substantially better than that provided
by explicit freeing of objects.

The SAP JVM Garbage Collectors

The SAP JVM comes with a set of different garbage collectors suited
for different runtime scenarios. The major features of this garbage collector
family are presented below. Overall, these capabilities are well-suited both
for applications where the highest possible performance is needed and for
long-running applications where memory leaks and memory inaccessibility due
to fragmentation are highly undesirable.

Accuracy

The SAP JVM garbage collectors are fully accurate collectors. In
contrast, many other garbage collectors are conservative or partially accurate.
While conservative garbage collection can be attractive because it is very
easy to add to a system without garbage collection support, it has certain
drawbacks. In general, conservative garbage collectors are prone to memory
leaks, disallow object migration, and can cause heap fragmentation.

Because the SAP JVM collectors are fully accurate, they can make
several strong design guarantees that a conservative collector cannot make:

An accurate garbage collection mechanism avoids accidental memory
leaks, enables object migration, and provides for full heap compaction. The
GC mechanism in the SAP JVM scales well to very large heaps.

Generational Copying Collection

The SAP JVM employs a state-of-the-art generational copying collector,
which provides two major benefits:

Corresponding decrease in the frequency and duration of user-perceivable
garbage collection pauses

A generational collector takes advantage of the fact that in most
programs, the vast majority of objects (often greater than 95 percent) are
very short lived (for example, they are used as temporary data structures).
By segregating newly created objects into an object nursery, a generational
collector can accomplish several things. First, because new objects are allocated
contiguously in stack-like fashion in the object nursery, allocation becomes
extremely fast, since it merely involves updating a single pointer and performing
a single check for nursery overflow. Secondly, by the time the nursery overflows,
most of the objects in the nursery are already dead, allowing the garbage
collector to simply move the few surviving objects elsewhere, and avoid doing
any reclamation work for dead objects in the nursery.

Parallel Young Generation Collector

The single-threaded copying collector described above, while suitable
for many deployments, could become a bottleneck to scaling in an application
that is otherwise parallelized to take advantage of multiple processors. To
take full advantage of all available CPUs on a multi-processor machine, the
SAP JVM offers an optional multithreaded collector for the young generation,
in which the tracing and copying of live objects is accomplished by multiple
threads working in parallel. The implementation has been carefully tuned to
balance the collection work between all available processors, allowing the
collector to scale up to large numbers of processors. This reduces the pause
times for collecting young space and maximizes garbage collection throughput.

When moving objects, the parallel collector tries to keep related
objects together, resulting in improved memory locality and cache utilization,
and leading to improved mutator performance. This is accomplished by copying
objects in depth first order.

The parallel collector also uses available memory more efficiently.
It does not need to keep a portion of the old object space in reserve to guarantee
space for copying all live objects. Instead, it uses a novel technique to
speculatively attempt to copy objects. If old object space is scarce this
technique allows the collector to switch smoothly to compacting the heap without
the need for holding any space in reserve. This results in better utilization
of the available heap space.

Finally, the parallel collector is able to dynamically adjust its
tunable parameters in response to the application's heap allocation behavior,
leading to improved garbage collection performance over a wide range of applications
and environments.

Mark-Compact Old Object Collector

Although the generational copying collector collects most dead
objects efficiently, longer-living objects still accumulate in the old object
memory area. Occasionally, because of low memory conditions or programmatic
requests, an old object garbage collection must be performed. The SAP JVM
by default uses a standard mark-compact collection algorithm, which traverses
the entire graph of live objects from its roots, then sweeps through memory,
compacting away the gaps left by dead objects. By compacting gaps in the heap,
rather than collecting them into a free-list, memory fragmentation is eliminated,
and old object allocation is streamlined by eliminating free-list searching.

Concurrent Mark and Sweep Collector

To provide support for applications that need low response times
(like dialog-oriented applications, where the users will not accept sporadic
longer response times due to a running “stop-the-world” garbage collection),
the SAP JVM provides a largely concurrent garbage collection, which runs in
parallel to the processed application and minimizes the use of stop-the-world
pauses. These kinds of pauses are only necessary for identifying the initial
set of root objects for object graph traversal and for marking of objects
modified during the (concurrent) garbage collection run.

Since the concurrent mark and sweep collector does not compact
the old generation, it utilizes free-lists for managing the free space. Potentially
more heap memory is necessary, since garbage is created while the collector
is running. The concurrent mark and sweep collector employs heuristics to
minimize the possible fragmentation due to the missing compaction and to start
the garbage collection early enough to reduce the probability of running into
a full old generation.

Parallel Old Generation Collector

In addition, the SAP JVM provides a parallel mark-compact collector
for the old generation designed to improve scalability for applications with
large heaps. Where the concurrent mark-sweep collector focuses on decreasing
pause times, the parallel old collector focuses on increasing throughput by
using many threads simultaneously to collect the old generation during stop-the-world
pauses.