Introduction

In a Java HotSpot virtual
machine, the compilation of bytecode into native code is normally
transparent to the developer. The primary performance criterion
is the average execution time of applications, which translates
to throughput for server-like
systems. Certain VM operations can be allowed to execute
in an unpredictable amount of time if they do not occur often and if
this allows the most common cases to execute faster. The gain on the
average execution time is positive with such a policy. For example,
in the case of compilation, since compiled code executes faster
than interpreted code, the overall program can run faster if frequently
executed pieces of code are compiled during program execution.

However, in a real-time environment, execution must
be deterministic, that is, response times should be guaranteed and
predictable. Variations in response time are generally of more
concern than execution speed.

Determinism can be expressed in terms of jitter, where
jitter measures the variation of response time or execution time.
One of the most common sources of jitter is run-time compilation.
If compilation interrupts the execution of a section of time-critical
code, determinism is seriously compromised.

Another source of unpredictable compilation-related jitter is the
on-demand resolution of symbols (fields, classes and methods). When an
unresolved symbol is encountered during the execution of the code,
the VM uses some extra, non-deterministic, code to resolve the symbol.
In this way the VM avoids resolving symbols that are never actually used.
However, since the timing of the resolution operation itself is
unpredictable, this is unacceptable in time-critical code.

This document will explain how to control these two sources of jitter.

In the Just-In-Time (JIT) compilation mode, compilation
is triggered during the execution of the program, when
certain internal counters reach specified limits. This mode is
designed to be optimal in a non-real-time system.

Since run-time compilation (JIT) cannot be controlled, Java
RTS provides the Initialization Time Compilation (ITC) mode. This mode
allows you to
control when specified classes will be compiled, thereby ensuring
that compilation will not interfere with the execution of critical
sections of code.

The basic principle of JIT compilation in Java RTS is the
same as in the Java HotSpot VM. Compilation of a method is triggered in two
situations:

When an interpreted call is executed, one of the first operations
is to increment the invocation counter for the called method. If
the counter's value reaches the compilation threshold, then the
method is compiled before being executed.

The number of loop iterations in the method is tracked by a
backedge counter. When this counter reaches a threshold value,
compilation is triggered. This process is also known as On-Stack Replacement (OSR).

The time at which the compilation occurs is totally under the control
of the virtual machine and cannot be predicted in 100% of the cases.
For example, if a section of critical code is only executed in rare
conditions, the invocation counter of a method in that code may reach
the threshold after a very long running time. However, there is no way
to know when the compilation will occur, and this uncertainty is not
acceptable in a real-time application.

Note that, if most of the methods of an application are interpreted,
the CPU load can be so heavy that compilation is delayed forever.
In that case, you should compile at least a few methods.

Invocation Counter Overflow:
Asynchronous or Synchronous Compilation

Compilation that is triggered by an overflow of the invocation counter can be
asynchronous or synchronous.
Asynchronous compilation is enabled by default in order to improve determinism. It can be disabled
on the command line with the flag -XX:-BackgroundCompilation.

In asynchronous (or background) compilation, the compilation
of the called method is initiated,
but the thread that initiated the compilation is not blocked
waiting for the compilation to complete; the thread continues executing
the method in
interpreted mode. Compilation continues asynchronously, in the
background. After the compilation is complete,
subsequent invocations of that method will execute the compiled
version.

In synchronous compilation, the thread that initiated the compilation
is blocked until the method is compiled. After the compilation is
complete, the thread executes the compiled method.
This improves throughput earlier in the execution, but the application
pauses during the compilation, and this can impact determinism.

Initialization
Time Compilation

A Java program is not usually deterministic during its entire
execution, that is,
only certain parts of it are deemed time-critical. It is convenient to
refer to a real-time application as running in different phases:

Initialization phase and reconfiguration phase

Time-critical phase

During the initialization and reconfiguration phases, the
application does not need to execute deterministic code, and these are the phases
where operations that potentially have an unpredictable execution time should
be executed.

With the Initialization Time Compilation (ITC) mode, specified methods
of a program are compiled during the initialization of their classes,
that is, in advance of the time they would be
compiled in JIT mode. In this way you can ensure that
compilations of critical code occur during the initialization phase.

Class initialization is non-deterministic, because it is executed at the
first use of the class. Hence class initialization must be executed during the
initialization phase of the application. Therefore, class
initialization is a natural place to spend CPU time compiling critical code.

As a result, if you ensure that all the classes that will be
touched during the time-critical phase are initialized before entering
this time-critical phase, then this also ensures that all the requested
compilations
are performed before entering the time critical-phase. This contributes
to the determinism of the application.

You can control three aspects of ITC:

Specification of the methods to be compiled at class
initialization time

Specification of the classes to be preinitialized

Specification of the classes to be preloaded

These specifications can be performed either on the command line at VM
start-up or programatically during runtime with the
Compiler class in the extended RTSJ package.
The ability to specify these methods and classes during execution is
especially useful in user-defined class-loader-based architectures,
where some methods might be written after the application has started running.

Specifying the
Methods for Initialization Time Compilation

With ITC, you provide the descriptors of the methods to be
precompiled in a list that is used to preload
the classes containing those methods and mark the methods for
compilation when the class is initialized.

This list contains a method description on each line in the following format:

<class name> <method name> <method signature>

The <class name> is the fully qualified name of the class
that defines the selected method, the <method name> is the
name of the method, and the <method signature> is the
signature formatted as specified in the Java Virtual Machine Specification.

Instead of manually creating the precompile list, you can ask Java RTS to do it for you.
See Generation of the Lists for details.

You can specify this list programmatically during runtime with the mark() method
of the
Compiler class.

Specifying the
Classes to be Preinitialized

To guarantee determinism, you must preinitialize the
classes that will be later touched in the time-critical phase.
To do so programmatically, you can use the following call to load and initialize a specified class:

java.lang.Class.forName(<class name>,true,<class loader>);

You can choose to make as many calls to this method as you have time-critical classes.

This scheme has a drawback: the initialization code must be
changed each time the time-critical code is modified in a way that adds a new class.

To facilitate your work, you can ask Java RTS to initialize classes at a specified time.
This is done by providing a file that contains a
list of the names of classes that must be preinitialized.

You can specify the list in two ways:

If you specify preinitialization at VM start-up, provide a file
with a fully qualified class name on each line and indicate this file
with the rtsj.preinit property on the command line,
as in the following example:

java <vm-options> \
-Drtsj.preinit=<preinit-file-name><your-program>

Instead of manually creating the preinit list, you can ask
Java RTS to do it for you.
See Generation of the Lists for details.

You can specify this list programmatically during runtime with the preinit() method
of the Compiler class.

Warning: In some rare cases, the order in which static initializers
are executed may be of some importance, and executing them in an
arbitrary order may generate problems. Even though this is considered
bad programming practice, some legacy code may cause problems of
this kind.

Specifying the
Classes to be Preloaded

Another source of unpredictable compilation
jitter is the on-demand symbol resolution. This is not acceptable
during critical phases of a real-time application. Java RTS solves this
problem
by assuring that necessary symbols are resolved before the
critical code is executed, that is, during the initialization phase.
Two kinds of references are resolved:

References to instance fields and virtual methods.
The referenced classes must be loaded and linked so that the various
offsets are known. This includes all classes that are touched during
compilation.

In order to resolve these references in advance, Java RTS
needs to preload all the classes that are referenced in the code.
This is accomplished by providing a file that contains a
list of these classes. You can provide this file in two ways:

If you specify preloading at VM start-up, provide a file
with a fully qualified class name on each line and indicate this file
with the rtsj.preload property on the command line,
as in the following example:

java <vm-options> \
-Drtsj.preload=<preload-file-name><your-program>

Instead of manually creating the preload list, you can ask
Java RTS to do it for you.
See Generation of the Lists for details.

You can specify this list programmatically during runtime with the preload() method
of the Compiler class.

Note that if you have specified a preinit list, you do not need
to specify a preload list, as preinitialization includes preloading.

Generation of the Lists

Java RTS can help you reduce your work and eliminate
oversights in creating the lists of methods and classes that might be needed at run-time.
You can make a preliminary run of your program and request Java
RTS to generate the precompile, preinit, and preload lists for you.
Then you can use these lists as input to final run of your program.

All these files are stored in the directory that was the current
directory when you launched the VM.

Precompile List.
When you run Java RTS with the option -XX:+RTSJBuildCompilationList,
the VM generates a file containing a list of all the methods that were interpreted or
compiled by a thread whose compilation policy is ITC.
(The default name for this output file is nhrt.precompile,
but you can specify a different name with the CompilationList option.)
This file can then be used as input to the final run, using the
rtsj.precompile property on the command line.

Preinit List.
When you run Java RTS with the option -XX:+RTSJBuildClassInitializationList,
the VM generates a file containing a list of all the classes that were referenced
during the compilation of methods at initialization, in the order in
which they were initialized.
The order might be important, depending on the way the static
initializers were written.
(The default name for the output file is itc.preinit,
but you can specify a different name with the PreInitList option.)
This file can then be used as input
to the final run, using the rtsj.preinit property on the
command line.

Preload List.
When you run Java RTS with the option -XX:+RTSJBuildPreloadList,
the VM generates a file containing a list of all the classes that were referenced
during the compilation of methods at initialization.
(The default name for the output file is itc.preload,
but you can specify a different name with the PreLoadList option.)
This file can then be used as input
to the final run, using the rtsj.preload property on the
command line.

You can also combine both the creation of the lists and the
use
of the lists on the same command line. Let's take the precompile
file as an example. If the file does not exist or is empty, Java RTS
will generate it during the run; you might
experience some jitter in this case, because the frequently-executed
methods were not precompiled. But during the next run,
Java RTS will use the file that was generated in the previous run
in order to precompile the classes containing those methods.
By specifying the file generation and file use in the same command,
you reduce the risk of forgetting a step. Moreover, the generated file
is cumulative: if additional methods are executed in the second run
of an application, these methods are added to the list of methods that were
written to the file during the first run.

Below is an example of combining the creation of a list and the
use of the list in one command, where we use
the default file name for the generated file:

You can safely leave these options on the command line for each
execution: they have an effect only at the shutdown of the
VM and therefore have no impact on the performance of your application.

Warning: In order for the generated lists to be complete, you
must make a set of runs that cover all of the
time-critical code. Statement coverage might not be enough in cases
where the application makes use of the polymorphism of method calls,
which is encouraged by object-oriented design. Branch or more general path
coverage might be necessary, depending on the complexity of the design.
In addition, the -XX:+LogJitterProneEvents
command-line parameter tracks down methods that are executed interpreted;
see the Debugging section.

Verifying the ITC Lists for Completion

One way to verify that the ITC lists contain all the methods that should
be precompiled is to add to the command line the option -XX:+PrintCompilation.
With this option, a message is printed every time a method is JIT-compiled.
The output includes an identifying compilation number, the full method name
(but not the signature), the total size of the bytecode, and a time stamp.
After the compilation number, and before the method name, five character
positions contain additional information about the compilation, where the
symbols have the following meanings:

Position

Symbol and Meaning

1

'%' - OSR compilation.

'*' - Native method call wrapper compilation.

blank - Normal method compilation.

2

's' - The method is synchronized.

blank - The method is not synchronized.

3

'!' - The method has exception handlers.

blank - The method does not have exception handlers.

4

'b' - Blocking, that is, the thread which had the counter overflow
is waiting for the compile to finish before continuing execution.

blank - Not blocking.

5

'R' - Compilation is for a RealtimeThread.

'N' - Compilation is for a NoHeapRealtimeThread.

blank - Compilation is for a non-real-time Java thread.

The following is some example output from the -XX:+PrintCompilation option.

This output shows, for example, that compilation number 106 had exception handlers, was blocking,
and was for a RealtimeThread. Compilation number 113 was wrapping a native method call,
was blocking, and was for a NoHeapRealtimeThread.

Note that each type of thread has its own copy of the compiled code.
For example, RealtimeThreads execute only code compiled for RealtimeThreads,
not code compiled for other types of threads. In the example output above you can see
that the same method was compiled once for NoHeapRealtimeThreads and again for RealtimeThreads.

Compilation Modes for the Thread Types

The different types of threads in a real-time application are instances of java.lang.Thread,
javax.realtime.RealtimeThread, and javax.realtime.NoHeapRealtimeThread.
These thread types have different restrictions to the level of jitter that they can
generate and still respect real-time requirements. In order to balance the trade-off between jitter and throughput,
it is important to consider the compilation modes that are
appropriate for each thread type. The choice of compilation mode
might also depend on whether you use the Serial GC or the RTGC.

NoHeapRealtimeThread

In the case of instances of NoHeapRealtimeThread (NHRT),
only ITC is allowed. In addition, you
should even consider not executing any NHRT code as interpreted,
because the best jitter numbers are obtained with compiled code.

RealtimeThread

For instances of RealtimeThread (RTT), the situation is a bit
more complex than for NoHeapRealtimeThreads.

Determinism is important for RTTs, and therefore ITC is enabled by default.
ITC can be disabled for RTTs with the -XX:-ITCRT flag.

However, RTTs must balance throughput and determinism. Therefore,
JIT compilation is also enabled by default.
JIT can be disabled for RTTs with the -XX:-JITRT flag.

When ITC is enabled for RTTs, the methods in the precompile list are compiled at
class initialization. All other methods are either JIT compiled
if JIT is enabled for RTTs (-XX:+JITRT), or interpreted
if JIT is disabled for RTTs (-XX:-JITRT).

When ITC is disabled for RTTs, RTT instances execute code either JIT compiled
if JIT is enabled for RTTs (-XX:+JITRT), or interpreted
if JIT is disabled for RTTs (-XX:-JITRT).

With the RTGC, we recommend asynchronous JIT compilation,
and this is the default mode for RTTs. This compilation mode
will produce code that runs faster at a later time,
after the compilation finishes, but there is no spike in
the execution time (see graph).
This mode usually does not endanger determinism if
interpreted code in this application was fast enough for the
particular deadline.

With the Serial GC, users may want to turn on synchronous JIT compilation.
(This is accomplished by disabling the default asynchronous JIT compilation
with the -XX:-BackgroundCompilation command-line option.)
However, this mode of compilation is not recommended with the RTGC because
the thread that triggers the compilation is blocked during
compilation. This causes a spike in the execution time (see the graph),
and this breaks the determinism requirement. In fact this can create
unbounded jitter for all the threads, which is unacceptable in a real-time system.

java.lang.Thread

For instances of the java.lang.Thread (JLT) class, throughput and
start-up time are more important than determinism. As a consequence,
by default ITC is disabled and JIT compilation is enabled.
Note that the only way to disable JIT compilation for JLTs
is to run the application in interpreted mode (-Xint).

Unlike with the Java HotSpot VM, with Java RTS the JIT
compilation is asynchronous by default. Synchronous compilation is
dangerous because a JLT blocked on compilation while
holding a critical lock could delay an RTT.
If you do not use shared locks, or if you use the Serial GC, you can
enable synchronous JIT with the flag -XX:-BackgroundCompilation.

ITC can be enabled for JLTs with the flag -XX:+ITCJLT.
Using ITC for JLTs can help performance. If you know in advance
the behavior of an application with respect to JIT compilation, then
you can further speed up the application by specifying the methods to be
compiled at class initialization (ITC). This reduces the
time spent executing interpreted code. For applications that have a
short run time, this almost always provides a gain in performance.

When ITC is enabled for JLTs (-XX:+ITCJLT),
the methods in the precompile list are compiled at
class initialization, and all other methods are JIT compiled.

When ITC is disabled for JLTs (-XX:-ITCJLT),
all JLTs execute code compiled in JIT mode.

Execution Time
Based on Compilation Mode

This section shows graphically the effect of the compilation
mode on execution time, as a method is invoked a number of times. In
each
graph, the x-axis represents the number of times the method is invoked,
and
the y-axis represents execution time. As you can see on the y-axis,
precompiled code takes the least amount of execution time, interpreted
mode takes more, and compilation itself takes even more execution time.

Execution of
Precompiled Code

In the figure below, a method is compiled at initialization
time (ITC).

Figure 1 Execution of Precompiled Code

Regardless of the number of times the method is invoked, the
execution
time remains the same. Therefore the jitter is theoretically zero
(in practice, very small). The method is not interpreted, and no
compilation
takes place during execution.

Execution of
Code With Asynchronous JIT Compilation

The next figure shows what happens with asynchronous JIT
compilation.

Figure 2 Execution of Code With Asynchronous JIT
Compilation

In the beginning of the run, the method is executed in
interpreted mode
until JIT compilation is triggered by one of the internal counters.
The method that triggered the compilation continues executing the
method
in interpreted mode, while the compilation runs in the background
(asynchronously).
When the compilation finishes, subsequent invocations of the method
will execute the compiled code, and execution time will be lower.

In this case, the maximum jitter is the difference between
the execution of the interpreted code in the beginning and the later
compiled code. But this jitter is known and predictable.

Execution of
Code With Synchronous JIT Compilation

In the figure below, synchronous JIT compilation has been
specified (by disabling the default asynchronous JIT compilation
with the -XX:-BackgroundCompilation command-line option).

Figure 3 Execution of Code With Synchronous JIT Compilation

In the beginning of the run, the method is executed in
interpreted mode
until JIT compilation is triggered by one of the internal counters.
This compilation causes a spike in the graph for the execution time,
because the thread that triggered the compilation is blocked while
the compilation runs (synchronously) to completion.

The maximum jitter is the difference between the CPU time to
perform the compilation itself and the CPU time to execute the
compiled code. This jitter is unpredictable in two ways: we cannot know
when the compilation will occur, nor how much time
it will take, because other compilation requests might be
enqueued. Since this could potentially seriously disrupt a critical
phase of the application, this situation is unacceptable in a
real-time system.

Debugging

The -XX:+LogJitterProneEvents
command-line option produces a log file of time-stamped events that can cause jitter.
One of the events tracked is the beginning of the interpreted execution of a method.
This information can help you debug the generation of the compilation list.

With this option, the VM also logs two other events: entry into and exit out of steady mode.
The VM is in steady mode when it is in a time-critical execution phase,
that is, a phase where it should not execute a non-deterministic operation.
Entry into and exit out of steady mode can be marked by making calls to
methods in the
SteadyMode class in the extended RTSJ package.
Having such events logged can help you determine whether or not an event that causes
jitter occurred in a time-critical phase.

See the Command-Line Options section of
the Java RTS Tools, Tuning, and Troubleshooting Guide for a description of the log file
produced by the -XX:+LogJitterProneEvents command-line option.

Compiler-Related
Command-Line Parameters

The table below summarizes the command-line parameters that
are used to control the compilation options for the various thread types.
These parameters are intended either for a production environment or
a development environment, as indicated in the "Env" column.

Optimization
Command-Line Parameters

The Java RTS virtual machine can be configured to run with or without
the Real-Time Garbage Collector (RTGC). If the RTGC is not used,
then the default HotSpot serial collector is used, along with a
safepoint mechanism for both standard java.lang.Threads (JLTs) and
RealtimeThreads (RTTs).
(The RTGC is enabled by default but can be disabled with the command-line parameter -XX:-UseRTGC.)

Java RTS provides the following command-line options to control the use of
a set of optimizations that impact the trade-off between performance
and determinism. These parameters are intended either for a production environment or
a development environment, as indicated in the "Env" column.

Parameter

Default

Env

Description

UseInlineCaches

true with Serial GC

false with RTGC

prod

Use HotSpot's original implementation of inline caches.

RTSJUseInlineCaches

false with Serial GC

true with RTGC

prod

Use a variant of the above that does not attempt to reclaim IC stubs
so as to avoid the use of safepoint.

UseCHA

true with Serial GC

false with RTGC

prod

Use HotSpot's original implementation of virtual method inlining.
If an inlining is invalidated because of dynamic class loading, the code is de-optimized.

RTSJUseGuardedCHA

false with Serial GC

true with RTGC

prod

Use virtual method inlining that does not require de-optimization.

The following table summarizes the thread types for which a
particular optimization can be enabled or not, depending on whether or
not RTGC is used.
In the table, "Y" means that the optimization can be enabled for that
thread type, and "N" indicates that it cannot.

"(Y)" means that the
optimization can be enabled but it does not provide any benefit
compared to
another optimization that can be enabled in the same situation. For
example, using RTSJUseGuardedCHA for JLT or RTT
(with Serial GC) does not provide any additional benefit because UseCHA can
be used
for these thread types when not using RTGC.