Summary

Extend the Java class-file format to support a new constant-pool form,
CONSTANT_Dynamic. Loading a CONSTANT_Dynamic will delegate creation
to a bootstrap method, just as linking an invokedynamic call site
delegates linkage to a bootstrap method.

Goals

We seek to reduce the cost and disruption of creating new forms of
materializable class-file constants, which in turn offers language
designers and compiler implementors broader options for expressivity
and performance. We do so by creating a single new constant-pool form
that can be parameterized with user-provided behavior, in the form of
a bootstrap method with static arguments.

We will also adjust the link-time handshake between the JVM
and bootstrap methods, so as to adapt the bootstrap API used by
invokedynamic to apply also to dynamic constants.

Based on experience with invokedynamic we will make adjustments to
the bootstrapping handshake of both invokedynamic and dynamic
constants, loosening certain restrictions on the processing of
argument lists to bootstrap methods.

This work requires some prototyping of JDK library support for a
representative sample of a few kinds of constant types, notably variable
handles (JEP 193). In support of such prototyping, this work will
coordinate with other work on basic language support for constant
expressions (JEP 303).

Non-Goals

This JEP aims to support arbitrary constants in the constant pool.
Although there are proposals for still other uses of bootstrap methods,
such as method recipes, this JEP concentrates on one use.

The success of this JEP does not depend on support from the Java
language or the Java compiler backend, although it is more likely to
succeed if it is used by the compiler backend.

Although large aggregate constants are a weak point in Java's
translation strategy, this JEP cannot solve for aggregates until
there are better ways to encapsulate them in constant forms, such
as frozen arrays or primitive-specialized lists.

Success Metrics

As a minimal requirement, it should be practical to expose constant-pool
forms to describe primitive class mirrors (int.class), null, enum
constants, and most forms of VarHandle in terms of CONSTANT_Dynamic.

Dynamic constants must be usable in any context which currently
allows general constant pool constants, such as CONSTANT_String
and CONSTANT_MethodType. Thus, they must be valid operands to
the ldc instruction and must be allowed as static parameters
to bootstrap methods.

The bootstrap-method handshake should support complex constants
which contain thousands of component arguments, lifting the current
limit of 251 constant arguments. As a stretch goal, there should
also be a way for the bootstrap method to more accurately control
linkage errors produced by resolving bootstrap method arguments.

At the end of the work we should also have cause to believe that this
mechanism can be made to work for a wide variety of library types,
such as derived method handles, small immutable collections (lists,
maps, sets), numerics, regular expressions, string formatters, or
simple data classes.

Followup work should be identified and documented. See "Possible
extensions" below.

Motivation

Section 4.4 of the Java Virtual Machine Specification describes the
format of the constant pool. Adding new constant-pool forms, such as
the support for MethodHandle and MethodType introduced in Java 7,
is a significant endeavor, and sends ripples through the ecosystem, as
it affects all code that parses or interprets class files. This
presents a very high bar to creating new constant-pool forms.

With invokedynamic, the value of storing complex data in the
constant pool is multiplied, since the static argument list for an
invokedynamic bootstrap is a sequence of constants. Designers of
invokedynamic protocols (such as the LambdaMetafactory added in
Java 8) routinely struggle with the need to encode behavior in terms
of the existing constant set—which in turn necessitates additional
error-prone validation and extraction logic in the bootstrap itself.
Richer, more flexible, more highly-typed constants remove friction
from the development of invokedynamic protocols, which in turn
facilitates the movement of complex logic from run time to linkage
time, improving program performance and simplifying compiler logic.

Description

Just as the linkage of an invokedynamic call site involves an upcall
from the JVM to Java-based linkage logic, we can apply this same trick
to the resolution of a constant-pool entry. A CONSTANT_Dynamic
constant-pool entry encodes the bootstrap method to perform the
resolution (a MethodHandle), the type of the constant (a Class),
and any static bootstrap arguments (an arbitrary sequence of
constants, barring cycles in the constant pool between dynamic
constants.)

We add a new constant-pool form, CONSTANT_Dynamic (new constant tag
17), which has two components following its tag byte: the index of a
bootstrap method, in the same format as the index found in a
CONSTANT_InvokeDynamic, and a CONSTANT_NameAndType, which
encodes the expected type.

Behaviorally, a CONSTANT_Dynamic constant is resolved by executing
its bootstrap method on the following parameters: 1. a local Lookup
object, 2. the String representing the name component of the
constant, 3. the Class representing the expected constant type,
and 4. any remaining bootstrap arguments. As with invokedynamic,
multiple threads can race to resolve, but a unique winner will be
chosen and any other contending answers discarded. Instead of
returning a CallSite object, as the invokedynamic instruction
requires, the bootstrap method would return a value which would be
immediately converted to the required type.

As with invokedynamic, the name component is an additional channel,
besides the type, for passing expression information to the bootstrap
method. It is expected that just as invokedynamic instructions find
uses for the name component (e.g., a method name or some ad-hoc
descriptor) dynamic constants will also find uses for the name (e.g.,
the name of a enum constant or the spelling of a symbolic constant).
Putting the CONSTANT_NameAndType in both places makes for a more
regular design. In effect, CONSTANT_Methodref and
CONSTANT_Fieldref constants are used to refer to named members of
classes, while the analogous CONSTANT_InvokeDynamic and
CONSTANT_Dynamic constants are used to refer to named entities
with user-programmed bootstraps.

The type component of the constant, with both invokedynamic and
CONSTANT_Dynamic, determines the effective type of the call site or
constant (respectively). The bootstrap method does not contribute
or constrain this type information, so that bootstrap methods may
be (and often are) weakly typed whereas the bytecodes themselves
are always strongly typed.

To relax length restrictions on bootstrap specifiers, the language
which defines the invocation of bootstrap methods will be adjusted
(with complete backward compatibility) to allow variable arity
(ACC_VARARGS) bootstrap methods to absorb, into their trailing
arguments, all remaining static arguments, even if there are 2^16-1 of
them. (The class-file format already allows this, although there is no
way to read over-long bootstrap argument lists.) For consistency, the
invokeWithArguments methods of MethodHandle will also be expanded
in this way, if the target method has variable arity. In this way the
invocation of bootstrap methods can be specified in terms of the
weakly typed methods invokeWithArguments and invoke, just as today
it is specified in terms of invoke alone.

Control of bootstrap linkage errors has proven to be a recurring
source of bugs and RFEs from users of invokedynamic and the trend is
likely to accelerate as bootstrap methods become more complex (as they
must, with dynamic constants). If we can find a way to offer fuller
control over exceptions to bootstrap methods, and it can be done
simply, we will consider delivering it as part of this JEP.
Otherwise, it will go on the list of future enhancements.

The draft Java Virtual Machine specification for CONSTANT_Dynamic can be
found in JDK-8189199,
the CSR issue associated with the main development issue of this JEP.

Future work

Possible future extensions include:

Support for bulk-scale constants such as arrays or resource tables

Further adjustments to the bootstrap-method handshake

Other uses of bootstrap methods which may synergize with dynamic constants

Attaching dynamic constants to the ConstantValue attribute of static fields

Surfacing the lazy initialization of constants in the Java language

Integrating new constants with special Java language rules for constant expressions

A discussion of design choices can be found in
JDK-8161256, which
deals with a number of related RFEs. The present JEP was distilled from
this larger list of features.

Alternatives

Many uses of CONSTANT_Dynamic can be replaced by equivalent
invokedynamic calls. (The call would take zero arguments and be
bound to a method handle that returns the desired constant.) Such a
workaround does not helps with a key requirement, however, which is
to be able to pass synthetic constants as bootstrap arguments.

Another alternative to CONSTANT_Dynamic is using static final
fields to name the desired constants, and computing their values in
the static initializer (<clinit>). This approach requires extra
metadata (a throwaway field definition per constant) and is not lazy
enough to avoid bootstrap cycle problems. Those problems are
routinely solved by building private nested classes with decoupled
static initializers, but that too requires extra metadata. If
languages evolve to use many such constants, there will be application
bloat from the excessive metadata.

Another approach is spinning static methods which perform the constant
elaboration logic, and then calling them lazily from invokedynamic.
Again, such throwaway methods are a metadata overhead which is large
compared to CONSTANT_Dynamic.

In practice the metadata overheads for simulating these features are
too large.

Dependencies

This feature is JVM-centric, and so does not depend on higher software
layers.

In order to ensure correct design, it requires at least experimental
adoption by several use cases. Library prototyping is a must, even if
the prototypes are thrown away.

As with invokedynamic, wide adoption requires use by the javac back
end, which in turn may require language extensions. As a basic first
step, translation workarounds which require hidden static methods, such
as translation of int.class or switch mapping tables should be examined
and reformulated with new constants, if possible.