The CallSite

The bootstrap method is passed the class of the caller, and the name and type of the method specified by the instance of the invokedynamic instruction, and is responsible for returning an instance of the class

java.dyn.CallSite

The returned CallSite instance contains a MethodHandle. The JVM uses the method referenced by the MethodHandle as the method to invoke.

While this is not of itself a particularly exciting example there are few things worth noting

most importantly, what happens when the method Bar.foo() is invoked is determined entirely at runtime, i.e., dynamically

the whole thing was done in Java. Systems which generate bytecode themselves can obviously generate invokedynamic instructions directly as well as generating bytecode on the basis of the behaviour which results

in the example the bootstrap method produces a single CallSite. In practice bootstrap methods would produce different instances of CallSite depending upon their arguments, and would also use different MethodHandles depending on their arguments.

the CallSite class can be sub-classed, which enables further specialization of certain aspects of its behavior

in the example the method actually invoked does not do anything particularly interesting. In practice the behaviour of invoked methods can be dependent on their argument(s), and in the case of systems generating bytecode dynamically, invoked methods can feedback information to the code generation stage.

...
java.dyn.WrongMethodTypeException: (Ljava/lang/String;)V cannot be called as (Ljava/io/PrintStream;Ljava/lang/String;)Z
at DynTest.main(DynTest.java:24)

It is also an error to call a method via a MethodHandle with the wrong number of arguments, or the wrong argument types. In short MethodHandles are type-safe at runtime.

I believe that MethodHandles are also intended to be access-controlled, but at the current time they do not appear to be or not in the ways I would expect them to be. However, this is probably just a function of the state of my particular build and/or the current state of the implementation.

MethodHandles Considered Magical

Invoking the method referenced by a MethodHandle is suprisingly straight forward. Call its invoke() method.

But its only got one publically accessible method and that is

public MethodType type()

True, but it turns out that a MethodHandle is sufficiently advanced to be indistinguishable from magic, and automatically acquires an invoke() method with a signature corresponding to the method it references !

Replacing the 64-bit version with the 32-bit version in a continuing spirit of gung-ho optimism confirms that this is the indeed the code involved in invoking the method. Unfortunately it does so by crashing.

The most interesting feature in JDK 7 for me, at least until the recent reappearance of closures, and probably even then, is the JSR 292 invokedynamic support. It is basically the reason why I started my attempt to build OpenJDK 7 on Snow Leopard.

Ostensibly to use the upcoming JSR 292 features it is necessary to apply a number of patches to a standard OpenJDK 7 release. See here for details. However the current revision of the BSD port seems to contain a lot of the necessary code already so I decided to see how far I could get using that rather than a patched version.

JSR 292 is very much a work-in-progress. There was an early-draft release of the specification almost eighteen months ago, but nothing since. You can see what is going on by looking at the code as it is released, but it is still changing so some or all of the following may change.

Basically JSR 292 defines a new JVM opcode

invokedynamic

which is intended to provide VM level support for method invocation in languages other than Java which have different invocation semantics. That is a drastic over-simplification but it will do for the moment.

There are also some features visible in Java itself, one of which is method handles, which can be thought of as pointers to individual methods. To obtain a handle to a given method you need an instance of a MethodType, which defines the return type of the method and the parameters it takes. The classes

MethodHandle

and

MethodType

are defined in the package

java.dyn

To determine how they currently work and what else you can do you need to look at the source code and the accompanying comments.

Making A MethodType

The following code is based on my understanding of the currently available revision of the JSR 292 code in the BSD port. Hopefully makes an instance of MethodType specifying a method which returns void and takes a single argument of type String.

Running this code results in the following. Note that although the JSR292 code is present in the current revision of the BSD port it is still deemed to be experimental and has to be unlocked using the command line incantation shown.

As can be seen it generates the warning message in three cases. Cases one and three are if the initialization of native methods fail. Case two is if the class

sun.dyn.MethodHandleImpl

does not have a method of the form

???? static void raiseException(int arg1, Object arg2, Object arg3)

This is easy enough to check. The class is defined in the file

$(JDK7_ROOT)/jdk/src/share/classes/sun/dyn/MethodHandleImpl.java

In fact it does not.

Although it is not possible to know exactly what the method should do, the only use made of it is a bit opaque,
it is certainly possible to add something like this just in case it actually does get invoked.

from which we can deduce, after looking at the macro definitions for CC, DMH, etc., that the function we are interested in is

MHI_init_DMH

This function throws an InternalError exception without an error message in two different places, and calls a function which also throws an InternalError exception without an error message in two different places, so it is not possible to statically determine what has caused the InternalError.

Some experimentation results in the discovery that the InternalError is being thrown from the function invoked by MHI_init_DMH, namely

It is not called from the corresponding version of the method in the file

$(JDK7_ROOT)/hotspot/src/cpu/x86/vm/stubGenerator_x86_64.cpp

which is the version used in the 64-bit version of the JVM that is running the code. Hence the problem. The array has not been initialized and all the entries are NULL.

The code in the 32-bit version does not appear to be 32-bit specific, and the method MethodHandles::generate_method_handle_stub() is defined in a generic x86 file rather than an x86_32 one so what happens if we graft the code segment above into the 64-bit version of StubGenerator::generate_all() ?

we can determine that the MethodType value passed as the oldType argument originates from a call to the type() method on an instance of MethodHandle which has been passed to the calling method makePairwiseConvert()

The MethodHandle instance being passed to makePairwiseConvert is a newly created instance of DirectMethodHandle. The instance of MethodType it contains and which is returned by the call to the type() method, is in theory, the instance of MethodType created by the test program. This has already been printed successfully which implies that at that point it was sufficiently valid, or not sufficiently invalid to cause problems. This assumes of cource that it is the object actually being returned from the call to the type() method.

Given that it is extremely difficult to do anything from inside Java to a Java object which can subsequently cause the VM to crash, the problem is most likely to originate somewhere in native code.

As we already know the DirectMethodHandle constructor is invoking the static native method init() in the class MethodHandlesNative. We also know that the native function being called is actually MHI_init_DMH

This function is predominantly performing checks. It does not actually do anything very much to the MethodHandle instance it is passed. The function that it finishes by invoking

Some experimentation reveals that the value in the instance variable type is changed during the call to

java_dyn_MethodHandle::set_vmentry()

In fact it is changed to the value of me. It looks very much as though the type instance variable is being over-written with the MethodEntry pointer which is not a pointer to a Java object at all.

The Magical vmentry Field ?

The vm_entry instance variable is defined in the class sun.dyn.MethodHandleImpl as follows.

private byte vmentry; // adapter stub or method entry point

Given that a CPU-level pointer is being stored in vmentry its declaration as a byte is rather curious.

Looking at the definition of set_vmentry() we can see that it is simply placing the pointer to the MethodEntry at _vmentry_offset within the instance which does not itself explain how the instance variable is capable of storing a 32-bit pointer let alone 64-bit one.

Maybe there is something magical about this particular instance variable ?

_vmentry_offset is itself set in the method java_dyn_MethodHandle::compute_offsets() along with the type and vmtarget instance variable offsets

The type and vmentry offsets are the same so set_vmentry() is indeed clobbering the value in the type instance variable. Something
has gone wrong with the computation of the instance variable offsets, or the instance variable vmentry is insufficently magical in the 64-bit case.

The compute_offset() function simply looks up the field in the class and returns the offset as held by the field descriptor.

The offsets of the fields in an instance of a Java class are computed in the method

ClassFileParser::parseClassFile()

which is defined in the file

$(JDK7_ROOT)/hotspot/src/share/vm/classfile/classFileParser.cpp

This a long and complicated method, its almost a thousand lines long, but approximately half-way down is this

The Magical vmentry Field Explained

The ClassFileParser::java_dyn_MethodHandle_fix_pre() method is what makes the instance variable vmentry magic. It redefines
its type as either a Java int or long depending upon whether it is a 32-bit or a 64-bit VM.

As an aside despite the name the method is actually operating on the class sun.dyn.MethodHandleImpl not on the class java.dyn.MethodHandle.

It begins by looking for a particular field in the constant pool of the sun.dyn.MethodHandleImpl class.

except that in this version of the code there is a bug in the 64-bit case. I didn’t spot it straight away but I did get it eventually, albeit after a lengthy excursion into the mechanics of laying out instance variables in Hotspot.