Java 8: Now You Have Mixins?

Java 8 starts to emerge. It comes with a full new feature: functional programming with the lambda expressions. Here I’ll discuss about a feature that is part of the Lambda project (JSR-335): the virtual extension methods, also called public defender methods. This feature will help you to provide a default implementation for the methods that you have declared in your interfaces. This allows, for example, the Java team to add method declarations in existing interfaces, like List or Map. On their side, the developers do not need to redefine their different implementations in Java libraries (like Hibernate). Thus, Java 8 will be theoritically compatible with existing libraries.

The virtual extension methods are a way to provide multiple inheritance in Java. The team who works on Lambda project argues that the kind of multiple inheritance provided with virtual extension methods is limited to behaviour inheritance. As a challenge, I show that with virtual extension methods, you can also emulate state inheritance. To do so, I describe in this article an implementation of mixin in Java 8.

What a mixin is?

The mixins are kind of composable abstract classes. They are used in a multi-inheritance context to add services to a class. The multi-inheritance is used to compose your class with as many mixins as you want. For example, if you have a class to represent houses, you can create your house from this class and extend it by inheriting from classes like Garage and Garden. Here is this example written in Scala:

val myHouse = new House with Garage with Garden

To inherit from a mixin is not really a specialization. It is rather a mean to collect functionalities and add them to a class. So, with mixins, you have a mean to do structural factorization of code in OOP and to enhance the readability of classes.

For example, here is a use of mixins in Python in the module socketserver. Here, the mixins help to declare four socket based servers: a UPD and a TCP services using multi-processes, a UPD and a TCP services using multi-threading.

What are virtual extension methods?

Java 8 will introduce the notion of virtual extension method, also called public defender method. Let’s called them VEM.

A VEM aims to provide a default implementation for a method declared in a Java interface. This helps to add a new method declaration in some widely used interfaces, like the interfaces of the JDK’s Collection API. Thus, all the existing libraries like Hibernate that reimplements the Collection API do not need to rewrite their implementation, because a default implementation is provided.

Here is an example of how you will declare a default implementation in an interface:

A naive emulation of mixin in Java 8

Now, we will implement a mixin by using the VEMs. But I have to warn you first: Please, don’t use it at work! Seriously, DON’T USE IT AT WORK!

The implementation below is not thread-safe, subject to memory leak, and completely depends on the way you have defined hashCode and equals in your classes. There is another major drawback that I will discuss later.

First, we define an interface with a default implementation that seems to be based on a stateful approach.

Second, we define a utility class that owns a map to store associations between the instances of our previous interface and their state. The state is represented by a private nested class in the utility class.

“And Now for Something Completely Different”

This implementation seems to work well. But Brian Goetz, the Oracle’s Java Language Architect, has suggested me a puzzle where the current implementation does not work (assuming thread-safety and memory leak are fixed).

Solution to the puzzle

At first sight, there seems to be no problem with this implementation. There, X is an interface with a single method, because the method getName and setName have a default implementation and not the method run from Runnable. So we can generate an instance of X from a lambda expression, that will provide an implementation to the method run, as the method makeX does. So, you expect that this program displays something like:

x1
x2

If you remove the calls to getName, you expect to display something like:

MyTest$1@30ae8764
MyTest$1@123acf34

Where the two lines indicates that makeX produces two separate instances. And this what the current OpenJDK 8 produces (here I have used the OpenJDK 8 24.0-b07).

However, the current OpenJDK 8 does not reflect the eventual behavior of Java 8. To do so, you need to run javac with the special option -XDlambdaToMethod. Here is the kind of output you get once the special option is used:

x2
x2

If you remove the calls to getName, you will get something like:

MyTest$$Lambda$1@5506d4ea
MyTest$$Lambda$1@5506d4ea

Each call to makeX seems to give a singleton instantiated from the same anonymous inner class. Nevertheless, if you look at the directory holding the Java binaries (having carefully removed all *.class files before), you will not find a file named MyTestClass$$Lambda$1.class.

The complete translation of the lambda expression is not really done at compile time. In fact, this translation is done at compile time and runtime. The compiler javac put in place of lambda expression an instruction recently added to JVM : the instruction invokedynamic (JSR292). This instruction comes with all the necessary meta-information to the translation of the lambda expression at runtime. This includes the name of the method to call, its input and output types, and also a method called bootstrap. The bootstrap method aims to define the instance that receive the method call, once the JVM execute the instruction invokedynamic. In presence of lambda expression the JVM uses a particular bootstrap called lambda metafactory method.

To come back to the puzzle, the body of the lambda expression is converted into a private static method. Thus, () -> { System.out.println("X"); } is converted in MyTest into

private static void lambda$0() {
System.out.println("X");
}

This can be seen if you use the decompiler javap with the option -private (you can even use the -c option to see the more complete translation).

When you run the program, once the JVM tries to interpret the invokedynamic instruction for the first time, the JVM call the lambda metafactory method, describes above. In our example, during the first call to makeX, the lambda metafactory method generates an instance of X and links the method run (from the interface Runnable) dynamically to the method lambda$0. The instance of X is then stored in the memory. During the second call to makeX, the instance is brought back. There you get the same instance as during the first call.

Fix? Workaround?

There is no direct fix or workaround for this problem. Even if it is planned to activate -XDlambdaToMethod by default for Oracle’s Java 8, this is not supposed to appear in the JVM specification. This kind of behavior might change over the time and among vendors. For a lambda expression, what you can only expect is to get something that implements your interface.

Another approach

So our emulation of mixins is not compatible with Java 8. But there is still a possibility to add services to an existing class by using multiple inheritance and delegation. This approach is referenced as virtual field pattern.

We need an interface based on Switchable with an additional abstract method that returns an implementation of Switchable. The inherited methods get a default implementation: they use the previous getter to transmit the call to the Switchable implementation.

Conclusion

In this post, we have seen two approaches to add services to a class in Java 8 with the help of virtual extension methods. The first approach uses a Map to store instance states. This approach is hazardous for more than one reason: it can be not thread-safe, there is a risk of memory leaks, it depends on the way your vendor has implemented the language, etc. Another approach, based on delegation and referenced as virtual field pattern, uses an abstract getter and let the final implementation describes which implementation of a service it uses. This last approach is more independent from the way the language is implemented and more secure.

The virtual extension method is something new in Java. It brings another mean of expression to create new patterns and best pratices. This article provides an example of such a pattern, enabled in Java 8 due to the use of virtual extension methods. I am sure you can extract other new patterns from them. But, as I have experienced here, you should not hesitate to share them in a view to check their validaty.

EDIT: after sharing in the lambda project mailing list, some modifications has needed to be reported in this post, in a view to reflect the new behavior of Java 8. Thank to Brian Goetz from Oracle and Yuval Shavit from Akiban for their suggestions.

So why would the JDK implement lambdas as cached objects, while they are being advertised to the public as sugared anonymous classes? It is an instance where the principle of least surprise is utterly violated.

I assume they replaced SAMs with Functional Interfaces to be able to do these optimizations, but they clearly forgot about external state. To me it sounds like premature optimization.

Actually default methods are not required to show this. There may already be libraries out there that associate external state with Functional Interface instances. Passing in a lambda could break existing code!

Ali, your comment is interesting. For the external state, I cannot give an answer and I suggest that you contact directly the mailing list of the Lambda Project. Nevertheless, do you an example or two of a library based on such external states?

For the way the lambdas are translated, indeed, to use cached objects (and this is the case only for completely independent lambdas) is an optimization. In a sense, it is welcome because when you are playing with functional programming, you tend to generate a wide range of functions, if they are not cached. And the JVM suffers in being efficient in this case, even if it provides some other optimizations for this. But, on the other hand, I was as surprised as you were once I discovered that some lambdas are cached. I am agree with the fact that this can lead to errors difficult to understand.

In fact, it is not “single-method interface” but “single abstract method (SAM) interface”. As far as I understand it, by “single abstract method interface”, you have to read: an interface where it misses the implementation of one method to represent a complete implementation of a class. For example, those interfaces below are SAM interfaces:

interface A { void m(); }

interface B { void n(); void o() default { … } }

A is a SAM interface because you should define m() to be complete. B is a SAM interface because you should define n() to be complete, knowing that there already is an implementation available for o().

If you apply a lambda to a SAM interface, the lambda is directly mapped on the abstract method.

A functional interface is defined as an interface that has EXACTLY one abstract method (an abstract method is a method that lacks a body). So this means you can have tons of default methods and one abstract method, that’s a functional interface (aka SAM).

Isn’t calling it “FakeBrokenMixin” and saying “our emulation of mixins is not compatible with Java 8″ a bit misleading?

The problem seems to be that makeX() returns the same object twice; the fact that it does so appears to have nothing to do with the fact that it’s using a mixin emulation technique. (If I’ve misunderstood this, please do correct me). Would the same problem occur – cached object returned – even if we weren’t using mixins?

The interesting and surprising thing here seems to be the behaviour of lambdas.

Please, do not write code like this one! It is like asking for troubles. Don’t keep state in singleton maps. If you want to inherit state use abstract classes no default methods, and dont keep state of your object out of it. Default methods are to be used to inherit behaviour, and they should act only on other methods defined in the interface.

This is something new for me. I have read couple of Java 8 tutorials about new features but this is clever use of that. By the way, I have also shared couple of Java 8 lambda expressions examples, which complements your post. Your reader may find that useful.

[…] of multiple types. Something similar can be achieved with default methods, though, either with an evil hack or the virtual field pattern. The former is dangerous and should never be used, the latter also has […]

[…] of multiple types. Something similar can be achieved with default methods, though, either with an evil hack or the virtual field pattern. The former is dangerous and should never be used, the latter also has […]