The Multi-Tasking Virtual Machine: Building a Highly Scalable JVM

Some programming tasks related to systems programming cannot be performed solely using the standard Java libraries. Developers might have to use native code, scripting languages, and C or C++ before returning to Java code. For example, stating that a given Java computation should have the exclusive right to use at least 50% of the processor time of its host machine, or that its network usage should be limited to at most 5 MBs, cannot be expressed with the current interfaces.

And, today, when a user runs multiple Java applications concurrently in different instances of the Java virtual machine (JVM), there tends to be substantial duplication of effort in each JVM. For example, when multiple applications are executed concurrently, each has to load, parse, verify, and create runtime representations of all the applications' classes, even such common ones as java.lang.Object. Application start-up time, memory footprint, and raw execution time are thus negatively impacted. Developers who try to circumvent the problem by using class loaders soon discover that they provide inadequate inter-application isolation and make it difficult to cleanly terminate applications.

Sun Microsystems Laboratories' Project Barcelona research team, led by Dr. Grzegorz Czajkowski, is addressing both of these problems by developing techniques that allow for collocation of multiple applications in a single instance of the virtual machine. Applications are isolated from one another and each "thinks" it has the whole virtual machine all to itself. Aggressive transparent sharing of metadata, such as bytecodes of methods, reduces application footprint and start-up time. Furthermore, through building on the foundation of an isolated Java computation (known as an "isolate"), the team has developed a set of resource management techniques and APIs which allow developers to define new resource types, partition resources among computations, and control resource management overhead.

The Multi-Tasking Virtual Machine

"Across all the metrics of scalability we want to improve the well-being of Java programs."

The proposed extensions and modifications to the JVM have been prototyped as the Multi-Tasking Virtual Machine (MVM). Existing Java bytecodes can run unmodified, enjoying scalability benefits. New applications can take advantage of new abstractions and interfaces currently missing from the Java platform to accomplish a variety of systems programming tasks. "Across all the metrics of scalability we want to improve the well-being of Java programs," remarks Czajkowski. "We are scaling the Java platform and turning it into a complete operating environment, by adding various APIs and extensions so that anything you currently want to do in the domain of systems programming, you can do from within the language."

The MVM is a general-purpose virtual machine for executing multiple applications written in the Java language, based on the Java HotSpot Virtual Machine and its client compiler. Through hosting multiple tasks, the MVM increases the scalability of the Java platform.

"We went through the JVM runtime and asked the same question about every component: 'Can this be shared?'"

"We went through the JVM runtime and asked the same question about every component," explains Czajkowski, "'Can this be shared?' If it could be shared, it was. If it could not, we either replicated it or modified something to end up with maximal sharing." The MVM achieves improved scalability through an aggressive application of its main design principle: transparently share as much of the runtime as possible among applications and replicate only the part of the runtime system that depends on an application state. All of the known APIs and mechanisms of the Java programming language are available to applications. The end result is that one cannot tell if an application is running as an isolate on MVM or on a JVM of its own, barring a faster start-up and lower memory footprint.

While the original MVM prototype targeted the Solaris/SPARC environment, its creators see no major problems in porting it to other platforms. MVM is also a step towards ultimately providing a complete operating environment for Java programs, through a set of carefully designed APIs for resource management, service management, and clustering.

The Isolate API

Java isolates are a key to programming MVM. An isolate is a Java application that shares no objects with other isolates. The Application Isolation API Specification, JSR 121 in the Java Community Process, allows for the dynamic creation and destruction of isolates. A simple example is the creation of an isolate that will execute MyClass with a single argument "abc": newIsolate("MyClass", new String[] { "abc" }).start();

The creating isolate may pass contextual information to its offspring that is required by the application that is started within the new isolate. While isolates do not share objects, they can communicate using inter-process communication mechanisms such as sockets, files, or Java Remote Method Invocation (Java RMI). The Isolate API is fully compatible with existing applications and middleware. Applications unaware of the API may be managed as isolates without modification. Similarly, middleware can be unaware of or ignore the API.

"The abstraction of an isolate is important because it is a type-safe language analog of an address space, in which operating system processes are housed," remarks Czajkowski. It offers data protection guarantees, enables clean application termination, and provides a good foundation for per-application resource usage accounting. At the same time, there is no single prescribed way to implement isolates. Some implementations may equate an isolate with an operating system process. Others, much more scalable, will host multiple isolates in a single instance of the virtual machine. The latter can be viewed as a type-safe language kernel, just as, for example, UNIX is a kernel for programs developed in C.

Performance Gains through MVM

MVM offers substantial performance gains. When compared to the HotSpot Virtual Machine, from which it derives, MVM reduces start-up time from 60% to 90%. Thus, a three-second wait for a GUI application is reduced to about one second. Footprint reductions have ranged from a half to a third, while several percent execution time gains were also recorded.

Performance gains derive from two sources. First, because the MVM shares class runtime representations across applications, when one application loads a class, the other can use that class without having to engage in file fetching, parsing, and verifying. And second, with several Java components working in the same process, process switching is avoided when they communicate. For example, MVM collocated a pure Java database and an application server in a single process, which led to an 11 percent increase in throughput, with a request response reduction of 36 percent.

These gains allow certain classes of applications to be practically realized in the Java language. Utilities such as grep, ls, date, and others can now be written in the Java language and executed without slowing start-up.

False Starts in Process Sharing

Before completing the MVM, the Project Barcelona team evaluated two alternative designs which allowed separate JVM processes to share data. In both models, each application executed its own JVM in its own operating system process, while sharing some code with other JVMs.

The first prototype involved sharing meta-data by modifying the Java HotSpot Virtual Machine so that data, classes, and compiled code could be shared through memory. The JVMs collectively maintained a shared-memory area holding runtime representation of loaded classes and dynamically generated native code. "For example," explains Dr. Laurent Daynes of the Project Barcelona team, "an initial JVM would start filling the shared memory area with classes it has loaded and with compiled native code for the methods it most frequently uses. The next JVM arrives and ask if a class it needs is present. If it is, it uses it and replicates and resets those parts that cannot be shared. Optimally, there will be compiled code for some of the methods it needs, and it will not pay any interpretation or dynamic compilation costs for these methods. What the JVM cannot find in the shared memory area, it loads and builds, and then stores it in the shared area for the benefit of all subsequent JVMs. Thus, the third JVM will have an even broader set of classes and compiled methods available, and may run even faster. With this architecture, JVMs collectively build up a shared area of classes and compiled code, and amortize the class loading and dynamic compilation costs."

For the second prototype, the Project Barcelona team encoded entire Java packages as shared libraries so that read-only sections could be automatically shared among JVMs.

Both systems provided mediocre results. Start-up time and memory footprint both decreased by around 10%, numbers that were insufficiently impressive to justify the engineering efforts. And they faced serious engineering challenges involving cross-process synchronization for updating the shared area. "We found that a substantial fraction of the cost and overhead that can be eliminated through sharing across applications actually comes from the existence of multiple processes and multiple runtimes," says Czajkowski.

In addition, Daynes adds, "Whenever a JVM starts up, it has to create a new runtime, with lots of data structures around it, and they actually contribute mostly to the overhead, both in memory and start-up terms."

Transferring the MVM to the Java 2 Platform, Micro Edition (J2ME)

"It is relatively easy to put the MVM in CLDC. It's fast and there is no AWT."

Laurent Daynes,
Project Barcelona
Sun Microsystems Laboratories

The Project Barcelona team has been collaborating with a group at Sun to build a virtual machine for cell phone devices in an attempt to productize the MVM for the Connected Limited Device Configuration (CLDC). "It is relatively easy to put the MVM in CLDC," observes Daynes. "It's fast and there is no AWT. MVM is a good fit for CLDC for at least two reasons. First, memory is scarce on the types of devices targeted by CLDC, and the ratio of runtime data to application data is typically large, that is, a large fraction of the footprint is taken by the runtime representation of classes. In these circumstances, a JVM that shares most of the runtime data across programs is extremely valuable. Second, CLDC targets a wide range of platforms, some of which have no support for isolation. The ability to launch and control the execution of multiple applications with OS/platform-independent mechanisms increases the portability of both the JVM and applications".

Applying the MVM to the Java 2 Platform, Enterprise Edition (J2EE)

The Java 2 Platform, Enterprise Edition (J2EE) is similar to an operating system in that a J2EE server can host multiple applications. In practice, this is rarely done due to limitations on scalability, weak inter-application isolation, and inadequate resource management facilities in the underlying Java platform. This leads to a proliferation of server instances, each typically hosting a single application, with a consequent dramatic increase in the total memory footprint and more complex system administration. The MVM supported multiple instances of the J2EE 1.3.1 Reference Implementation through an efficient implementation of isolates, thus, substantially increasing scalability, reducing memory footprint, and server startup time when compared with the Java HotSpot Virtual Machine. This was achieved without changing server code and with no loss of performance.

The Resource Management (RM) Interface

While a single J2EE server can support multiple applications, much like a traditional operating system, performance levels may be difficult to control due to the absence of resource management facilities in the Java platform. The Project Barcelona team has created a Resource Management (RM) interface, implemented in the MVM and based on isolates, which offers a flexible, extensible framework for managing resources. The framework is applicable across a broad spectrum, from low-level resources like CPU time, to higher level ones, such as database connections. For example applying the RM API in a J2EE server allows application performance to be controlled flexibly and easily with low overhead and minimal intrusion.

The RM prototype was particularly effective in two ways: First, it eased the retrofitting of existing resource implementations to fit the RM frameworks; and second, it improved the programmable cost/precision tradeoff of the RM API. The prototype is already satisfactory for practical use.

On the Horizon

Work is underway to extend the isolate programming model, the RM, and MVM to clusters of separate machines in an effort to investigate the utility of these mechanisms in large, horizontally scaled, multi-tier J2EE configurations. "Isolates have turned out to have attractive properties as a model for distributed components: the API does not require expressing any collocation preferences, so a two-isolate application should execute unmodified in a single instance of MVM, as well as when each isolate is in a different MVM, on a different computer," explains Czajkowski.

The resource management API naturally extends to the cluster case, due to a strict separation between the decision to consume, or not consume a resource, and the actual implementation of the resource. Another, orthogonal, extension of the APIs is service management. The goal here is to express dependencies between various services in order to automate restart after a given service has gone off-line. Isolates provide a good foundation for defining Java services: they can be asynchronously and independently started and terminated.

MVM Robustness Issues

One challenge facing the MVM parallels a common operating systems problem: A bug in an application brings the application down, but a bug in an operating system brings all applications down. "If there is a problem with an implementation of MVM," admits Czajkowski, "all applications could be affected". This challenge to robustness can be dealt with by running several MVMs, to collocate related computations only. Czajkowski points out that OS kernels get enough debugging attention today so that they are reasonably stable. "The same thing is bound to happen with JVMs," he argues. "Over time, the issue of robustness in MVMs will fade."

Benefits for Users

The scalability improvements may impact users in myriad ways. For example, desktop users will be able to simultaneously run more Java applications due to lowered memory consumption. At the same time, reduced start-up time will significantly improve their experience. Administrators of large computer installations will welcome improved scalability. Developers of middleware infrastructure, such as application servers, as well as enterprise applications, which typically execute on server-class computers or clusters, will also find MVM's programming interfaces attractive. They simplify the task of managing large numbers of components and provide a means to exercise a high level of control over their lifecycle and resource consumption. For example, the Project Barcelona team was able to write a simple Grid system based on isolates and related APIs in four man-months. "I am quite excited about the prospects of writing command line tools in the Java language, or in scripting environments executed on the JVM, such as Groovy or BeanShell," comments Daynes.

The Future of the MVM

"The Java platform can grow to become a complete operating environment, subsuming the role of modern operating systems as well as middleware platforms."

The Project Barcelona team is working on a research release of MVM based on the Java HotSpot Virtual Machine, version 1.5. The initial release will contain a version of the Isolate API. Later releases will add the resource management, clustering, and service management APIs. Over time, the members of the Project Barcelona hope to demonstrate that, as Czajkowski puts it, "The Java platform can grow to become a complete operating environment, subsuming the role of modern operating systems as well as middleware platforms."

Daynes adds: "Moreover, the Java Platform might not be limited to programs written in the Java language: one can have programs written in various programming languages that target JVM bytecodes, and such applications can co-exist in the same JVM due to MVM's scalable isolation techniques."

Productizing of the MVM itself in the Java 2 Platform, Standard Edition, (J2SE) space is under consideration, but no decisions have yet been made.

In the meantime, the Project Barcelona team is planning the research release of MVM. Please check it out for more information about the release date and contents.