Getting Started With the Mobile 3D Graphics API for J2ME

This tutorial introduces the Mobile 3D Graphics API for the Java 2 Platform, Micro Edition (J2ME), defined by the Java Community Process (JCP) in JSR 184. The API is an optional package to be used with profiles like the Mobile Information Device Profile (MIDP). After an overview of the API, the article discusses potential application areas, the differences between JSR 184 and two related APIs, the classes in the new optional package, the programming model, the reference implementation, and some programming examples.

Overview of the Mobile 3D Graphics API for J2ME

JSR is the first Java-specific standard for three-dimensional graphics on mobile devices. The JSR's 26-member expert group includes all the major players in the mobile arena, including Sun Microsystems, Sony Ericsson, Symbian, Motorola, ARM, Cingular Wireless, and specification lead Nokia. The API takes the form of an optional package expected to be used with MIDP and version 1.1 of the Connected Limited Device Configuration (CLDC). It defines low- and high-level programming interfaces that bring efficient, interactive 3D graphics to devices with little memory and processing power, and with no hardware support for 3D graphics or floating-point operations. As new phones with diverse functionality appear, however, the API can scale up to higher-end devices that have color displays, 3D graphics hardware, and support for floating-point operations.

Application areas that will benefit from a 3D graphics API include games, map visualization, user interfaces, animated messages, and screen savers. Each of these areas requires simple content creation, some require high polygon throughput, and others require high-quality still images with special effects. To meet this wide spectrum of needs, the API supports both high-level and low-level graphics features, with a footprint of only 150 KB. In the high-level implementation (called retained mode), the developer works with scene graphs, and the world renders itself based on the positions of virtual cameras and lights. The low-level access ( immediate mode) allows applications to draw objects directly. You can use either mode, or both at the same time, depending on the task at hand.

The features of immediate mode are aligned with OpenGL ES standardization by Khronos. OpenGL ES (from "OpenGL for Embedded Systems") is a low-level, lightweight API for advanced embedded graphics using well-defined subset profiles of OpenGL. It provides a low-level interface between applications and hardware or software graphics engines. This standard makes it easy and inexpensive to offer a variety of advanced 3D graphics and games across all major mobile and embedded platforms. Because OpenGL ES is based on OpenGL, no new technologies are needed, which ensures synergy with, and a migration path to, the most widely adopted cross-platform graphics API, full OpenGL.

Requirements

The JSR 184 expert group has agreed on a set of capabilities that the API must support:

The API must support both retained-mode access (scene graphs) and immediate-mode access (the OpenGL ES subset or similar), and allow mixing and matching of the two modes in a unified way.

The API must have no optional parts; all methods must be implemented.

To reduce the amount of programming required, the API must include importers for certain key data types, including meshes, textures, and scene graphs.

Data must be encoded in a binary format for compact storage and transmission.

It must be possible to implement the API efficiently on top of OpenGL ES, without floating-point hardware.

The API must use the float data type of the Java programming language, not introduce a custom type.

Because using integer arithmetic is difficult and error-prone, floating-point values should be used wherever feasible.

The ROM and RAM footprint must be small; the API should be implementable within 150KB on a real mobile terminal.

The API must provide minimal garbage collection.

The API must interoperate properly with other Java APIs, especially MIDP.

JSR 184 requires version 1.1 of CLDC for its floating-point capability. Because most mobile devices do not actually have floating-point hardware, the API designers struck a balance between the speed obtained through integer operations and the ease of programming provided by floating-point operations. Calculations that require the fastest processing accept 8- or 16-bit integer parameters, and other calculations are performed with floating-point math for easier application programming. Figure 1 shows how the relevant APIs relate:

Figure 1: The Mobile 3D Graphics API Software Stack

Contrasting JSR 184 With Other APIs

Newcomers to the Mobile 3D Graphics API for J2ME might find it all too easy to confuse it with two other interfaces. Note these distinctions:

Java Bindings for OpenGL ES (JSR 239), which aims to provide Java bindings to the OpenGL ES native 3D graphics library, is still in its early stages. JSR 239 aims to provide the same functionality as JSR 184, but it caters to those developers who are already familiar with OpenGL.

The API

The Mobile 3D Graphics API for J2ME is defined in the package javax.microedition.m3g, which provides an easy-to-use API for rendering 3D graphics in retained mode and immediate mode.

In addition to the APIs, the package defines a scene graph structure and a corresponding file format for managing and deploying 3D content efficiently, along with all other necessary data: meshes, scene hierarchies, material properties, textures, animation keyframes, and so on. This data is written to a file using content-creation tools and loaded into the API through the Loader class. The most important class is Graphics3D, because all rendering is done there. The World class serves as the root of the scene graph structure. Object3D is the base class of all objects that can be rendered or loaded from a file, as well as the place where animations are applied. Table 1 briefly describes all the classes in the javax.microedition.m3g package.

Table 1: Classes in the m3g Package

Class

Description

AnimationController

Controls the position, speed, and weight of an animation sequence. For example, it can be used to control the blinking and movement of a light in an animation application.

AnimationTrack

Associates a KeyframeSequence with an AnimationController and an animatable property, which is a scalar or vector variable that the animation system can update directly.

Appearance

A set of component objects that define the rendering attributes of a Mesh or a Spring3D.

Background

Defines how to clear the viewport. In retained mode, or when rendering a World, the Background object associated with the World is used. In immediate mode, a Background object is given a parameter to clear. If a Background object is not present, the default values specified in the constructor are used.

Camera

A scene graph node that defines the position of the viewer in the scene and the projection from 3D to 2D. The camera always faces towards the negative end of the Z axis (0, 0, -1).

A singleton 3D graphics context that can be bound to a rendering target. All rendering is done through the render() methods in this class.

Group

A scene graph node that stores an unordered set of nodes as its children.

Image2D

A two-dimensional image that can be used as a texture, background, or sprite image. There are two types: mutable images can be updated at any time; immutable images are fixed at construction time and cannot be changed later.

IndexBuffer

Defines how to connect vertices to form a geometric object.

KeyframeSequence

Encapsulates animation data as a sequence of time-stamped, vector-valued keyframes, each of which represents the value of an animated quantity at a specified instant. Can be associated with multiple animation targets.

Light

A scene graph node that represents different kinds of light sources, which are used to determine the color of each object, according to its Material attributes.

Loader

Downloads and deserializes graph nodes and node components, as well as entire scene graphs. Downloading ready-made pieces of 3D content from an M3G file is generally the most convenient way for an application to create and populate a 3D scene.

Material

An Appearance component encapsulating material attributes for lighting computations. Other attributes for lighting are defined in Light, PolygonMode, and VertexBuffer.

Mesh

A scene graph node that represents a 3D object defined as a polygonal surface. It represents a conventional rigid body mesh, and its subclasses MorphingMesh and SkinnedMesh extend it with capabilities to transform vertices independently of each other.

MorphingMesh

A scene graph node that represents a vertex-morphing polygon mesh.

Node

An abstract class for all scene graph nodes. There are five different kinds: Camera defines the projection from 3D to 2D as well as the position of the viewer in the scene. Mesh defines a 3D object consisting of triangles with associated material properties. Sprite3D defines a screen-aligned 2D image with a position in 3D space. Light defines the position, direction, colors, and other attributes of a light source. Group serves as a root for scene graph branches.

Object3D

An abstract base class for all objects that can be part of a 3D world. These include the world itself, other scene graph nodes, animations, textures, and so on. Everything in the API is an Object3D except Graphics3D, Loader, RayIntersection, and Transform.

Stores a reference to an intersected Mesh or Sprite3D, and information about the intersection point. Strictly a run-time object, a RayIntersection is filled in by the pick() methods in the Group class, and cannot be loaded from a file by a Loader.

A scene graph node that represents a 2D image with a 3D position. This is a fast but functionally restricted alternative to textured geometry. It is rendered as a screen-aligned rectangular array of pixels with a constant depth.

Texture2D

An Appearance component encapsulating a 2D texture image and a set of attributes specifying how the image is to be applied on sub-meshes. The attributes include wrapping, filtering, blending, and texture coordinate transformation.

Transform

A generic 4x4 floating-point matrix representing a transformation.

Transformable

An abstract base class for Node and Texture2D, defining common methods for manipulating node and texture transformations.

TriangleStripArray

Defines an array of triangle strips. In a triangle strip, the first three vertex indices defines the first triangle. Each subsequent index together with the two previous indices defines a new triangle. For example, the strip S = (2, 0, 1, 4) defines two triangles: (2, 0, 1) and (0, 1, 4).

Holds references to VertexArrays that contain the positions, colors, normals, and texture coordinates for a set of vertices.

World

A special Group node that is a top-level container for scene graphs. A scene graph is constructed from a hierarchy of nodes. In a complete scene graph, all nodes are ultimately connected to each other by a common root, which is a World node.

Creating 3D Applications

An application that uses the Mobile 3D Graphics API for J2ME must be a valid MIDP application that follows the conventional MIDlet life-cycle. Use existing MIDP user interface classes such as Canvas and CustomItem to access the screen. To render the user interface, follow these steps:

Create a Graphics3D object.

Bind it to a 2D graphics object by linking the Graphics3D object to a 2D buffer.

Render the desired World, Node, or sub-mesh.

Release the graphics object, which will flush the rendered 3D view to the 2D buffer.

Complete functional examples appear later in this article, but this quick sample shows you the basics:

Version 2.2 of the J2ME Wireless Toolkit provides support for mobile 3D graphics by implementing JSR 184, including three demonstration MIDlets. To try these out, open the Demo3D project and run it. You'll see three samples listed, as in Figure 2:

The JSR 184 Reference Implementation (RI) was developed by Nokia. To get access to it and use it you must sign a limited license agreement with Nokia. The process is fairly straightforward and fast – it took me just a couple of days.

The RI, which is based on CLDC 1.1 and MIDP 2.0, comes in binary format. Note that you don't need to install CLDC or MIDP, as the RI includes them. It uses the open source Mesa 3D graphics library for rasterization. Mesa is a 3D graphics library similar to OpenGL. The RI has been developed and tested on Windows 2000 only, and doesn't come with the DLL files needed for Windows 2000. You'll need to download and build the Mesa DLL libraries yourself, especially because the RI works only with Mesa version 5.0.2, which was the release available at the time the RI was developed. Basically you need two DLL files: MesaGL.dll and osmesa.dll. You can either download the source code and build the libraries from scratch – you'll need Visual C++ – or build it following these steps:

Follow the instructions in the readme.win32 file in the MesaLib package to build the DLLs. The process will generate MesaGL.dll and osmesa.dll and write them to the lib directory of your Mesa installation.

Move the Mesa DLLs, MesaGL.dll and osmesa.dll, to the system folder c:\winnt\system32.

Once you have installed the RI and copied the DLL files to the right directory, change to the bin directory of your reference implementation and type midp. If all goes well, you'll see the emulator as in Figure 4.

Figure 4: Nokia's Mobile 3D Graphics API Emulator

The RI comes with a sample rendering MIDlet. To run it, use the following command:

midp -classpath ..\examples\jar\RenderTarget.jar RenderTargetMIDlet

Figure 5 is a snapshot of the running animation.

Figure 5: The RI's Sample Rendering MIDlet

Sample Applications

1. Retained Mode

In the first full-length example, a MIDlet uses the high-level API (retained mode) to play back a ready-made animation that it downloads over an HTTP connection. Note that the application plays the pogoroo.m3g file that comes with the J2ME Wireless Toolkit 2.2, and can be found at \WTK-HOME\Demo3D\res\com\superscape\m3g\wtksamples\pogoroo\content. Once you've located it, you'll need to modify the parameter to the call Loader.load("pogoroo.m3g"), in the following code.

To experiment with this MIDlet, create two subdirectories under your JSR 184 RI installation directory. You can call them whatever you like; for this illustration I name them mysrc and myclasses. All .java and .class files will be written to mysrc, and all verified.class files will be written to myclasses. Now do the following:

Copy the above MIDlet and save it in mysrc.

Compile it:

javac -classpath ..\classes.zip JesterTeslet.java

Preverify the .class files using the preverify command that comes with the RI. From the bin directory:

preverify -classpath ..\classes.zip -d ..\myclasses ..\mysrc.

This command verifies all classes in the mysrc directory and writes the verified classes to myclasses.

Run the MIDlet. From bin:

midp -classpath ..\myclasses JesterTestlet

You can also test JesterTestlet in the J2ME Wireless Toolkit 2.2; it should work just fine.

2. Immediate Mode

The second full example uses the low-level interface (immediate mode) to display a rotating, texture-mapped cube. The MIDlet initializes a 3D graphics context and binds it to a MIDP canvas. It then illustrates how to create a Mesh object manually, and set up the coordinates, triangle connectivity, texture maps, and materials. In normal practice, however, you'll create Mesh objects with a 3D modeling tool rather than programmatically, then retrieve them using the load() method.

To play with this example, you can follow the same steps as with the retained-mode example – or you can run it using the J2ME Wireless Toolkit 2.2:

In the toolkit create a new project and give it any name you like – I call it 3d – but make sure the name of the MIDlet is MIDletMain. After you click on the Create Project button, in the Settings window be sure you include the Mobile 3D Graphics API.

Copy the two example classes, MyCanvas and MIDletMain, from this article and save them as .java files in \WTK-HOME\apps\3d\src.

The emerging Mobile 3D Graphics API for J2ME provides an efficient interface for rendering 3D graphics on CLDC/MIDP devices. It supports devices with little memory and processing power and no hardware support for 3D graphics or floating-point operations, but the API can scale up to higher-end devices that have a color display, floating-point unit, and 3D graphics hardware, as they come along. The applications of this API are many, including games, screen savers, map visualization, and animated messages.

The code samples in this article demonstrate how easy it is to integrate 3D graphics into your MIDlets – but remember that this API is an optional package, and may not be supported on all devices.

TS-2121 Advanced Game Development with the Mobile 3D Graphics API
The presenters describe the design, programming philosophy, and feature set of M3G in detail, contrasting those with OpenGL and other existing APIs. The presentation closes with a review of the status of the API in the marketplace, and a discussion of where and how it is being used. This includes live demonstrations, running on off-the-shelf mobile phones. Plans for future evolution of the API also are discussed. You should have basic knowledge of 3D computer graphics. To get the most out of the presentation, some level of familiarity with OpenGL or similar would be helpful.

Qusay H. Mahmoud provides Java consulting and training services. He has published dozens of articles on Java, and is the author of Distributed Programming with Java (Manning Publications, 1999) and Learning Wireless Java (O'Reilly, 2002).