The Return of the Blue Q

When last we saw QuickTime for Java (QTJ), the media API had been unceremoniously and unapologetically broken by Apple's Java 1.4.1 implementation for Mac OS X. As I wrote in a weblog entry at the time, the workaround was to force apps to run with Java 1.3.1, which all Mac OS X users could still use. However, this was an
intolerable situation to Mac- and Windows-based Java developers who
needed a high-performance media API that supported modern formats and
features -- who wants to depend on an API with no apparent future?

With the release of QuickTime 6.4 for Windows and Mac OS X,
QTJ has a future again. However,
the API has changed greatly in 2003, so much so that nearly any existing
QTJ application will remain broken unless it is rewritten for the new
QTJ 6.1.

This article will describe the new QTJ by relating the history of
why it was broken in the first place, how it was fixed, how to use the
new version, and what we might expect to see from QTJ going
forward.

From Biscotti to Broken

QuickTime for Java is one of the oldest third-party Java APIs.
Originally developed under the codename "Biscotti," and formally known as QuickTime for Java, QTJ was
announced in 1998. Starting with QuickTime 3, QTJ development continued alongside the
rest of the QuickTime API. As the underlying native library gained
features, QTJ would grow more powerful. Features that didn't require
new method calls, such as support for a new media format or codec,
were available immediately to QTJ. In other cases, a feature might
wait one version -- for example, QuickTime 5's API for
broadcasting became available to Java developers in QTJ 6.

It helps to understand the nature of QTJ's architecture. QTJ's
goal is to present a native, straight-C media API to Java
developers in a way that makes sense in Java
(because it is mostly composed of calls to a native
library, it can be thought of as "QuickTime from Java").
It creates an object-oriented view of QuickTime by crafting together C
structs and the functions that use them into sensible
objects. For example, the Moviestruct and several
dozen functions from Movies.h are represented as a class
called quicktime.std.movies.Movie. This collection of
classes is then farmed out into two sets of packages:

A wrapper to the underlying QuickTime API, directly
translating Java calls into native equivalents. This includes all
packages in the quicktime.std hierarchy, as well as a
few others.

A bridge into Java, providing points of contact between
QuickTime and Java, most obviously in providing a means of rendering
QuickTime content into AWT or Swing components. These classes
are in the quicktime.app hierarchy.

Apple's J2SE 1.4.1 implementation featured a rewrite of the GUI layer in
Cocoa, whose object-orientation and threading models were a better fit
for Java. QTJ continued to work with Java 1.4 on Windows, but the Mac version of QTJ was still Carbon-based, and calls from Cocoa's world of
pre-emptive threading to Carbon's cooperative threading were a dicey
proposition. There was also a too-tight integration with the older QuickDraw graphics API. For whatever reason, Apple chose not to address these problems before the launch of their 1.4.1, instead letting QTJ apps just fail with a NoClassDefFoundError when they attempted to access Carbon. Mac developers were up in arms that Apple's QTJ worked on Windows but not on their own platform.

After their Java 1.4.1 release, Apple posted information about the
incompatibility on QTJ's home page, saying that the future of QTJ was
under consideration, and soliciting feedback as to what features of
QTJ were actually being used by developers and users. The quicktime.app was very generous in
offering a large collection of display classes: effects and
rendering conveniences like an off-screen
Compositor, 2D "sprites," AWT and Swing
components with a wide range of resizing and display options, etc.
Fixing all of that code would require a lot of rewriting. With limited resources, Apple was asking the community to help identify the pieces that would be fixed.

Downsizing QTJ

The new release of QTJ, version 6.1, solves this problem by radically
reducing the size and scope of the Java bridge. This new version is
delivered as part of QuickTime 6.4 for Mac OS X 10.3 ("Panther")
and Windows. Note that on Windows, you need to do a custom install
or upgrade to get QTJ, as shown in Figure 1, since QTJ is not part of
the default installation.

Figure 1. Custom QuickTime install on Windows

On Mac OS X 10.2 ("Jaguar"),
the situation is a little confusing as of this
writing. Updating to QuickTime 6.4 wipes out QuickTime for Java, but
a separate update for QTJ 6.1 is available via the Software Update utility. Upgrading to QuickTime 6.4 but failing to get the QuickTime Java update will totally break all QTJ apps, so it's
important to update both. It's not unreasonable to expect that some
future version of the QT 6.4 updater would install QTJ 6.1
automatically, so this warning may not be necessary in the future.

Downloading an SDK to write QTJ apps is not strictly necessary,
since all you need is to put the
QTJava.zip file in your classpath. The installer puts
this file in /System/Library/Java/Extensions on Mac OS
X, or the lib\ext directories of any JRE folders it finds
on a Windows machine. However, the QTJ SDK has traditionally included a
generous collection of sample code, as well as Javadocs for the QTJ
API. A Windows version of this SDK is available from Apple's SDK page. The Mac version currently exists as separate pieces, with the Javadocs in one place and the demos available in another. Mac developers will want to consult the QTJ section of What's New in QuickTime 6.4, which indicates which of these demos do and don't work with QTJ 6.1 and Java 1.4.1.

Looking at the new QTJ, the solution to the complex legacies of the
old Carbon QTJ code is apparently to start over
with a radically simplified quicktime.app hierarchy, one
that only provides a basic ability to get QuickTime content, such as
movies and images, into the Java display space. Presumably, this
smaller set of functionality represents what developers said they
needed, and omits the extra goodies.

Specifically, all of these packages from earlier versions
of QTJ are now off-limits when developing for Java 1.4.1 or higher on
Mac OS X:

quicktime.app.actions

quicktime.app.anim

quicktime.app.audio

quicktime.app.display

quicktime.app.event

quicktime.app.image

quicktime.app.players

quicktime.app.sg

quicktime.app.spaces

quicktime.app.ui

These packages, however, are still available, and represent the
core of QuickTime functionality:

quicktime

quicktime.io

quicktime.jdirect

quicktime.qd

quicktime.qd.text

quicktime.sound

quicktime.std

quicktime.std.anim

quicktime.std.clocks

quicktime.std.comp

quicktime.std.image

quicktime.std.movies

quicktime.std.movies.media

quicktime.std.music

quicktime.std.qtcomponents

quicktime.std.sg

quicktime.streaming

quicktime.util

quicktime.vr

Note: several qd3d packages have been omitted from
this list because while they still exist, they do nothing on Mac OS X,
which does not support QuickDraw 3D.

The only surviving part of the bridge is the
quicktime.app.time package, which provides time-based
callback functions, such as the TaskAllMovies class.
It can be used to give movies time to redraw themselves, though
this is rarely used by application developers, since adding a movie to
a GUI sets up tasking callbacks for you.

Of course, if you look at the Javadocs, or peer into the .zip file
with a jar tf QTJava.zip, you'll see all of the old packages
and classes are still present. This is apparently to give developers
the opportunity to keep using those classes with Java 1.3.1 on Mac OS
X, or to use them with Windows. However, the affected classes are clearly
marked as deprecated, and attempting to use one may bring up a
runtime exception. For example, if you attempt to run an old QTJ
app in Java 1.4.1, trying to set up the GUI's QTCanvas
will kick up this exception:

Replacing the old QTJ GUI classes is a small new
package called quicktime.app.view, not to be confused
with the deprecated quicktime.app.display. This class
defines interfaces for components that can be used in AWT and Swing:
QTComponent and QTJComponent. In the latter
case, the "J" should be understood in the sense of a Swing
JComponent, not the last letter of "QTJ."

Using the New Classes

The means of getting and using these components has changed
radically. In the old QTJ, you would wire up a Movie to
a Java GUI like this:

Create a QTCanvas or JQTCanvas component.

Add it to your GUI. Optionally, set resizing behavior for the component.

Get a Drawable, such as MoviePlayer or QTPlayer, from a Movie, GraphicImporter, or other visual QuickTime source.

Call QTCanvas.setClient with the Drawable to wire up the component to the movie, by way
of the Drawable.

In the new QTJ, it works like this:

Call QTFactory.makeQTComponent() or QTFactory.makeQTJComponent() with your
Movie to get a QTComponent or QTJComponent, respectively. Another signature of makeQTComponent() takes a MovieController argument, which causes the returned widget to include a typical QuickTime movie control bar.

Call QTComponent.asComponent() or QTJComponent.asJComponent() to get an AWT or Swing
component that can be added to your GUI.

One thing to notice about this is that while the classes returned
by QTFactory may call themselves "components,"
suggesting membership in the AWT Component hierarchy,
these interfaces really only provide methods to
get and set their source movies or images. For compile-time type safety,
you need to
make the extra asComponent() call to ensure that you have a
real AWT Component.

You might also have noticed that the new workflow says nothing
about resizing behavior. While the old QTCanvas
described specific behaviors for resizing, such as maintaining a
movie's aspect ratio or only resizing to even multiples of its
original size, the new components have no such controls. Instead,
they make the fairly natural default choice of reporting their
content's size in getPreferredSize(), while scaling into
any size that might be imposed by the AWT layout. Practically
speaking, this means if you add a QTComponent as the only
member of a java.awt.Frame and then pack()
it, the resulting window will be just the right size for the movie.
On the other hand, if you want to force a movie to always be, say,
360 by 240, you could force this with something like a
GridBagLayout.

Other Changes

A few other changes are worth noting. The QTFactory
class that provides these components used to be in
quicktime.app, and provided methods for getting
Drawables from media sources like files or URLs. The
new QTFactory is in quicktime.app.view.
Since both versions exist inside of QTJava.zip, it's
theoretically possible for the names to collide, but only if you
import quicktime.app.*, whose classes have all been
deprecated.

One important feature that is now missing is capture. The
underlying classes for creating a SequenceGrabber exist
in quicktime.std.sg, but there's no clear way to get the
captured images onscreen. It may be possible to capture to an offscreen
GWorld and then devise a way to get those bits into a
component, but I'm not aware that anyone has gotten this working yet.
We may just have to wait for the next QTJ.

Rewriting the Examples

I've rewritten all of the example code from previous ONJava.com
articles for QTJ 6.1 and included it with this article. For instance,
Figure 2 shows the "Simple QTJ Editor" from a few articles
back running on Mac OS X 10.3 in Java 1.4.1 with QTJ 6.1.

"A Gentle Re-Introduction to QuickTime for Java, Part 2": The editor in this article is a little tricky, since I wanted to force the sizes of the components to be 320 by 240 for the target movie, and 160 by 120 for the source movie. I forced this by putting the QTComponent into the center of a BorderLayout, and then using Swing's Box to create horizontal and vertical struts of exactly the desired width or height, putting those struts in the south and east of the layout, thereby forcing a suitable size for the center.

"Making Media From Scratch": This two-part article stays entirely within the quicktime.std packages, making rewriting unnecessary. There are a couple of quicktime.app imports, but these appear to have been left over from debugging and aren't used at runtime.

I haven't included an update to "Java Media Development With QuickTime for Java," since I haven't been able to work out some problems with its code. That article offers a bridge from the Java
Media Framework to QTJ. More accurately, it uses QTJ as a
Player for JMF. One of JMF's basic assumptions is that a
movie's "visual component" and its "control panel
component" are two separate widgets. In QuickTime, it's more
typical to include the default control as part of the visual
presentation of the movie. In the old QTJ, it was
possible to create a "detached controller," which allowed
you to create two QTCanvases or JQTCanvases,
one for showing the movie and the other for the control bar. This
technique, as described in an Apple tutorial, crashes in QTJ 6.1. An
alternative would be to create a controller-less
QTComponent with
QTFactory.makeQTComponent(Movie), and then make a custom
AWT component, for which you'd have to handle mouse-clicks and make
appropriate calls to Movie.start(),
Movie.setTime(), etc.

Also crashing is the MovieExporter code used in the
article "Parsing and Writing QuickTime Files with Java." This is somewhat more troublesome, since the MovieExporters
are entirely within the quicktime.std hierarchy, which
seemingly didn't change for QTJ 6.1.

I've filed bug reports for these, and Apple developers have
repeatedly stressed on mailing lists that they want bug reports and
feature requests to be properly filed, which you can do by going to Apple's Bug Reporter. One trait of QuickTime for Java development is that new functions in the
native API often don't show up in QTJ until someone asks for them. I
complained in an earlier article about the lack of QTJ constants for
MPEG-4 and Sorenson 3 video codecs; filing a bug on these took five
minutes, and kMPEG4VisualCodecType and
kSorenson3CodecType appeared in QTJ 6.1 a short time
later.

The Future of QTJ

Is the shrunken quicktime.app going to hold back QTJ
in the future? I don't think so. It's likely that many developers
use a media API just to play media, meaning a simplified bridge could
actually be better, since it is now easier to use -- it was arguably
confusing to have to get a Displayable from a
Movie and then use setClient() to connect it
to a QTCanvas, all just to show some video.
Also, QTJ still picks up new features every time
the underlying library gains support for new media types or
codecs. That brought MPEG-4 playback to Java developers with QuickTime
6, and means that the professional-quality "Pixlet" codec
introduced in Mac OS X 10.3 should work in QTJ with no code changes
required.

Moreover, the core QuickTime functions in
quicktime.std provide APIs for movie editing,
transcoding between formats, metadata, low-level sample access, and
other tasks. This is the heart and soul of QuickTime, and it's not
going anywhere. For creating and working with media, the QuickTime
API remains unmatched.