Re-Introducing QuickTime for Java, Part 2

In the first
part of this re-introduction, we looked at the history of QuickTime for
Java (QTJ), the issues involved with writing, compiling, and running QTJ apps, and
presented a very basic movie player. In this second part, we'll explore the
organization of QTJ and use its editing API to write a small video editor.

Getting Object-Oriented

In the first article, I noted that a newcomer might be surprised at the
conventions of QTJ programming and the remarkable number of packages that even
a simple app needs to import. Both of these traits have their beginnings in
the fact that QTJ did not come about the way that many Java APIs do, with the
design of the API and then a reference implementation. In QTJ's case, the
implementation already existed, in the form of the native QuickTime libraries
on Mac and Windows. The Java API had to come second.

Moreover, though there is a lot of method to QuickTime's seeming madness,
there's no getting around the fact that it is a legacy API written for straight
C (or "procedural C," as Apple sometimes
calls it). While the Java API needed to be a wrapper to the native API,
there was no straightforward object-to-object mapping available.

So they made one.

Consider the following functions from the Movies.h header file:

StartMovie()

StopMovie()

GoToEndOfMovie()

CopyMovieSelection()

PasteMovieSelection()

LoadMovieIntoRam()

There's something all of these have in common — they all take a
Movie structure as an argument, as suggested by the naming
convention. This leads naturally to a Movie class in Java, with
the methods:

start()

stop()

goToEnd()

copySelection()

pasteSelection()

loadIntoRam()

Similarly, where a function returns a new Movie structure, the
Java equivalent is a static from-type method. Thus,
NewMovieFromFile becomes Movie.fromFile().

As for the packages, yes, there are a lot of them. By a quick count of the
package and class frames in the QTJ
JavaDocs, QTJ 6.0 has 32 packages, containing a total of 472 classes.
That's more than double the size of Java 1.0 (212 classes in 8 packages) and
almost as big as Java 1.1 (504 classes in 23 packages), according to the Java
history in David Flanagan's Java in a Nutshell. So
don't freak out if it takes a while to learn where things are. It's a big
API.

When writing QTJ apps, I've found it's more a matter of grabbing the
classes I need from the JavaDocs, noticing what packages those classes are in,
and importing appropriately. With that in mind, here are the packages you are
most likely going to be importing in most or all QTJ work:

quicktime: Defines the QTSession, whose
open() method must be called to initialize QuickTime before any
other QTJ calls are made. It also contains the QTException that
practically any useful method throws.

quicktime.io: File I/O classes, like
QTFile and the OpenMovieFile, used by
Movie.fromFile().

quicktime.std: Stuff from the QuickTime header files,
most notably a collection of constants in the StdQTConstants,
StdQTConstants4, and other classes. All of the function calls are
farmed out to subpackages.

quicktime.std.movies: The Movie and
Track classes, as well as the Atom and
AtomContainers that represent their low-level internal
structure.

quicktime.std.qtcomp: QuickTime
components, which are modular plug-ins that can come from the original
QuickTime install or from third parties, like MovieImporters and
MovieExporters to handle various media formats.

quicktime.app.display: Java GUI widgets, like
QTCanvas and the Swing-friendly JQTCanvas.

quicktime.app.players: Drawable
implementations like MoviePlayer and QTPlayer that
can render a movie into a QTCanvas.

For an official overview, Apple has an online package
roadmap you can look through.

task(): The Most Import Method You'll Never Call

One curious Movie method you might notice in the JavaDocs is
task(), whose description reads: "the moviesTask method
services this movie. You should call moviesTask as often as possible."
First of all, "moviesTask" is the name of the C function.
As you might guess, the Java equivalent is task() in the
Movie class. I guess they copy-and-pasted this from the C docs
without "Java-izing" it.

More interestingly, you could look through all 50+ sample apps in the SDK
and not once see a call to task(). What's the deal?

It is true that task() must be called frequently for your
Movie to do anything, but as it turns out, QTJ is very generous in
providing these calls for you. In our simple player example in the previous
article, the MoviePlayer registers with the
TaskAllMovies class to provide regular calls to
task(). Some of the other Drawables, like
QTPlayer (which we'll use in the next section), implement the
Tasking interface, which gives them access to a "tasker"
thread, which periodically calls back to their task() methods.

So you're largely off the hook for having to call task()
periodically. The rare case where you have to care is when you have a
Movie that has not yet been added to a GUI, yet is performing some
kind of activity. In a previous article about extending the
Java Media Framework with QTJ, we wanted to load a Movie from
a URL and while we didn't want to wait for the entire file, we needed to wait until there
was some data so we'd know the dimensions of the Movie.
Here's an edited version of the code:

This code tells QuickTime to get the movie from the URL (wrapped in a
DataRef object), and to prePreroll and
preroll it (which let QuickTime allocate resources for playing
back the movie). Since the Movie hasn't been added to a
MoviePlayer, QTPlayer, or anything else that will
provide timed calls to task(), we have to provide the task()
calls, at least until we add it to a MoviePlayer,
QTPlayer, or similar task()-caller.

One other interesting thing to note in this example is how we call the
fromDataRef with two behavior flags,
StdQTConstants4.newMovieAsyncOK and
StdQTConstants.newMovieActive, mathematically OR'ed together with
the | operator. This is a very common practice throughout the QTJ
API. Flag constants are defined in the various StdQTConstants
classes, and are typically ints whose values are powers of two,
meaning they have exactly one bit turned on. Here's an example from the native
Movies.h file:

By OR'ing them together, you can pack the flags into one int.
The method receiving the call will mask off bits to determine with which flags it
was called. Unfortunately, the appropriate flags are often not described
in the JavaDocs — for Movie.fromDataRef(), the JavaDocs
advocate using newMovieAsyncOK, but other appropriate flags like
newMovieActive are only described in the appropriate
section of the C documentation. Fortunately, methods in the JavaDocs
generally link to the appropriate C call's documentation, but sometimes you
still have to think about how the C call translates to Java.

Creating Movies

Like I said earlier, what sets QuickTime apart is its focus on being a media
creation API. Our closing example will show off some of those features.

While we don't have the space here to rewrite iMovie completely in QTJ, we can
certainly replicate the core of its functionality — combining video clips
from multiple sources and saving them into a new movie file.

To keep things simple, our "cuts-only" editor will simply let the
user load a source movie, select some or all of it, and copy that to the system
clipboard. Paste will put that video (or whatever video is in the system
clipboard) into a target movie, either appending to the target or replacing the
target's selection. If you don't want to touch the clipboard, there's a
"low-level" editing API, specifically the method
Movie.insertSegment(), that could be used instead. Post-paste, the
user can then make another selection from the source movie, or open a different
source movie. When done, the user can save the target movie to disk.

In general, our needs can be broken up pretty simply:

At launch time, create an empty target movie.

When the user clicks a "Load Source" button or menu item, show a
FileDialog and load a new movie from the selected file.

When the user does a "copy," put the source movie's selection on
the system clipboard.

When the user does a "paste," put whatever is on the system
clipboard (provided QuickTime can handle it) into the target movie.

When the user does a "save," we save the target movie to disk.

How much does QTJ help us to do this? The sample
SimpleQTEditor class is about 360 lines, and 100 of that is in GUI
layout. By way of comparison, it takes the Java Media Framework over 1,000
lines just to
concatenate two media files together on the command line.

Here's a sneak peek of the editor, scaled down to fit on the page:

Figure 1. Running SimpleQTEditor

QTJ makes handling the above tasks remarkably straightforward:

Creating the empty target movie. For this, we
construct a new Movie and get a MovieController,
which is needed for the QTPlayer to show an onscreen control bar.
We enable editing and add it to the GUI.

Notice, by the way, that using enableEditing changes the
slider (AKA the "play head") on the control bar (AKA the
"scrubber") from its usual ball shape to this:

The pointy shape is probably meant to make it easier to see where you're
clicking on the scrubber, but this looks a lot different than the custom
controllers in the QuickTime Player or iMovie. Frankly, it looks kind of
weird. Of course, you could always create your own controller widget in Java
(by subclassing Canvas or JComponent), track the mouse
and keyboard actions, and call methods on the Movie or a
MovieController to play, stop, move around, etc.

Loading the source movie. Here we bring up a
FileDialog and try to load it with Movie.fromFile().
If that works, we replace any movie currently in the source
QTCanvas with this one. We also switch the states of the open and
close buttons.

The copy is fast and the paste will be too, even if you're working with
many megabytes of media. That's because QuickTime is all about working with
references to media, so in this case, we're basically copying pointers
to the media in their original locations (in the source movie files), not
copying over all the media itself.

Pasting to the target movie. The paste would be trivial
if not for a few UI niceties I've included. First, we create the empty target
movie if it doesn't already exist. Movie.fromScrap() is a simple
enough call, though we need to double-check that it actually did return a
Movie and not null. The first nicety here is that
the paste is made to the end of the movie if there is no selection — it
would otherwise go at the beginning, and that's counter-intuitive when you're
selecting sources and putting them into the target one after another.
Post-paste, we clear the selection (so the user doesn't inadvertently wipe out
some or all of this movie with the next paste), and jump to the end of the
target movie to show that we did something (it might be nicer still to jump to
the end of the pasted segment).

The use of removeClient and setClient is kind of
overkill, but the preferred method of sizing a QTCanvas —
setting a resize behavior and informing it of changes to sizes in the
underlying movie — doesn't apply in this case. There is no target movie
attached to the QTCanvas when the GUI is created; that's done by
lazy instantiation in the first call to pasteToTargetMovie().

Saving the target movie to a file. There are several ways
to save a QuickTime movie to disk. In this case, we use the very handy
Movie.flatten(), which takes all the references to media in other
locations (files, URLs, etc.) and "flattens" the references into a
self-contained movie file, using whatever encoding and compression formats are
present in the originals.

QuickTime will let you paste in video of different sizes and will try to
scale them appropriately in the target QTCanvas. It's generally
pretty intelligent, though you can do some silly things by mixing movies with
different aspect ratios, like 4x3 home movies with 16x9 movie trailers.
Speaking of scaling, it's worth noting a difference between this example, where
we created the GUI first, and the SimpleQTPlayer, where the movie
was connected to the QTCanvas before bringing up the GUI. The
movie supplies the QTCanvas with a preferred size, which was
honored in the original example. In our editor, we set a size for the source
and target QTCanvases, forcing QuickTime to scale the movies
displayed in those widgets. In this case, we're depending on the default
QTCanvas behavior of allowing resizing to any dimensions —
there are several other modes described in the JavaDocs, such as resizing to
even multiples of the original movie's size, for performance reasons.

Onwards and Backwards

That's it for our post-facto introduction to QuickTime for Java. You should
now have a basic understanding of how to write and build simple apps with this
API. Future articles will head into more of the API, but if you want to try
out a few more things, check out the previous articles in the series. In Java Media
Development with QuickTime for Java, which is about writing a limited
JMF-to-QTJ bridge, we noted Movie.setRate() for playing a movie
faster, slower, or even backwards, and included a recipe for getting the
current QuickTime frame as an AWT Image. In Parsing
and Writing the QuickTime File Format, we looked at the data structure that
makes up a QuickTime movie, dumped its raw bytes to disk to create a playable
all-reference movie, and iterated through each of the ways to save a movie to
disk, from creating simple shortcut files to using MovieExporters
to convert a movie to any QuickTime-supported format.

Chris Adamson
is an author, editor, and developer specializing in iPhone and Mac.