It's not that we think O'Reilly readers skip the developer docs or ignore
the demos that come with the SDK, but we just want to be sure we're good before
we hit the hard stuff. After all, there's matrix math coming up when
we get to sprite animation ...

I will be using this article in the future for reference purposes, in the
sense of "If this is your first time writing a QTJ app, go back to this
article and make sure you've got everything set up correctly." Between
horrid CLASSPATH issues and the obtuse situation with QTJ and Java 1.4.1 on Mac OS X, it'll be nice to not have to reiterate three paragraphs of
gotchas every time.

A Little Background ...

Apple's QuickTime for
Java is a Java wrapper around much of the native QuickTime media API, which
itself has been available and under ongoing development since 1991. It is
available for the classic Mac OS, Mac OS X, and Windows. Note the obvious
omission: it is not available for Linux or any other OS at
this time; nor is there an all-Java version.

QuickTime is sometimes thought of as just a media format and player, a rival
to Windows Media and Real Player, with support for a wide
variety of media formats, including some that most people aren't aware of
(such as Flash 5 and Photoshop). But that thinking overlooks the fact that it
was designed as a media creation API, with tools to capture and edit
media, apply effects, and to export to different formats. QuickTime also includes a
legacy imaging API, sprites, and support for interactivity within a movie. QuickTime used
to include a 3D API, QuickDraw 3D, but it was dropped in Mac OS X in favor of OpenGL. We won't discuss it
again.

QuickTime had been around for years before the QTJ project started, and was
designed for use with straight C — not necessarily C++ or Objective-C.
While QTJ re-envisions QuickTime from a more object-oriented perspective, QTJ
code still seems different than what you might be used to working with in
JavaSoft's APIs. For example, while most classes in core Java and the standard
extensions define constants in the classes where they are used (e.g.,
Label.LEFT, Integer.MAX_VALUE), QTJ defines massive
lists of constants in StdQTConstants,
StdQTConstants4, etc., which hints at their origin as members of
big C header files like Movies.h.

By the way, Apple's sample code has a tendency to declare that it
implements these no-method interfaces, which is an easy-to-write
but hard-to-follow way of inheriting the constants. In my sample code, I will
always explicitly reference the constant by class so you can find it again
later.

When Not to Use QuickTime for Java

Let me start with the ultimate in simplicity — not using QTJ at all.
Every once in a while, someone will post the to the quicktime-java
list that they need to learn QTJ because they want to put a movie in an applet
in a web page.

That is overkill. Since it requires the user to have Java, QuickTime, and
QuickTime for Java all installed, the size of the potential audience is smaller
than it would be without so many dependencies.

If all you want to do is put a QuickTime movie in a web page, just use a
pair of simple HTML tags. Most browsers will pick up the <EMBED> tag, but since Internet Explorer is special, it needs an <OBJECT> tag. Here's a simple example of a suitable tag:

Some of these values are boilerplate and never change, namely the
<object>'s classid and codebase,
and the <embed>'s pluginspage. Beyond that,
the variables that you populate simply need to get repeated as both
<param> children of the <object> tag and
attributes of the <embed> tag. The only required values for
the simplest case are:

src: The URL of the movie

height and width: Dimensions of the movie. If
you intend to show the controller (the bar with the volume control, play/pause
button, and the "scrubber"), add 16 pixels to the height.

I've added an entry, autoplay, as another attribute, mostly just
to keep things interesting — it doesn't have a default value, since
general autoplay behavior is determined by the user's QuickTime preferences.
This and many more attributes are described in Apple's document "Embedding QuickTime
for Web Delivery."

You could also drop the controller, instead controlling the movie through
your own HTML buttons, links, or other page elements. You can do this with the
QuickTime plug-in's JavaScript
support. In this case, just add the attribute enablejavascript =
true to the <embed> tag (it's not necessary for the
<object> tag), give it a name attribute and
parameter, and then refer to the movie by that name when you call methods such
as Play() and Stop(), as in this example. More sophisticated methods are available for getting and setting movie properties like the playback rate
and volume, examining the track structure of the movie, and getting limited
access to sprite and QuickTime VR properties.

The downside of the JavaScript support is that it is only as good as the
LiveConnect — or, in QuickTime 6, XPCOM — support in your browser.
For QuickTime 6, that means that Mozilla and Internet Explorer work fine on
Windows, but the situation is a disaster on the Mac, where only Mozilla
supports JavaScript control of the QuickTime plug-in. IE is known not to work,
and while Apple says that Mozilla-based browsers should be OK, Camino and
OmniWeb don't work with my example, nor does Safari beta 2 (v.72).

One option to get a lot of mileage out of authoring (in lieu of coding) is
to use the "Synchronized Multimedia Integration Language", or SMIL
(pronounced "smile"). This XML markup, a WC3 standard, allows you to present audio, video, text, and other elements, from multiple sources, in one display, using the markup to arrange the elements temporally and spatially, indicating
what goes where and when.

In the <head> tag, we define a number of spatial
"regions". Our content will refer to these to indicate where it will
be placed on the screen. You can have any number of regions, with several used at
a time.

References to media go in the <body> section, grouped
according to whether different items are "parallel" or in
"sequence." Items in a <par> tag are shown or
played at the same time, while those in a <seq> tag are
shown or played one after another, either for their natural duration (if
appropriate) or for the time specified in a "dur" attribute.

Our example pairs an audio clip (a 15-second soundtrack from FreePlay Music, whose royalty-free audio clips are available to .Mac users via the iDisk Software folder), with a sequence of elements. The first four
members of the sequence are graphics: the ONJava logo, an "and" graphic, the MacDevCenter logo, and a "presents" graphic. Note that Apple's SMIL docs include a way to include text from a file or a data:
URL, but I used graphics so I could control the font and color.

The last member of the sequence is another <par> tag,
this time combining a poster frame, keagy-closeup.jpeg, with two
image links to the movie. Notice that the link elements refer to two regions
whose z-index is 2, while the main
region has a z-index of 1. This causes the link graphics to be
drawn on top of the poster image.

Among the many possible uses of SMIL, this illustrates a way to create
"alternate movies" (movies that redirect to other QuickTime movies,
perhaps to offer different sizes or bitrates). This is easier than the technique
offered here. (If you don't scare easily, check out the low-level
details of how this kind of real "alternate movie" works, along
with an audacious QuickTime
for Java implementation.)

Getting Started With QTJ

Let's assume that you really do need QuickTime for Java — you need to
deliver as an application, you're writing a cool transcoding servlet, or
whatever. Where should you begin?

First, check the system on your desk. Is it a Mac or a Windows box? Good.
Sorry, no QuickTime on other operating systems, and thus no QTJ — and
that includes CodeWeavers' CrossOver for Linux,
which supports only the QuickTime browser plug-in, not QT in applications.

Next, you need to install QuickTime, which is available from Apple's site, if you don't already have it. I will assume from this point on that everyone is using
QuickTime 6, which you need for some of the more interesting topics we'll be
covering, such as MPEG-4. By the way, you don't need to purchase
QuickTime Pro for QTJ development, though unlocking the editing and exporting
features of Apple's player does make it easier to create test media for your
development needs.

Mac OS X users can expect Java 1.3.1, QuickTime, and QuickTime for
Java to be installed with the operating system. Windows users don't
necessarily have any of these. Start at java.sun.com to get Java, then go get QuickTime from the link above. QuickTime for Java is not part
of the "Recommended" QuickTime install, so you need to do a
"Custom" install and specifically select QuickTime for Java.

Figure 1. Custom install of QuickTime on Windows

It's important to install Java first, since the QTJ installer needs
to find Java installations on your system and install its
QTJava.zip and QTJava.dll files where Java can find
them.

There are QuickTime for Java
SDKs available for download on Apple's site, but they are not strictly
required for development, as all you really need to compile is the
QTJava.zip file that is placed in your $JAVA_HOME/lib/ext directory. The SDK contains the javadocs and about 50 demo
applications. While both are available on the web, it's well worth your time to get the SDK to have local copies.

Your First QTJ App

It's not quite HelloWorld, but a basic player is pretty much
the first thing to create with a media API. Here's a simple player that asks
the user to locate a QuickTime-playable file on the filesystem, opens the movie
in a new frame, and plays it:

Here's what the output looks like — in my case, a home movie running
in a simple AWT Frame:

Figure 2. Simple QTJ Player

Try it with different kinds of media that QuickTime supports, such as
.mov and .avi videos, still images in various
formats, and .mp3 and .wav audio files. (You'll get a
zero-size window if you play an audio-only file.) This simple app should play
most, if not all, of the supported
media types.

Before we fight with the compiler — which will be a hassle the first
time you do it — let's examine what the code does. The essential
QuickTime stuff is in the constructor:

QTSession.open() initializes QuickTime and must be called
before any other QTJ calls are made.

Movie.fromFile creates a Movie object from an
OpenMovieFile.

A MoviePlayer is created from the Movie.

A QTCanvas object is created, and accepts a reference to the
MoviePlayer in its setClient() method.

The QTCanvas is added as the only component of the
Frame, making it the only thing we'll see.

An AWT WindowListener is set up to shut down QuickTime when
the application goes away.

Finally, the main() method calls start() on the
Movie, which starts playing in our window.

There are some peculiarities here: the number of imports is probably
surprising for such a small application, and you might wonder if there aren't
some unnecessary extra steps here. In particular, why do we hand the
Movie to this MoviePlayer object? Why is it then
sent to the QTPlayer, not in a constructor, but in the curiously
named setClient method?

Don't worry, all will be covered in due course.

Let's look at compiling first. The QTJava.zip file that
contains the QTJ classes is put in your Java extensions directory by the
installer. Strangely, it only gets picked up for running applications and
applets, not for compiling. When you compile, you must explicitly put the file
in your CLASSPATH. On the command line, use a javac
or jikes argument like -classpath
/System/Library/Java/Extensions/QTJava.zip (that's the Mac OS X path;
the Windows path will depend on where you installed the JDK). In an IDE, it's
usually a matter of dragging and dropping QTJava.zip into your
current project:

It will be pretty easy to tell if you don't have the CLASSPATH
correct, as all of those imports will fail, looking something like this:

SimpleQTJPlayer.java:5: package quicktime does not exist
SimpleQTJPlayer.java:6: package quicktime.app does not exist
SimpleQTJPlayer.java:7: package quicktime.app.players does not exist
SimpleQTJPlayer.java:8: package quicktime.app.display does not exist

Obviously, this is enough of a hassle that if you are not using an IDE, you
may want to set up a Makefile or an Antbuild.xml file to save you
from ongoing CLASSPATH misery. The example code for these articles will continue to provide Ant files to build the code, and you're
welcome to use them for your own QTJ development.

There's one additional compile-time hassle: in making huge changes to how
Java works in Mac OS X, Apple's 1.4.1 implementation is not currently
compatible with QuickTime for Java. The problem comes from Apple's move to
using Cocoa for their AWT/Swing
implementation in 1.4.1, instead of Carbon, which they used for 1.3.1 and QTJ. I wrote about this in an O'Reilly weblog last month,
and the problem has not been resolved as of this writing.

The workaround on Mac OS X is to use Java 1.3.1 explicitly, which will
reportedly still be in the next major release of Mac OS X. In fact, since
1.4.1 is currently an optional download, it's safer to code to 1.3.1 anyway.
Unfortunately, 1.4.1 becomes the default when installed, so it's a hassle.
When working from the command line, you can specifically point to the 1.3.1
javac and java by using the path:

/System/Library/Frameworks/JavaVM.Framework/Versions/1.3.1/Commands

I've included this path in the sample code's
my.ant.properties.mac file, which you need to rename to
my.ant.properties to get picked up by ant. If you're using
an IDE, there's probably a way to set your compiler and interpreter version to
1.3 in the GUI — in Project Builder, it's under "Java Compiler
Settings" in the Target Settings:

Figure 4. Setting Project Builder to use Java 1.3.1 for QTJ on
Mac

By the way, people have been asking me if I think the 1.4.1 problem means
that QTJ is doomed and if they should go looking for another media API. For
what it's worth, I prefer to just wait and see what happens — Apple
hasn't officially dropped or deprecated QuickTime for Java. In the past, they've
been pretty clear about when a technology is toast. Furthermore, Amazon still
has a
listing for a new edition of a QTJ book, written by Apple insiders, due in
September. The situation is unclear — maybe we'll get more information
at WWDC. In the meantime, while
I'd prefer not having to work around the 1.4.1 situation, it's not time to
panic. Not yet, anyway.

Movie and Other Essential Classes

In our example, it was simple to create a Movie object from a
file reference, by way of the OpenMovieFile object. Appropriately
enough, Movie is the heart and soul of the QuickTime for Java API,
offering method calls for most common tasks:

Creating Movie objects from files, the system clipboard,
memory locations, capture devices, and DataRef objects (commonly
used as a wrapper for URLs), or creating empty Movies
that we will create at runtime.

Getting metadata about the movie, including its duration, visual
dimensions, preferred and current playback rate, and preferred and current
volume.

Copying and pasting parts of a movie, or performing "low-level"
inserts that don't involve the clipboard.

Starting and stopping playback.

It's important to think of a Movie not just in terms of what
you can call on it, but also what it represents. In QuickTime, a
Movie represents an organization of different kinds of time-based
data, but it does not represent the data itself. In the QuickTime way of
thinking, Movies have Tracks that refer to
Media, which represent the low-level data like media samples (for
example, audio samples or video frames).

This division allows you to go as deeply into the details as you need.
Suppose you have a Movie that's made up of successive video tracks
of different sizes. (You'll be able to create such a thing by the end of part
2.) If you want to know the size it will take up on screen, perhaps to create
a rectangle big enough to accommodate all of the tracks, then you could call
Movie.getBox(). But if you need to know the size of just one of
those tracks, you could iterate over the Track objects, find the
one you want, and call its getSize().

Another key concept of Movies is that they have a time scale,
an integer that represents the time-keeping system within the movie. This
value defaults to 600, meaning the movie has 600 time units per
second. Values returned by methods like getDuration() and
getTime() use this time scale, so a 30-second movie with a
time scale of 600 returns 18000 for getDuration().
More interesting, though, is the fact that the various Media
inside this Movie can and will have totally different time scales
— uncompressed CD-quality audio might have a time scale of 44100, since
it's 44.1 kHz — but the Movie isolates you from such
details, so you don't have to worry about it, unless you want to iterate over
the Media objects and investigate them specifically.

The Movie object wraps a structure and many functions from the
native QuickTime API. Other classes in QTJ are unique to the Java version.
The QTCanvas is an AWT component that offers a bridge from the
QuickTime world to the Java display. To use a QTCanvas, call the
setClient() method to hook it up to a class implementing the
Drawable interface. Drawable is another QTJ-only
creation, which indicates the ability of certain QTJ classes to provide pixels
and sizing information to a QTCanvas. MoviePlayer is
used for the simplest playback needs — other special-purpose
Drawables exist for playing movies with a controller
(QTPlayer), showing output from a capture device
(SGDrawer), rendering effects (QTEffectPresenter),
and for stream-broadcasting (PresentationDrawer).

Notice that I described the QTCanvas as an AWT component.
SimpleQTPlayer is written entirely in AWT, not Swing. That's
because the QTCanvas is a native component, allowing QuickTime to
use hardware-accelerated rendering of your movie. The disadvantage, of course,
is the difficulty of mixing AWT and Swing components. Because heavyweight AWT
components will appear on top of any Swing components, a QTCanvas
will appear above Swing JMenus, blasting through
JTabbedPanes, and generally making a nuisance of itself, if you're
not careful.

If you must use Swing for a QTJ app, there are two options. First, you can
use the JQTCanvas, a lightweight Swing component introduced in
QuickTime 6 that behaves like other JComponents in honoring Swing
z-axis ordering. Unfortunately, its performance is generally poor; the movie
must be re-imaged in software to get it into Swing's graphic space so that it can be
painted. The second alternative is to carefully design your Swing layout to
accomodate the heavyweight QTCanvas, not using overlapping
components. The most common problem people experience with this is
JMenus disappearing under the QTCanvas. You can get
around this by calling setLightweightPopupEnabled (false) on your
JMenus.

Coming soon ...

Now we've got a basic player, but of course, that's what we got from the
QuickTime plug-in when we started. In Part 2, we'll delve into what makes
QuickTime unique by getting into the editing API, allowing us to write a basic
video editor with just a handful of code.

Chris Adamson
is an author, editor, and developer specializing in iPhone and Mac.