What's New For Developers in QuickTime

QuickTime 7 has made a splash since premiering as part of Mac OS
X 10.4 (Tiger) and as a separate download for users still on 10.3
(Panther). In
Part 1 of this series, I covered the features of QT7 that
end users will see, specifically:

In this second article, I'll cover some of the most prominent
changes from a developer point of view. These include:

The QTKit API for Cocoa

The new QuickTime MetaData API

Support for frame-reordering codecs

Core Audio-based sound enhancements

Miscellany: JavaScript and QuickTime for Java

Programming with QTKit

If, like me, you've picked up Cocoa and Objective-C after
spending years in other object-oriented languages like Java, you
probably don't mind the differences in syntax—you say square
brace, I say dot operator—because the ideas are fundamentally
similar. And to be sure, developing Cocoa applications is quite
pleasant when you get to use tools like Interface Builder, more
pleasant than writing hundreds of lines of
GridBagLayout code, like us J2SE guys are used to.

Unfortunately, Cocoa access to QuickTime functionality has been
limited up to now. Two Application Kit classes,
NSMovie and NSMovieView, provided a
bare-bones ability to open movies from the clipboard or a URL and
to send them a few basic messages: start and
stop; cut, copy, and
paste; setEditable and
isEditable, etc. Deeper needs than this required
taking a pointer to the QuickTime Movie and heading
off to the straight-C QuickTime API.

QuickTime 7's SDK—installed by Tiger or via Software Update
if you have XCode Tools installed on Panther—includes a new
framework called QTKit, which significantly increases the
number of things you can do in Cocoa and Objective-C, without
needing to resort to more difficult C calls to QuickTime
itself.

QTKit is still fairly small, consisting of just five
classes:

QTMovie: a collection of media resources and
their relative arrangement.

QTMovieView: a visual component for displaying
movie contents and (optionally) a controller widget.

QTTrack: an organization of one kind of media,
describing its timing, spatial, layering, volume, etc.

QTMedia: a reference to actual samples of some
kind of media—frames of video, clips of text, samples of audio,
etc. Each track has exactly one media.

QTDataReference: a specification of the location
of some media data, potentially in memory, on disk, or on the
network.

You might be tempted to think that QTMovie and
QTMovieView would be simple replacements for
NSMovie and NSMovieView, but the docs
reveal a surprise. While NSMovie offered just 11
methods, QTMovie offers 77, including some shockingly
powerful calls. For example,
addImage:forDuration:withAttributes: effectively
allows you to build video media, frame by frame, without even
looking at the
compression sequence API.

QTKit has its limits, of course. There's no object to represent
the SequenceGrabber for media capture, nor a wrapper
to the Presentation for broadcasting. Furthermore, the
objects that do exist don't wrap every possible QuickTime call with which
each object might be used. Think of it as an
80-20
proposition: what's provided so far is a small subset of the
QuickTime API, the most popular calls that will allow many
developers to meet their needs without delving into the straight-C
API.

A programming example of QTKit appears later in this article,
after covering another new feature.

Meeting Metadata

The ability to "tag" media with descriptive data is becoming
essential for supporting search, as well as providing a better
end-user experience. However, schemes for tagging media are all
over the map: the ID3 standard
for MP3's, EXIF for
JPEG's, and an undocumented and proprietary format for
iTunes-encoded AACs (that I investigated and provided a parser for
in the ONJava article
QTJ Audio and in the book
QuickTime
for Java: A Developer's Notebook). To top it off, QuickTime
movies allow for a
user data container, into which you can place pretty much any
data you like.

Implementing support for all of these formats in your
application would be a headache.

Realizing this, QuickTime 7 provides a new metadata API that
abstracts away the differences between the many schemes, exposing
all of them through a common interface. In its first
implementation, QuickTime supports three "containers":

Classic QuickTime user data: along with supporting
genuine QuickTime user data, this also provides access to MP3s' ID3
tags and other forms of metadata that are parsed into user data
items when files and streams are imported into QuickTime.

iTunes Metadata: album, artist, song, user rating,
cover art, etc., from files encoded by the user or purchased from
the iTunes Music Store. Note that according to
Movies.h, "The list of keys for iTunes metadata
is TBA."

New QuickTime metadata storage format: the docs advertise this as simply "a richer QuickTime metadata
container format." Among its most interesting traits is the
abandonment of the four-character-code scheme of user data keys in
favor of a reverse-DNS format, similar to package names in Java
(e.g.,
com.apple.something.somethingelse). This
has the advantage of avoiding name collisions, not requiring Apple
to act as a universal registrar of key names, and being infinitely
extensible.

The idea is one of simple key-value coding: you supply a known
key (like "description" or "album name"), and you get back 0 or
more values. In practice, using the Metadata API has significant
similarities to getting Components from QuickTime: you search for
them by providing criteria to match against, you call the search
method with the last match (null on the first call) to
find the next match, and there can be several matches even when
asking for fairly specific criteria. To clarify the last point, you
can ask for a given key in a given container format and get
multiple hits. After all, there might be multiple "author" or
"artist" tags.

Demo: QTKit and Metadata in Practice

By way of a demo, here's a short Cocoa application to show:

How to use QTKit's QTMovieView in a nib

How to get a QTMovie and put it into the
QTMovieView

How to move from QTKit to QuickTime, by getting the underlying
Movie pointer from a QTMovie object

How to get metadata from the movie with the new metadata
API

I'll walk through all the Xcode steps, but this is going to be
lightning fast; for a proper introduction to Cocoa development,
check out James Duncan Davidson's
Learning
Cocoa with Objective-C, 2nd Edition. The complete Xcode project
is available in the resources at the end of the article.

Building the QTKit GUI

First, kick off a new Cocoa-application project in Xcode. Double
click the MainMenu.nib to launch Interface Builder (IB).
The first thing you need to do in IB is to add the QTKit palette,
so that you'll be able to add a QTMovieView to your
GUI. So, do a "Show Palettes" (if the Palettes window isn't already
up) and go to "Palette Preferences..." Click the "Add..." button
and navigate to /Developer/Extras/Palettes/QTKit.palette to
add the QTKit palette. You should see the QTKit palette in your
Palettes window, as seen in Figure 1.

Figure 1. QTKit Palette in Interface Builder

The big "Q" is a QTMovieView. Drag it to the main
window to add a QTMovieView to your GUI. It will
originally appear as a very small component; I used the Inspector
to set its size to 320 x 256 (the common 320 x 240 movie size, plus
the usual 16 pixels for the scrubber bar) and resized the window to
align it to the bounds suggested by IB's friendly blue lines. The
resulting window should look like Figure 2.

Figure 2. QTMovieView in an IB window

Finish off the window by adding an NSTextView at
the bottom and a label above it that says "Metadata".

Now you need to start wiring things up, by creating a controller
object to handle UI events. In the MainMenu.nib window, click the
Classes tab, and then click NSObject. Go to the
Classes menu and do "Subclass NSObject". Rename the new class
MyController.

Now tell the controller about its actions and outlets: with
MyController selected, bring up the inspector. Add two
outlets, naming them metadataView and
movieView. Also add an action, which you can call
handleOpenMovie.

You need an instance of the controller, which you'll wire up to
the GUI components. With MyController selected, pull
down "Instantiate MyController" from the Classes menu. This will
create a "MyController" object, represented as a blue cube in the
Instances tab of the MainMenu.nib window. To wire up the outlets,
control drag from the object to the QTMovieView, as
illustrated in Figure 3. This will bring up the inspector, and
allow you to select which of MyController's outlets
you're connecting. Choose movieView. Now repeat the
process to wire up the metadataView to the
NSTextView.

Figure 3. Wiring up a QTMovieView as an
outlet

In the MainMenu.nib window, open the MainMenu object and trash
all the menus except "File", and delete all of its menu items
except for open. Control drag from the open menu item to the
MyController object to wire up the menu item to the
handleOpenMovie action.

Everything is now wired up. In the MainMenu.nib window, switch
to the "Classes" tab and select "MyController". Go up to the
"Classes" menu and select "Create classes for MyController". The
default location should be good, and you should see that "insert
into targets" says "QT7MiniDemo" (or whatever you named your
project). Save up and quit IB.

QTKit Coding

Back in Project Builder, your next job is to add in the QTKit
and QuickTime frameworks, so you can compile code against them.
Control click on "Linked Frameworks" and choose "Add -> Existing
Frameworks...". Navigate to /System/Library/Frameworks and
choose QTKit.framework. Then do the same thing to add
QuickTime.framework. You'll also need to import these
frameworks in your header file, so open MyController.h
and add the lines:

#import <QuickTime/Movies.h>
#import <QTKit/QTKit.h>

Next, you should strongly type the outlets listed in
Movies.h so the compiler can better check your code.
Your outlets should look like this:

All right, now you're ready to code! In the
handleOpenMovie method, your job is to present a file
dialog, open the selected file as a QTMovie, and put
the movie in the QTMovieView. The first part is pretty
easy... OK, it's all pretty easy, but this part you might have seen
before:

...or not. Getting a QTMovie from a URL is a
one-line call. OK, a few more lines if you want to inspect the
error if it isn't nil; this is in the downloadable
code, but I've omitted it here. Assume for now that it worked and
that you have a valid QTMovie object. Now you need to
put it in the view:

[movieView setControllerVisible: YES];
[movieView setMovie: qtMovie];

And... we're done. Seriously. This is all you need to do to open
and show a QuickTime movie with QTKit. The control bar is live, so
you don't even need to provide menu items or buttons to start and
stop movie playback, though it's not like a [qtMovie
start]; would exactly kill you, now would it?

This is an extremely simple example of QTKit, practically the
"Hello World" of using the framework. For a far more ambitious
example, check out Apple's tutorial
QuickTime Kit Programming Guide.

One little note here: I've noticed that when I do a development
build, the QTMovieView doesn't become active until I
switch to another application, and then back to QT7MiniDemo. This
doesn't happen when I build the application via a deployment build.
Just so you don't think you made a mistake...