Modding Apple Sample Code

Apple develops sample code to demonstrate and showcase its developer technology. Often the sample code provides end users with applications that offer unexpected utility. Sometimes you can leverage this technology directly. Sometimes you need to modify the source code to create the results you want.

This article focuses on MovieVideoChart, a typical Apple sample app that's meant to demonstrate how to access video frames in QuickTime 7. But that's just where things begin to get interesting. The scrolling layout and built-in video frame displays make for a perfect starting point, allowing you to re-aim the application toward a different end point.

In this article you'll discover how to modify the MovieVideoChart sample code to create a comic-book-like video layout tool. This tool will be able to scroll through video frame by frame, letting the end user compare successive frames in the same window. And along the way, you'll pick up some important techniques that will help you modify any Apple sample app you encounter.

Tip: You'll need to be at least slightly familiar with both Xcode and C programming to work through this article.

Finding Useful Code

Apple's generous library of sample applications is chock full of gems like HackTV, which you can use to monitor and record Firewire video, and SimpleVideoOutX, which creates a DV output stream from QuickTime videos. These utilities can be used exactly as produced, with terrific results and without any further modification needed on your part.

Apple's sample apps aren't intended to provide libraries of free software, even though in some cases they do. They're meant to showcase a specific Apple technology and provide the source code that demonstrates how to use that technology in your own applications.

Some sample applications simply provide a jumping-off point; many of these apps lend themselves to adaptation. You can modify the source code to build your desired utility by grabbing the preprogrammed chunks at a macro level and making relatively minor tweaks to create the results you want.

MovieVideoChart is one such application. By itself, the sample app is fascinating. It is not, however, useful in any general way. This isn't to say that the information provided isn't helpful or important, but that the utility isn't general-purpose. Yet with a little tweaking the app easily lends itself to modification. Here's a quick overview of the intended purpose and function of MovieVideoChart. Feel free to skip ahead to the next section if you start to nod off.

More than You Wanted to Know About MovieVideoChart

As designed, MovieVideoChart demonstrates how to access MPEG frames in QuickTime 7. Because MPEG compression uses three frame types, QuickTime's underlying technology uses slightly different techniques to access each kind of frame. As the name indicates, key frames (or I-frames) store complete frame shots. Predictive frames (or P-frames) store the difference between one frame and the next. Bidirectional frames (or B-frames) look for changes in both directions--forward and backward--providing the highest level of compression. B-frames introduce the most difficulty with respect to recovering full-data frames because of the interframe dependencies.

Since B-frames rely on data both before and after them, they're tricky to handle when you're trying to retrieve an entire image from the movie data. New QuickTime 7 support allows you to pull B-frames at will, accessing both previous and future frames to create the entire image. This is very convenient and much easier to handle than in the past.

B-frames are the frames that can be dropped most readily during playback on slower connections and machines. As Figure 1 shows, in the MovieVideoChart display, blue-dotted B-frames are marked "droppable" for this very reason. The red-dotted "Sync" frames are I-frames, and the green dots indicate P-frames.

In addition to identifying frames, MovieVideoChart gives you insight into the frame order of your movies. It demonstrates how the new APIs take care of all interframe dependencies, so as a programmer, you can grab data on a frame-by-frame basis. Notice how the actual media frame order in the top row is 1, 3, 2, 5, 4, 7, 6 and so forth? Via QuickTime, this decodes to the expected display order of 1, 2, 3, 4, 5, 6, 7 shown in the bottom row.

Tip: If you want to create a movie that takes advantage of B-frame compression, export from QuickTime using a highly compressed format like MPEG-4 using Broadband Low settings.

Figure 1: MovieVideoChart in its original form as distributed by Apple

Download the Sample Code

You'll find a copy of the MovieVideoChart sample code at the Apple Developer site. Download your copy using either zip or dmg format, and decompress or mount the archive to access the actual files. You'll discover that Apple has included a compiled, ready-to-use version at the top level of both archives.

Survey the Software

Unsurprisingly, before you start modding sample code, you've got to figure out which bits you want to keep and which you want to toss. Run the application and open a movie or two. (To open a QuickTime movie to test the software, select File → Open.) Play with the software and try to determine the core functionality you want to extract and build on. In this case the goal is already stated: to create a proper frame-by-frame, ordered display of (unencrypted) QuickTime movies.

Develop a To-Do Checklist

Here are the things that stand out about MovieVideoChart in its current state and what needs to be changed to get it to the goal. These are typical of the kinds of critiques you'll want to apply to any sample app. Try to enumerate your desired changes with as much detail as possible while keeping your changes realistic and achievable.

The pictures are quite small. They need to be larger and easier to view.

The only pictures of interest are the media (decode-ordered) pictures. Eliminate the others and promote the decode-ordered frames to the top of the window.

The default value for the frame display slider (initially in the lower-left of the window) is half width. It should be full width by default, and moved to a more appropriate part of the window.

A lot of text will become unnecessary and should be hidden or removed. Once the text is dealt with, the resulting white space should be used by the main display area.

Much of the interframe indicators, like the dots and the edits and the diagonal lines, are not of interest and can be tossed.

The Use Seconds/Use Time Scales pop-up is not relevant to the goal application. Disable and hide this pop-up.

Prepare a Backup and a Modification Log

Once you've established your modification list, you'll want to back up the original, unmodified application. Copy the entire sample folder with all the source code and other material provided by Apple. Keep this original copy on hand as a reference for changes and a source for reversion if necessary.

Next, create a modification log, either on paper or in your favorite text editor. As much as you might like the progress of sample code modification to be forward, it's often depressingly back and forth as you test out new mods and then have to revert them until you find the right balance of change and stability. Prepare to take copious notes as you test. Modding apps is more an exercise of exploration than of original design.

Tip: You can "undo" past previous saves in Xcode if you don't quit out of the program. When you're prompted to confirm that you want to undo past a save point, just let the program know that it's okay.

Read Through the Source Code

Most Apple sample app packages are small. Because they're meant to demonstrate a single technology, there's not a lot of excess code that you'll have to slog through. This means that you can scan through the source code fairly quickly to get at least a gist of the items you'll want to work with and change. In the case of MovieVideoChart, pretty much all the changes you'd want to make are limited to ChartView.c (Figure 2). You wouldn't know this, however, until you read through the code and discovered where the meat of the program was located.

Figure 2: Apple sample apps are usually sparse and easy to follow. The only two files of interest here are ChartWindow.c and ChartView.c. As a rule, you can basically ignore main.c, at least as far as application semantics are concerned.

Start Making Changes in the Code

I know how useless the directive "start making changes" must sound. It's a basic truth, however, that one is either a programmer or one is not. Feel your way around the source code and make a change here or there to discover the effect, exploring how your modifications alter the program's functionality and always keeping your goals in mind.

Keep your changes small. Compile them, observe the effects, and (here's the important bit) don't hesitate to undo each change after seeing what it does. Make sure to use your modification log to record the results of your exploration so you begin to understand the bigger picture of where you can make changes and where you cannot.

These are the changes made between the standard Apple distribution and the modified version available here. Figure 3 shows the result of these changes when incorporated into the program, compiled, and run.

Deleted draw_blob entirely and the call to draw_blob. This got rid of the small colored circles, in which I had no interest.

Deleted draweditlist and the reference to it, and drawtrackthumbnails and the reference to it. This removed the two undesired rows of thumbnails, leaving just the frames with the proper decode order.

Removed the line between decode time stamp and display time stamp. These were used to show the differences between the media order and the decode order and were of no interest.

Deleted the "Y axis" calculation and removed decodeAxisY and displayAxisY as variables. These weren't used anymore.

Updated the frame thumbnail drawing from framesInMediaDecodeRect to framesInTrackRect (four occurrences total). This instructed the program to draw using the top row rather than the bottom row.

Added a constant: #define custom_multiplier 5. I used this factor to expand the size of the thumbnails as follows: multiplied the decodeWidth, displayWidth, framesInTrackRect height and the height in the frame thumbnail drawing by my multiplier.

Figure 3: After applying a few simple code changes including a custom multiplication factor for the thumbnail size, the updated software looks like this. The frames are far bigger and the extra rows and superfluous graphics are omitted.

Update the Interface

Interface Builder provides the key to finishing the modding process for MovieVideoChart. It allows you to achieve interface goals like removing text, setting the default value for the slider, and moving items around in the window. Here are a few key pointers to keep in mind:

Adjust your layout when you move items. You'll want your windows to resize smoothly, with the interface elements pinned to the proper edges. Check the layout palette and make sure the settings reflect your item's new position.

Keep back-to-front layout in your mind. If you want to hide things, send them to the back of the layout. Bring your primary items to the front.

Avoid changing menu text. The program may depend on the current wording to operate properly.

Here are the steps to finish hacking MovieVideoChart, modifying its user interface in Interface Builder:

Remove the text from all the text fields on the left side of the window, starting with Frames In Track and ending with Sample Flags.

Select the main chart view. Bring it to the front (Layout → Bring to Front) and resize it to fit to the left of the window.

Resize the ScrollBar under the ChartView to fit to the left of the window as well.

Select the Use Seconds/Use Timescales pop-up and, in the Control inspector, disable and hide it.

Move the frame-size slider onto the Use Seconds/Use Timescales pop-up and bring it to the front. In the Layout inspector, pin it to the top and right, but not to the bottom or left. Run the simulator to make sure this works properly. In the Attributes inspector, set the Initial value to 100.

The updated interface will look like the one shown in Figure 4. Save your changes to disk.

Figure 4: The modified user interface is much simpler than the original

Test Your Finished Product

After saving your changes in Interface Builder, return to Xcode and build your completed application. Figure 5 shows the final result. Here, all the changes come together to tick each of the to-do item checkboxes in the original survey list. Test the program out. See whether it works as expected or whether you need or want to make further modifications.

Figure 5: The updated application

Conclusions

Apple sample code rocks. It is an under-tapped treasure available to anyone who signs up for the free online Apple Developer program. And MovieVideoChart is typical for the space. It's small. It's focused on demonstrating one ability. And it's fully hackable. If this article has shown anything, it is that it takes a surprisingly minimal amount of effort to capture the essence of a sample code project and transform it into something new. So here is the moral of the story: if a sample application almost does what you want it to, but not quite, take matters into your own hands. Hack the code to get the result you need. Do not fear the code.

Erica Sadun
has written, co-written, and contributed to almost two dozen books about technology, particularly in the areas of programming, digital video, and digital photography.