The Devil’s in the Data Details: Why a Good Data Management Tech is Critical to Your Digital Workflow

If you’re planning to shoulder a Super 16mm camera on your next project, I applaud your reverence for aesthetics.

Beasts of the Southern Wild probably wouldn’t have looked so magical if Ben Richardson had shot it on a Red. But with the unbeatable affordability of digital cameras, there’s a good chance you’ll be committing your movie’s raw footage to .MOV files instead of Kodak Vision 320T negatives. So, assuming you can’t afford to shoot on film—or simply choose not to—you’re going to need to manage your digital data. Accordingly, having a good Data Management Technician (DMT) to offload and organize your sound and video files, sync sound, and start rough scene assembly is absolutely critical.

As with film, the various digital formats for capturing and storing images and sound come with their own perils. The DMT position is one of the most important in the digital moviemaking workflow. Above all else, the DMT should possess impeccable organizational skills, an incredible attention to detail, and a problem-solving mind. On top of that, a DMT should be cool-headed. When things get stressful, if your data manager gets flustered, you could lose footage. It goes without saying, but you don’t want to lose footage.

Preparation

Prior to the first day of principle photography, a good DMT will work with the producer and/or post-production supervisor, and the director of photography/cinematographer to acquire the correct size, speed and number of hard drives needed to digitally store the entire project. The DMT will calculate the number of terabytes needed based on the number of shoot days, the number of camera(s) and their resolution(s), and how many channels of audio (as well as sampling rates) your sound department will be recording. When acquiring hard drives, make sure they get tested. Brand new hard drives can get damaged during shipping and could bring your offload to a standstill if they aren’t replaced during pre-production. I’ve had brand new—but untested—hard drives fail on set while copying large files, a problem that would’ve been largely avoidable if I’d had the opportunity to test the drives beforehand.

Offloading Media

The DMT will take the camera storage media (CF cards, SD cards, P2 cards, hard drives—four of the most common on-camera data storage devices) from the camera operator and the recorded sound files from the sound mixer, and copy the files directly to another storage medium—such as an external hard drive. It is very important to make at least one backup copy of the media. If a hard drive should crash, get lost, stolen, or otherwise damaged, that media is lost forever. The last thing you want to tell the producer is that the hard work the cast and crew has done was all for naught. With that in mind, at the end of each day of production, the primary and backup drives should be kept in different places, preferably under lock and key. If you keep the drives together and someone malicious has access to one, he has access to them all. When I’m on a job, I will typically save files to a total of three drives: one primary or “Master” drive; one backup; and one drive for shuttling dailies to the post house. When all files are transferred at the end of the day, the shuttle should be taken immediately to the post house, the master should go with the DMT’s supervisor (usually a producer), and, depending on the job, one should go home with the DMT himself. Especially on small projects, I almost always store the backup wherever I’m sleeping.

Verification

The DMT should be verifying, at the very least, that all media gets successfully copied to the various storage devices. There are many low-cost programs that you can use to do this step automatically, such as Shot Put Pro by Imagine Products, Inc. The DMT should also check to make sure the audio and video sync together, that takes are labeled correctly, and that the image and audio look and sound as intended. Overexposure is a common problem, as is audio clipping (when sounds are so loud they exceed the microphone’s preset recording levels), and the DMT should notify the appropriate person immediately so changes can be made—and scenes re-shot—if necessary. I was working on a low-budget feature where the sound mixer had to be replaced at the last minute by a (very smart) PA. This PA had read the online manual for the mixer that morning and was very timid with the levels, trying to make sure the audio never clipped. The problem, though, was that by ensuring the sound never clipped, he also ensured that the audio got recorded too low. We could hardly hear the actors. I let him know this as soon as I reviewed the sound files, and by that afternoon, everything was running smoothly again.

Workflow

The DMT should coordinate with the post house and/or editor to define an organizational strategy. Typically, most projects get organized by Shoot Day, Camera, and finally, by Roll, with each media file assigned a unique file name that includes the date and time it was shot. Every time a new card or hard drive gets inserted into the camera, the files get erased, and the card or drive formatted. That’s why it’s incredibly important for the DMT and the camera operator(s) to keep track of the current roll, and relay that information to the sound mixer and script supervisor. Miscommunication on this front results in—you guessed it—lost footage.

Post-Production

By the time a film is in post, the DMT has largely fulfilled his job responsibilities. However, each production is different, and more tasks may get added to the DMT’s list. Additional roles could include transcoding files, color grading (which falls under the larger Digital Imaging Technician job description), sound synchronizing, putting together an assembly edit, or even a rough cut for the DP and director to review. Typically, the larger the production (and the larger the budget), the more limited the DMT’s role will be in post. But if you’re working on a small show, it may make sense for the producer to delegate some of these post-production organizational tasks to the guy who’s the most familiar with the data.

Equipment

At the very least, the DMT must have a computer and a monitor. Laptops are fine, and will handle most of your requirements quite well. Where you will need extra computing power is in transcoding and/or color grading the coverage. This is especially true when you’re shooting at 2k, 4k, or larger (you’ll need a color-correct monitor if you’re doing color grading on set). The DMT needs to have a high-speed, multi-card reader to read compact flash cards, SD media cards, memory sticks, and whatever else gets thrown his or her way.

When offloading media, speed is very important. The DMT’s computer should have at least one Firewire 800 port and, if possible, an eSATA and/or Thunderbolt port(s). My “small footprint” kit includes a 2009 15” MacBook Pro with Express 34 card slot for additional peripherals, a dual-port eSATA Express 34 card, a Firewire 800 CF card reader, a multi-card reader, an array of cables to handle the wide variety of devices I might have to read from or write to, a power strip, and an Uninterruptible Power Supply (UPS). That last one is vital when you’re on set and an uninformed PA decides to move your power from point A to B without warning, or the electricians turn off the generator.

So, whether you’re planning to manage your video and audio data yourself, or whether you’re hiring someone to help, my hope is that this article will start you off on the right path. The bottom line is you want to make sure your data is backed up to multiple sources with integrity and security. If not, you might not have a film to edit! MM

This article appeared in MovieMaker‘s Complete Guide to Making Movies, 2013.