Reinventing Star Trek's VFX

How J.J. Abrams Demanded, and ILM Delivered, More Story from Every Shot

By Barbara Robertson / May 13, 2009

We’re sitting with Paul Kavanagh in a screening room at Industrial Light & Magic, watching director J. J. Abrams’ early cuts for the film Star Trek. Kavanagh was the animation supervisor at ILM, which created 864 visual effects shots for the spacefaring adventure. Abrams’ reboot of the venerable franchise scored big over opening weekend, raking in $76.5 million at the North American box office amidst galaxies of critical acclaim. ILM’s Roger Guyett, who was also the second unit director, supervised the visual effects, which also included work by Digital Domain, Evil Eye, Lola, Svengali and Kerner Optical.

Kavanagh is eager to make a point about the post-production work at ILM, and to do so, he’s showing some of the early cuts. Onscreen, the iconic U.S.S. Enterprise, streamlined, modeled, animated, and rendered at ILM, is voyaging somewhere in the final frontier. And then, as if we were watching a silent movie, a black card with white text interrupts the sequence. It reads:

As Kavanagh shows another sequence, he explains that ILM had created a different shot based on strong previs, but Abrams replaced that shot with the black card. “J. J. changed every sequence,” Kavanagh says. “Mary Jo and Maryann just had these black cards with descriptions of what he wanted.” (Kavanagh is referring to editors Mary Jo Markey and Maryann Brandon.)

Another black card appears:

“Camera is behind the Enterprise and a giant piece of black dish is heading for it.”

“These black cards were all over the place,” Kavanagh says. “We’re like, Ã¢Â€Â˜Oh, the shot changed.’ And J. J. would say, Ã¢Â€Â˜Yeah, but it’s so much better, isn’t it?’ And, it really was. He was telling a better story, or making the story more personal, or showing a conflict within, and he would be really smart about doing that. But we had to redo about half the shots. The third act was just black card after black card after black card, cut with some live action.” And by then, the previs team was long gone.

Working with J.J. Abrams(click to open/close sidebar)

Director J. J. Abrams, who was born three months before the first episode of the Star Trek television series aired, surrounded himself with Star Trek history, according to visual effects supervisor and second unit director Roger Guyett. Guyett, who was visual effects supervisor at ILM on Abrams’ Mission: Impossible III, talks about working with Abrams on Star Trek.

“He’s imaginative, creative, and he has a lot of energy,” Guyett says. “He does things for a reason, and you better have your shit together. He’ll ask you why you’re doing something. Ã¢Â€Â˜Why is that spaceship doing that? How does this advance the story?’ And you better know and understand why. You can’t just throw ice cream at him. You can get away with saying, Ã¢Â€Â˜I think that, at this moment in the movie, this looks really cool.’ But you can only do that once.”

Usually, at ILM, the pipeline works like this: the layout department receives background plates ‘ the footage shot on location ‘ or, for all-digital shots, the previs. From those sources, match-move and layout artists derive the camera moves and lens information to give animators 3D scenes viewed through a virtual camera that matches footage shot on set, or the approved previs. But the black cards meant the layout artists and animators needed to design the shots as well as create them. And on a tight schedule.

“We had from May ’08 until October/November ’08,” says Guyett, “because we were still on a Christmas release schedule. And in a space movie, you have large parts that don’t exist, which means there are many decisions.”

With all that in mind, Kavanagh decided to compress the pipeline. “I thought that we had to work as efficiently as possible or we would be spinning our wheels for a long time,” he says. “We needed to get multiple takes to J. J. quickly and get him to agree. So I combined layout and animation into one department for the show. I picked animators who had done cameras and layout artists keen to animate.”

All told, the team, which at the peak included 18 artists, animated shots with around 15 spaceships, two creatures, the Enterprise, the seven-mile long Romulan mining vessel called the Narada, and some digital doubles.

For the black-card shots, Kavanagh assigned sequences to artists in his new department who handled the work from previs to final animation. “We did something like 70 shots for the third act in five days,” Kavanagh says. “And we took them, almost to final, to LA and presented them to J. J. We came back with about 60 percent headed in the right direction and hammered again, hard, to take them to final. By doing the previs in our pipeline, the shots were almost ready to go.”

New Tech for Star Trek(click to open/close sidebar)

The action in this fast-paced film leaves tons of destruction in its wake ‘ even the destruction of two planets. To handle all the broken bits, from continents to dust, ILM’s R&D department created a new tool called Fracture within its proprietary Zeno software. Large chunks broken by Fracture can be fed into the studio’s rigid body simulator and then procedurally emit particles and instance smaller and smaller bits. “It’s all about adding multitudes of scale and using the engine best suited for that,” says Joakim Arnesson, CG supervisor.

Colin Benoit, a layout artist at ILM, was one of the people who moved into Kavanagh’s neo-animation department and became the animation sequence lead for the “space jump” sequence. In this sequence, Sulu, Kirk, and a red-shirted ensign named Olson jump down through space from their shuttle following a kind of umbilical cord that extends from the Narada, a huge Romulan ship, to a mining platform 1000 feet above the surface of Vulcan. When the villains pull the platform back to their ship, Sulu slips off. Because his parachute doesn’t open, Kirk jumps after him, and they free-fall to the surface of Vulcan.

One slice of the pie-shaped platform was a set piece. ILM created the rest of the platform, the seven-mile-long, claw-like Narada, the umbilical cord, the plasma drill that extends from the platform to the planet, the planet itself, the surrounding environment, and digital doubles as needed to match the actors skydiving on wires on a greenscreen set.

Although the fast-paced action makes the sequence move quickly, it comprises almost a third of the film. Benoit and his team, which included as many as three layout artists and three animators at times, started with the previs animatics when they could and the ubiquitous black cards when they couldn’t. To help quickly design the black card shots, Benoit used an orientation sensor.

“It’s a little device from Xsens, a gyroscope and magnetometer about two or three inches long and three inches high, that gave us orientation control at our desk,” Benoit says. By plugging the Xsens device into a USB port, the shot designers could plot the position of the camera and the pan, tilt and roll in real time.

“It was like I was looking through a real camera,” Benoit says. “I could watch all the 3D geometry, everything in 3D space, the characters fly by, the umbilical cord, the Narada, the drill platform, all optimized to play back in realtime.”

By holding the device in his hand, Benoit could move the camera through all three degrees of freedom while watching the footage. “I’d let the footage loop and do multiple takes,” he says. “I could crank out 10 or 15 takes, pick the best one, tweak it if I wanted, and render it out. I showed the device to J. J. and he was very excited. Sometimes, it’s hard to interact so directly with CG. This bridges that gap.”

Modeling the Future(click to open/close sidebar)

Modelers and view painters (texture painters) at ILM built a larger, more streamlined Enterprise, the Romulan Narada, the drilling platform, and other ships and objects for the film, using Autodesk’s Maya, ILM’s Zeno, and Adobe’s Photoshop.

Although it stayed true to form, the Enterprise grew from 1300 feet to 2000 feet in length for this film. Its “neck” is thinner and more graceful, and it has more visible moving parts. The team worked from concept art created by Ryan Church and leaned on the experience of such crewmembers as Alex Jaeger, who was the visual effects art director at ILM for Star Trek: First Contact in 1996, and John Goodson, who was the model project supervisor for that film.

“The Enterprise is an icon,” says Bruce Holcomb, model supervisor. “We didn’t redesign it. We brought it into the future in a sophisticated way. One of our big tasks was to make people go into the film and believe they’re looking at a practical model. That’s the hardest thing to achieve.”

The Enterprise is always CG when seen from the outside, as are the other space vehicles, all of which ILM created. For the Narada, the studio consulted reference materials from production designer Scott Chambliss, who called for a mysterious, asymmetrical vessel. “The reference materials were all over the place,” Holcomb says, “from architecture to a photo of tangled hair. So there was a lot of back and forth.”

For Holcomb, the scale of the work was most challenging. “The Narada was six miles long and the Enterprise is 2000 feet. When they fly into camera, we always had to do something to make them look that big in CG.” For example, Holcomb estimates that modelers and painters altered the Narada on a shot by shot basis 70 percent of the time. “J. J. let us explore the dark aspect of it,” Holcomb says. “We dreamed up solutions and built and painted it accordingly.”

The combination of layout and animation worked so well that the studio is planning to use the same system for future shows. “Animation could have done all the camera work themselves, but I think that having people in layout doing the cameras is very important, so I’m glad Paul brought me and the others into the department,” Benoit says. “We have people on our team who were camera operators on The Empire Strikes Back.”

Even though Benoit hasn’t been a camera operator, after having match-moved the cameras in hundreds of shots while working in the layout department for post-production work on films at ILM since 2002, he has learned more than a few moves.

“[To match-move,] you have to understand exactly what the camera operator is doing with what type of camera,” he says. “Technocrane vs Spydercam vs dolly, pan, tilt. You have to know before you start a match move. I think that doing camera work that blends properly with real footage, going from practical to digital [and] back to practical, is an art form that’s gradually becoming more and more important.”

For Kavanagh, the experiment proved that the studio could promote its previs capability as well as its post-production skills. “We have good artists who aren’t afraid of that daunting blank scene in Maya,” he says. “And, like a lot of studios, we’re looking at how we can make our own previs department cost-effective.”