How to create Oscar-winning VFX

Ben Grossmann reveals how artistic use of stereoscopic 3D – enhanced with restrained use of VFX – can envelop an audience in a story.

Pixomondo's VFX supervisor for Hugo, Ben Grossmann, reveals how effects used for beauty and to subtly affect the audience's emotions took this year's VFX Oscar - and how artistic use of stereoscopic 3D can envelop an audience in a story.

At the Oscars last month, Martin Scorsese's Hugo picked up the prestigious Visual Effects gong - as well as awards for Art Direction, Cinematography, Sound Editing and Sound Mixing. The movie is an enchanting tale of a child who lives in a train station in Paris fixing clocks and who goes on to connect with one of the pioneers of early filmmaking and special effects, Georges Mlis. Its creators indulged in a huge amount of experimenting of their own - this being Scorsese's first kids' film and first foray into stereoscopic 3D.

Key to bringing to life the golden hues of the train station, where most of the film is set, were the visual effects created by VFX group Pixomondo across its studios in London, Los Angeles, Toronto, Germany and China. While Hugo's rivals for the VFX Oscar - Harry Potter and the Deathly Hallows Part 2 , Real Steel, Rise of the Planet of the Apes and Transformers: Dark of the Moon - used visual effects primarily to add excitement, Scorsese's film used them to inspire a much wider range of emotions and wonder, and to subtly enhance the interplay between characters.

We sat down with Pixomondo's VFX supervisor Ben Grossmann (below) to discover how they tapped into the spirit of the film's subject to produce a tribute to cinema's pioneers.

DA: How did it feel to win an Oscar?

BG: "It was quite a surprise, as [the contenders] were such a great body of work. Harry Potter represents work that's gone on for over 10 years. We really expected the Academy to give some recognition [to them]. Any of the other films deserved to win on different merits."

DA: What was your stylistic approach to the VFX in Hugo?

BG: "It started out with a concept and we tried to weave it into all of the visual effects work in the film. We weren't going to showcase new technology for performance capture [as in Rise of the Planet of the Apes] or create these amazing hero characters [Transformers..., Real Steel] - it was more adding [to] the tone of the film.

"To us in visual effects, it was about one of the leading sources of innovation in the world of visual effects - the magician Georges Mlis.

"We tried to take that opportunity to have our work here represent something greater than just the story, or creating characters or all the things visual effects normally do. Our work needs to pay tribute to this visual effects pioneer, not just visually, but also through our approach to technique."

DA: Did you feel you were a pioneer in the same way as Mlis?

BG: "Because we were exploring this new realm, we inherited many of the same characteristics as the original filmmakers where we would wander around and try things. If that didn't work, we'd try something else.

"And then we'd make it better, and we'd mix it with a little of something else - until finally we'd push it off in a new direction, and get a new emotional response from the audience. It wasn't like we were masters of the craft, we were learning as we went - just like the Lumire Brothers and Mlis."

DA: Can you be subtle with stereoscopic 3D?

BG: "Yes, but chances are you didn't notice it. There are some big spectacle shots where you're very aware we've created something you're supposed to look at. But there's so much that can be done subconsciously. By changing the stereo values [to alter the apparent distance between characters], I can push that actor away from you, so that you feel there's some distance between, for example, a shot of Mlis and a shot of Hugo.

"We can separate the characters psychologically so that despite them not having changed positions - they're still only five feet away from each other and the lenses haven't changed - you can make them feel there is this immense separation between them. At a moment where Georges and Hugo become close to each other, you can physically bring them closer together in 3D, so the audience reacts to that.

"We did this throughout this movie,[such as] in scenes where Hugo was looking through the windows at things that were happening in the station. [This was] in order to make Hugo feel like he was isolated, like he was watching another world take place: the separation between himself and society as represented by the train station. We'd push the train station really far away so that the viewer felt this distance as if they were empathising better.

"In another famous scene, the head of the station inspector [above, played by Sacha Baron Cohen, channeling 'Allo 'Allo's Officer Crabtree, Ed] becomes very prominent as he leans in toward Hugo, and [appears to come] off the screen. We did that to give the sensation that he was encroaching into Hugo's space.

"Again, that shot was the tip of the iceberg. Most of the times we used these techniques in the movie, it happened 'under the water'. It was very interesting to see a filmmaker take that subconscious approach and say 'what can we do with stereoscopic filmmaking?', and use these techniques to heighten or enhance the emotional reaction of the audience to the story we've created, rather than just creating visuals.

DA: Do you require VFX for this?

BG: "Half of the [subconscious stereoscopic effects] you could do with no visual effects. Some things you can't do in camera.

"At the end of the film, Mlis is at the gala giving a speech. He's talking about how his reappearance was the result of the hard work of just one boy. Slowly over the course of that shot, it goes from a normal [stereo] 3D shot to a hyper-exaggerated shot where Georges comes so far off the screen that he's almost in the audience looking right at you. You forget for a moment that you're in a movie theatre.

"We tested that shot for months across various incarnations. Our goal was to use a variety of techniques including darkening down the edges of the frame to create a new perceptual view. By reprojecting George's texture onto newly create geometry, we were able to create a camera rig with visual effects that couldn't be done in the real world. It allowed us to make the audience seem extremely small and George seem extremely large.

"By slowly animating him off the screen over the course of very long shots, we could pull him far into the audience."

"[We need] to give filmmakers the ability to see them while they are shooting, so they can take this into account. It's difficult to imagine what's on a greenscreen while concentrating on an actor's performance. There's a tendency to frame the camera for what you can see, because the brain does not subconsciously recognise that the greenscreen is actually going to be the rooftops of Paris with the Eiffel Tower in the background and clouds rolling.

"For Hugo, with the help of [London-based pre-viz company] Nvisage, we set up a system that allowed us to composite approximate versions of scenes with the live footage while we were shooting. We had a motion-tracking system on-set, so that whenever Bob Richardson moved the camera, that information was feeding into a 3D computer that would create a pre-viz quality version of the background, and it would key out the greenscreen and composite the two elements and feed it on a monitor to Marty.

[By] incorporating a crude layout of Paris with the Eiffel Tower on set, I have given the filmmaker an opportunity to change their composition subconsciously. They will intuitively find a better shot.

"The same thing is true in editorial. Many films are edited with the greenscreens intact, and then turned over as locked edits to visual effects [teams]. And the editor will only go in and change the edit [if problems occur]. We never attempted it that way. In Hugo, the editor created a rough assembly of approximate shots and we would do temp visual effects on the entirety of each take used.

"For a shot in the movie that would be only 10 seconds long, the take might be 45 seconds long - or a minute, or two minutes. We would [create VFX for] the entire shot, give it back to them as a piece of footage so that they could start editing without the constraint of having just a little portion of it.

"It changes the way that they edit. Editors say 'I didn't like this particular part of the performance because it seemed oddly composed - as it seemed to just be a sea of greenscreen. Now that I see that the visual effects team have put a train back there and I can see people working on the train; and now that scene makes so much more sense. So I change the edit to accommodate that.'"

DA: What are you most proud of with Hugo?

BG: "If I was to pick out a specific shot or scene: the second shot of the film that starts out high and wide over the back of the station and slowly swoops down between the trains and the people, through the smoke and up into the clock. As a single achievement in the film, that's the one I'm most proud of being a part of. It was such a huge technological and creative challenge. There were staggering logistics in terms of what we had to do to make that one shot happen. The shot took us over a year just to get the first version of it done.

"Overall though, the thing that was the most satisfying to me - given that everything in Hugo felt like an experiment, and it felt like there was so much more we could have done to the point that we never psychologically felt like anything was completely done - was that everything felt unified. It didn't feel like a work in progress when we'd finished it."