A Brief History of Motion-Capture in the Movies

Share.

The evolution of the technology examined.

By Ali Gray

Motion-capture – or 'performance-capture', to give the method the actor-friendly name it's more recently come to be known by – has now become the norm for the contemporary blockbuster. We barely bat an eyelid at a fully-rendered CG character these days, so ubiquitous are they as to remove all mystery of how they were created. As evidenced by the flawless, lifelike animation we see on screen today, motion-capture has evolved at a frightening rate over the last few decades, which is fitting, as it reaches previously inconceivable highs in Dawn Of The Planet Of The Apes this month, with entire simian armies animated using the technique. But how did motion-capture get off the ground, and which films – and filmmakers – do we have to thank for its existence?

You probably know motion-capture as the thing with the ping pong balls and the silly lycra suits. Well, you're not wrong. In actual fact, rudimentary motion-studies have been around since the 19th century, pre-dating cinema itself, and were primarily used for understanding better how humans and animals move. Late in the 20th century, before it was co-opted by Hollywood, motion-capture was a photogrammetric analysis tool in the field of biomechanics, and was used for sports, education and even videogames like Prince Of Persia.

The basics of motion-capture are thus: you have real participants provide a template of human movement and translate that movement into another medium. The earliest example of the entertainment industry using motion-capture is probably the short films of animator Max Fleischer. In 1915, he devised the Rotoscope, a concept which saw animators trace over frames of live-action film to give cartoon characters a recognisable fluidity, if not a photo-realistic look. Fleischer's series Out Of The Inkwell utilised this method, including animated classics like Koko the Clown and Fitz the Dog.

Mo-cap, Prince of Persia-style.

Walt Disney liked what he saw. Looking for new and exciting ways to bring their characters to life, Disney adopted rotoscoping – also known as live-action reference – and began to experiment using it in motion pictures. In 1937, the company released their first feature-length movie, Snow White And The Seven Dwarfs, which was animated by shooting actors playing out the parts and then tracing over the motion they provided. It was a smash hit, and Disney used rotoscoping for several other features including Peter Pan, Alice In Wonderland and Sleeping Beauty.

Motion-capture didn't really advance much over the subsequent decades, although rotoscoping never really went out of fashion. In 1978, Ralph Bakshi used the technique to animate his vastly truncated Lord Of The Rings movie and followed it up with American Pop (1981) and Cool World (1992), starring Brad Pitt opposite an animated Kim Basinger. In the 21st century, director Richard Linklater adopted the rotoscope technique in 2001's experimental drama Waking Life (each frame of reference animated digitally) and again in A Scanner Darkly in 2006. But by this point, motion-capture had found a new lease of life.

You may not have heard of Sinbad: Beyond The Veil Of Mists, but it's an important milestone in the history of animated film: it's the first movie made primarily with motion-capture. Though the film did utilise traditional animation in places, most of the character movement was filmed with a groundbreaking 3D optical technique, whereby actors played out their roles on camera covered in dozens of tiny reference markers, triangulating their movements on a computer. By the time the hundreds of Indian animators had finished the film for release in 2000 it was old news (it grossed under $30,000 domestically), but it blazed a trail for films to come.

Hollywood quickly picked up on this method of optical motion-capture and the results it provided. Expensive and prohibitive, MoCap was used sparingly at first. Arnold Vosloo wore tracking lights for certain special effects shots in The Mummy ("A lot of the time I was walking around the set looking like a Christmas tree," he said), while Ridley Scott motion-captured 2,000 extras for Gladiator and replicated them en masse for realistic looking crowd scenes. Motion-capture was fast becoming an essential tool in the forward-looking filmmaker's arsenal. George Lucas gambled on motion-capture in 1999, with Ahmed Best providing the movements for Jar-Jar Binks in Star Wars: Episode I – The Phantom Menace in a "tight scuba suit" over which animators painted the Gungan gimp. The first ever fully digital main character in cinema history was also perhaps the most hated film character in cinema history, but that didn't change the fact that motion-capture was fast becoming an essential tool in the forward-looking filmmaker's arsenal.

Arnold Vosloo in The Mummy.

You have to feel for the creators of Final Fantasy: The Spirits Within. You suspect the entire industry was watching Square Pictures to see if their feature-length, photo-real, motion-captured, computer-animated epic would be a hit or a miss. Sadly it was the latter, and though Square Pictures claimed they could change the face of cinema forever with 'CG actors' like Aki Ross – who Square planned to cast in more movies in other roles – Final Fantasy: The Spirits Within crashed and burned in 2001, taking $137 million of the company's money with it. Nonetheless, it was a benchmark film in the evolution of motion-capture: 1,327 live-action scenes were filmed and subsequently mapped into digital animation. Even the film's harshest critics had to admit the visuals were jaw-dropping. But the best was yet to come.

That same year, Peter Jackson teased the appearance of Gollum in his first Lord Of The Rings movie, The Fellowship Of The Ring. By the time Gollum was ready to make his full movie debut in 2002, Jackson had taken motion-capture technology to the next level: The Two Towers was the first film to use real-time, full-on performance-capture, so the actor – still dressed in this season's least flattering lycra and ping pong couture – could be on the set, outdoors, with other actors, in the moment.

That crucial difference was key: no longer did actors have to provide motion-capture at a later date and a separate location, far removed from the action – they could inhabit their role as they would if they were flesh and blood. Andy Serkis' performance was part of why the Lord Of The Rings movies were held in such high regard, as was WETA's mastery of the technique.

Serkis, of course, would go on to become a pioneer in the process, co-founding his own studio The Imaginarium and cornering the market in CG characters who needed a human touch – he later re-teamed with Jackson to give King Kong his swing. Meanwhile, other filmmakers were realising the limitless benefits of MoCap: Ang Lee even donned a lycra spacesuit and provided select scenes of motion-capture for Hulk himself. Suddenly it didn't seem so silly.

Andy Serkis as Gollum in The Two Towers.

Director Robert Zemeckis took the idea and ran with it – indeed he ran and ran until he couldn't see conventional filmmaking any more. Where Final Fantasy: The Spirits Within failed, he succeeded – Christmas fable The Polar Express was filmed using new performance-capture techniques and, thanks to a box-office tally boosted by 3D showings at IMAX, was a festive hit. Subsequent films from Zemeckis' studio ImageMovers, however, were met with less enthusiastic responses from audiences: Beowulf flopped, while Jim Carrey vehicle A Christmas Carol couldn't repeat the success of The Polar Express. When Mars Needs Moms bombed spectacularly, Zemeckis cancelled his proposed animated Beatles vehicle Yellow Submarine and quietly disassembled ImageMovers for good.

The reason many believe that audiences reacted poorly to Zemeckis' later movies was something called the 'uncanny valley' – a state reached when animated movies look almost human... but not quite human enough. The brain is tricked by photo-real visuals and recognisable movement, but one small element gives up the effect, making the image unappealing to the viewers. (Hint: it's almost always the 'dead' eyes).

While the photo-real approach to animation was short-lived, motion-capture was being put to good use in good old-fashioned cartoon animation. In 2006, two of the three nominees for Best Animated Feature at the Academy Awards used motion-capture: winner Happy Feet and Zemeckis-produced animation Monster House were MoCapped, while Pixar's Cars was not. Pixar have never been great fans of the technique, preferring instead to let their animators use instincts to inform their art instead of raw data. The credits for 2007's Ratatouille proudly featured the claim "100% Pure Animation – No Motion Capture!"

As has so often proved to be the case, James Cameron changed the game. The director waited for over a decade for technology to catch up with his vision. Cameron pioneered a new system he called his 'virtual camera', a brand new way to shoot. While actors provided the movement in Cameron's own MoCap playroom (nicknamed 'The Vault') and 'performance-capture' mapped his cast's slightest facial expressions, the director was able to see their CG counterparts streamed live on his monitor, inhabiting the entirely digital world of Pandora. Now you didn't even need an outside: Cameron called the virtual camera a "form of pure creation". Even Steven Spielberg was wowed by the tech, later half-inching it for his own animated feature, The Adventures Of Tintin. Combining the best bits of live-action performance and digital artistry, James Cameron's Avatar went on to gross more money than any other flm in box office history.

Sigourney Weaver and Sam worthington get their Avatar mo-cap on.

The technique didn't truly evolve, however, until Rise Of The Planet Of The Apes in 2011. The simian character of Caesar – arguably the movie's lead character – was the most ambitious use of the technology yet. But with Andy Serkis in full monkey mode and cutting-edge tech squeezing every single nuance from his performance, the Apes prequel came closer than ever to pulling off the impossible: it almost made you forget you were looking at a collection of ones and zeroes. Serkis' turn as Caesar was so well received, there were even calls from some quarters for motion-captured performances to be officially recognised at the Academy Awards.

Serkis continues to be the major player in mo-cap circles, returning to the role of Caesar for Dawn Of The Planet Of The Apes this month but also landing work as Godzilla in Gareth Edwards' reboot and unspecified roles in uber-blockbusters Avengers: Age Of Ultron and Star Wars: Episode VII. Performance-capture is now a method so widely accepted, it's become the norm – even bizarre instances like Benedict Cumberbatch (very much a biped) providing MoCap performance for Smaug (very much a dragon) in the second Hobbit movie didn't raise too many eyebrows.

Performance-capture has finally found the sweet spot in providing animated characters with real human input. Once upon a time, Hollywood's acting community feared they would one day be replaced by fully subservient digital counterparts, who could be made to sing and dance and sell any old piece of tat by boffins with computers. Modern motion-capture proves that may very well one day be the case, but there's a silver lining: those actors will still be able to find work strapping on the ping pong balls if they so choose.