Night at the Museum (opening Dec. 22 from Twentieth Century Fox) offers one of the most diverse arrays of creatures and characters ever put on screen. In less than two hours, director Shawn Levy brings most of the display artifacts at New Yorks National Museum of History to life. This includes three-inch Diorama characters, a T-Rex skeleton, Egyptian statues, Easter Island rock heads, wax figures, Neanderthals, mummies, dozens of African mammals, a mammoth and even a full-size whale. Overseen by vfx supervisor Jim Rygiel and VFX producer Ellen Somers, the visual effects achievement was deemed spectacular enough for the movie to be included on the VFX Bakeoff short list.

Responsible for more than 300 shots, Rhythm & Hues was the lead vendor on the demanding project. VFX supervisor Dan DeLeeuw was impressed by the wide range of effects technique that the movie required: We had major character animation with the T-Rex and the African mammals stampede, tricky compositing with the miniature Diorama people and complex crowd simulation with the Diorama battle scenes, not counting dozen of additional effects. The T-Rex was undoubtedly the show stealer!

Taming The Show Stealer

VFX producer Julie DAntoni, animation director Craig Talmy, digital supervisor Will Telford, compositing supervisor Craig Seitz and their team started by building a CG replica of the full-size T-Rex skeleton that had been used on set. Gentle Giant performed Lidar scans of the prop from multiple angles to provide a three dimensional data set. Modeling and texturing the T-Rex turned out to be a huge endeavor as each piece had to be treated individually. We took photographic references of every single bone and joint, and then applied these textures to each corresponding bone and joint geometry, DeLeeuw comments. It was very time-consuming. Rigging the model was also tricky, as we were obliged to make it work as a real skeleton. Interpenetration was a big problem. The bones on the spine basically moved against each other, and each vertebra wanted to go inside of each other. So, special care had to be taken in this area. The joints were based on how the actual prop was built, with metal pieces holding the bones together. We used those pieces to rotate the limbs.

Animation required a lot of tweaking, as this was not supposed to be a living, breathing dinosaur, but a mere skeleton brought to life by a magic spell. The team started by doing tests in which the T-Rex had the weight of its skeleton. Since the creature was much lighter than its fully fleshed and muscled counterpart, it should logically move much faster. However, this reality-based approach didnt provide a pleasing animation. Something was missing. So, using a great deal of artistic license -- this was a comedy, after all -- the T-Rex was eventually animated as if it had all its flesh and muscles on. Right away, the animation was much more effective, DeLeeuw says. Anyway, absolute reality was never our main goal here, since later on, the T-Rex start acting like a dog: it turns out that it just wants to play! So, our lightweight creature ended up being animated just like a very heavy dinosaur We noticed that the shots even worked better when we added a camera shake each time the feet touched the ground. Its absurd, when you think of it, but thats kind of the joke of it.

Modeled in Maya, the T-Rex was imported into Rhythm & Hues proprietary package Voodoo where it was rigged, animated, and lit. It was then brought into proprietary tool Lighthouse where light and color values were assigned. The information was then combined with the Voodoo lighting, and exported to Rhythm & Hues in-house render engine, Wren. Compositing was carried out in proprietary Icy. When we rendered the images, we broke out all the lights in different passes, DeLeeuw explains. It allowed us to mix the lights in the compositor directly, and get a better integration of the character. HDRI was gathered on set to record the lighting set-up and to provide texture maps. In order to give a sense of scale, the creature was always lit with multiple lights.

replace_caption_night02_NightAtTheMuseum-39.jpg

Creating Three-Inch Tall Characters

The next big challenge for Rhythm & Hues was bringing hundreds of Diorama characters to life in a believable way. For tight shots, the actors were shot on a green screen stage, and combined -- at a reduced scale -- with a full size environment. For the wider shots, the team used two different techniques. When the action was minimal, such as railroad workers laying out tracks, or characters walking around, we used CG animation with motion capture, DeLeeuw explains. Wide shots featuring large numbers of characters fighting each other were created with Massive. In order to create CG characters that perfectly matched the live-action actors, we had Gentle Giant cyberscan the main actors, but also 40 different extras. It allowed us to cut from tight shot to wide shot, and still have the same body proportions on the extras. Romans were easy to create as they were based on two different models, the Centurion and the Archer. The costume and the props were more or less the same on all the soldiers; we simply modified the mass and height. The same was done for the Mayans and their loincloth. The Cowboy world was far more difficult as all the characters had different costumes. In the end, we built about 80 costumes for the three Dioramas. Sub-surface scattering was utilized on all the human characters, as well as on CG animals.

A key aspect of the Diorama sequences was making Owen Wilson and Steve Coogan look three inches tall. Due to constant improvisation of the comedians, it turned out to be impossible to shoot background plates first, and then have the performance match that footage. DeLeeuw and his team managed to shoot adequate background plates after the principals had been photographed. Correct eyelines were obtained by placing Ben Stiller -- or the director himself -- high above the greenscreen stage on a platform. Coogan, Wilson and the extras could then focus on the proper point in space, and make real eye contact with their partner. Precise measurements were taken on the set with the plan to have them scaled up -- or down -- for the other part of the shoot. A lot of time, though, the crew made it up on the spot, and established the proportions simply by eye, with a video mix.

During background plate -- or still -- photography, the team shot with multiple focus points that were later blended together. At the compositing stage, it allowed us to decide where we wanted to set the depth of field, DeLeeuw notes. Whether people know it or not, a shallow depth of field does imply something small. It then became a question of trying different depths of field and seeing what looked good on a shot per shot basis. There was no rule. Although realistic, it just wasnt interesting to have Owen Wilson with a giant blur behind him.

For the stampede sequence, Rhythm & Hues was asked to create dozens of African mammals rushing through the Museum and out to Central Park. The software pipeline was identical to the one that had been established for the T-Rex sequence, except that in-house Technical Animation provided an extra animation pass with the muscle, fat and hair dynamics simulations.

Virtual sets and 3D character animation combine to create the Jackal sequences. Image courtesy of Rainmaker.

When Egyptian Jackals Come to Life

Several additional vendors worked alongside Rhythm & Hues to complete the project. Among them, Rainmaker was responsible for 38 shots across four different scenes. The team was led by vfx supervisors Bruce Woloshyn and Randall Rosa, vfx producer Jinnie Pak, CG supervisor Geoff Hancock, compositing supervisor Colin Liggett and lead character animator Mark Pullyblank.

A large part of the assignment involved bringing two giant Jackal statues in the Egyptian Hall to life. The statues were modeled in LightWave, utilizing a 3D Lidar scan by Gentle Giant Studios of the practical set pieces as a reference. The completed LightWave model was then converted to Maya, where a fully articulated animation rig was attached. This rig allowed for complex shifts in the forward and inverse kinematics of the limbs, as the creatures manipulated their spears in both hands, Woloshyn says. The process of rigging and skinning the Jackals included finding just the right balance between a believably rigid stone, and enough flexibility in the joints to allow for the wide range of motion dictated by the specific shots.

A laser scanned survey of the Egyptian Hall set was used to ensure an accurate projection of the HDRI onto the virtual CG set for use with mental rays illumination methods and spatially accurate reflections in the gold portions of the statues. Multiple passes of the Jackals were rendered in EXR floating point format to separate nearly all of the components of the mental ray shaders, thus allowing the compositing team a great deal of control in adjusting the various shader attributes.

The base animation of the Jackals cloth skirts were simulated using the Syflex cloth simulator, and required finding a balance between the carved rigidity of the statues and a fluid folding, Woloshyn adds. Once the simulations were complete, the additional keyframe motion of the skirts codpiece was completed by the animation team and then integrated into the completed scene file. The Jackals headdress was keyframe animated on a rig built in Maya for exact control of each individual key-- or specific flexible piece.

Rainmaker also worked on a scene involving Diorama characters. The elements were shot as a series of greenscreen foreground plates with the soon-to-be tiny actors, that would then be scaled and integrated with both filmed 35mm background plates and high-resolution digital stills. A full size set of the museums exterior was built indoors in Vancouver, Woloshyn explains. So, there was a great deal of control in shooting the very scaled background plates required for our tiny people. In some cases, 35mm photography on a motion control base was required; in others, high-resolution (4K) digital stills were used as a base to extend the scope and scale of the already large set. There was, however, the question of adding snow effects to match the practical snow being dropped on the set. We did this by creating a series of snow curtain particle emitters in Maya. By producing the falling snow with a series of separately rendered curtains in z-space, we gave the compositing team a lot of control when they adjusted the depth-of-field. This was to be very important when it came to preserving the tiny scale of the Dioramas.

When shooting the background plates, exacting on-set measurements were taken of each set-up to determine the precise placement of the camera in relation to the set. Knowing these measurements, allowed Rainmaker to scale them when it came to placing the camera on the greenscreen stage. Utilizing on-set playback, complete with basic compositing for mixing and matching plates, the team was then able to precisely position the camera to match the previously photographed backgrounds. Set measurements also allowed for the placement of set markers on the greenscreen stage for the actors, thus maintaining consistent eyelines. To aid in keeping the lighting down to the scale of the Dioramas, we did a great deal of highlight isolation on the greenscreen elements in compositing. By reducing the amount of specularity in things like the armor of Octavius and the other Roman soldiers, we were better able to preserve their tiny scale as we added them to the full size background plates.

The Rock Head That Talked

On their part, The Orphanage was responsible for completing 20 shots. The core team included vfx supervisor Kevin Baillie, vfx senior producer Paul Hettler, vfx producer Leslie Valentino, CG supervisor Jonathan Harman and tds Alex Wang and Renee Binkowski. The assignment involved turning a 12-foot tall Easter Island solid rock head into a talking character. Using Maya for animation and shader management, along with mental ray for rendering, we concocted an elaborate animated cracking system, Baillie explains. This allowed the artists to create moving fissures and chunks of rock shifting as the mouth moved. The cracking enabled the animators and lighters to all but do away with the rubbery or fleshy feel typical of traditional animated characters. In several shots, the head blows an enormous chewing gum bubble. For this effect, we once again employed Maya and mental ray, but added the Syflex cloth simulation tools into the mix to get the stretchy bubble gum-like movement out of the surface. Final touches and plate integration were completed in the compositing phase using Digital Fusion.

There She Blows!

At Weta Digital, vfx supervisor Eric Saindon and vfx producer Eileen Moran were asked to bring a display whale to life. Modeler Marco Revelant first used the existing display whale as the starting point to build the model in Maya. He then utilized Mudbox to sculpt in more realistic details, such as baleen and displacement detail. The team employed a new volume preserving dynamic muscle solution for skinning the whale, which was rendered in RenderMan and composited in Shake.

Not The Same Old T-Rex

I think we really added a lot of production value to the movie, R&Hs DeLeeuw concludes. The fun aspect of this project was certainly the complexity of each part, but then also the large variety of effects. It was not just the same old T-Rex that youve seen over and over again! So, it was a great experience to work on a production where there were so many different things to do.

Alain Bielik is the founder and editor of renowned effects magazine S.F.X, published in France since 1991. He also contributes to various French publications and occasionally to Cinefex. Last year, he organized a major special effects exhibition at the Musée International de la Miniature in Lyon, France.