The Sculptural Cinema

A change has taken place in the way some stereographers look at the most basic aspect of stereoscopic composition. This change made its appearance only a few years ago. It provides what could well be described by that tired cliché as a paradigm shift. It’s a new way of looking at the world of stereoscopic composition.

So that we’re all on the same page, I’ll just say a little bit about how people have viewed controlling of the depth effect in stereoscopic images. It is well known and understood, and first enunciated by MacAdam in the SMPTE Journal in the early ‘50s, that the stereoscopic depth effect is weighted or scaled by extra-stereoscopic or monocular depth cues. That is, the depth cues you can see with only one eye tend to strengthen the stereoscopic depth effect. That doesn’t mean that you can’t have a stereoscopic depth effect without monocular cues, as Wheatstone first demonstrated in 1838, or was shown by Julesz in his Foundations of Cyclopean Perception.

The strongest extra-stereoscopic depth cues are perspective and motion parallax as noted by MacAdam (who was an important contributor to Kodak’s color photography efforts). Perspective is easily grasped. I can think of two examples, one of which is the use of a wide angle lens in which the background appears to be farther away than it would have if a long focal length lens was used. A good example of geometric perspective is a shot of railway ties receding to infinity. Another depth cue, as mentioned, is motion parallax, which involves the temporal rejuxtaposition of foreground and background. In this case there are perspective changes with time, and often achieved in cinematography with a traveling shot in which the direction of the lens axis is more or less perpendicular to the direction of camera travel. It doesn’t matter what direction you’re moving in. You can be moving vertically, and you will increase the apparent stereoscopic depth effect of the image using a moving camera.

Te basic parameter setting, without regard to monocular cues, and the most direct way to increase the stereoscopic depth effect, is to change the distance between the cameras, or more properly the camera lenses, so that the interaxial separation is greater. There is an easy and enlightening experiment to do involving a series of images of the same scene gradually increasing the distance between the cameras. In the limiting case when there is no distance between the cameras (i.e. they coincide), the image has no stereoscopic depth effect. It is the job of the stereographer to understand how far apart the cameras (or camera heads) should be placed, based on the distance to the principle object and other objects in the scene, the focal length used, and, as mentioned, the extra-stereoscopic depth cues.

This is the heart of my little article: A relatively recent change in the way stereographers can control the depth in a scene is based on the insight that it is possible to use more than one interaxial spacing for objects in a shot or for foreground and background. I can tell you that it’s a technique that occurred to me years ago in conjunction with the work StereoGraphics was doing with computer generated images, but there was no motive to employ it. We were happy to have application developers simply add stereo drivers to their software and asking them to do more was, I thought, looking for trouble. The first place I read about the idea was in a paper by Nick Holliman given at the SPIE Stereoscopic Displays and Applications Conference, in which Nick suggested changing the distance between the cameras or lenses for different parts of the scene. (It’s also described in U.S. Patent application number 10/596,536 in terms of a specific algorithm for the selection of parameters.)

The most convenient way to do this is with a computer generated image. And indeed this technique has been used, and is being used, by stereoscopic supervisors such as Phil McNally, and Rob Engle. Phil used it on Meet the Robinsons, and Rob used it extensively on the Beowulf. I watched him use it for one shot in particular–the shot where Angelina Jolie appears out of the pool in the cave. I saw several versions of the shot in which she originally appeared covered with scales, in the next with mud, and in the final version with liquid gold dripping from her body (or somebody’s body). In this case, Rob judiciously used different interaxial separations for Angelina and for the cave wall that was behind her.

For CG animation the technique it’s a good idea, because nobody cares about reality–whatever that is. Stereoscopic supervisors who are working in CG have no impediment to having interaxial separations that are different for different parts of the shot. They can use two, three, four, or even more to control the depth of characters and the stereo strength of the backgrounds.

This makes me think of those old stereo cards I’ve seen, where somebody has taken a picture of Niagara Falls, for example, and the left and right images are essentially identical–so when you look through the stereoscope you see a planar image surrounded by a stereo window. If that’s what people think stereo is, fine, but I think that’s not asking for much. That shot of Niagara Falls should have been taken with the cameras placed feet or maybe yards apart, to get a decent 3-D effect. What’s the point of having a 3-D picture if it doesn’t have a stereoscopic effect, unless the lack of depth is the effect sought?

Live action can use the technique too with blue or green screen. In fact, without trying it’s automatically going to happen. Blue screen or green screen is almost always going to be CG originated these days, and the interaxial the director of special effects chose–hopefully in conjunction with the stereographer on the shoot–is not going to be the same as what was used in the camera setup. It won’t be for reasons of scale. I’ve seen recent fun examples of this combination of live action and CG in Journey to the Center of the Earth 3D. In addition, science fiction movies, in order to be satisfying, are going to need camera separations that might be miles or hundreds or thousands of miles apart to get decent depth effects for planets or spaceships.

In addition to these are examples of interaxial separations that can be controlled based on the ability to have multiple stereo-cameras in the shot there is another approach that also tosses out the (stereo) window the notion of a fixed interaxial. The technique that has arisen I’m going to call the “virtual interaxial separation.” For converting (stereoizing or dimensionalizing) planar to 3-D, the technique that’s usually used–and there are really just a couple of them–is to outline the principle object, typically a person, against a background. Roundness is added to the foreground figure by treating it with an operator selected depth map. The hidden or missing background material is filled in by cloning or an artistic endeavor. Philips, in their patents, calls it “plugging,” which is a pretty good term–you’re plugging a hole. Depth maps can be created for the background as well.

What you’re creating is a bas-relief or a false front; and this false front can be manipulated so that the objects are sculpted to be elongated or manipulated in the Z axis. The concept of interaxial may not make a lot of sense because you’re actually creating stereo-pairs independent of cameras and their limitations. Sometimes you might be simply accepting the left view and then creating a new right view, and how that right view is sculpted will determine the depth effect in the shot. Another way to handle conversion is to take a 2-D skin and lay it on a 3-D model, in which case you can also manipulate the Z axis locally and place “cameras” in the space any which way. The stereo strength of objects can be sculpted. This technique has been used in CG animated movies.

There is a concept called orthostereoscopy, and one condition it must fulfill is that the interaxial separation must be equal to the observer’s interpupillary separation (obviously impossible in most cases since peoples’ eyes have different Interocular separations). Other ortho conditions have to do with the geometry of the shot so that the perspective considerations are maintained, with regard to the projected image and the conditions of photography. The concept of orthostereoscopy doesn’t have much importance in entertainment. The creative people in the industry have grasped this and entirely new techniques are being developed that heretofore hadn’t been imagined. There is now, in this the early days of the stereoscopic cinema renaissance, a breakthrough in the concept of stereoscopic composition, making the creation of images more nearly related to sculpture than photography. I’m sure there are more innovations to follow.

One Response to “The Sculptural Cinema”

Dear Lenny
I produced an IMAX 3D film called Bugs! (stereography by Sean Phillips). I have not had the pleasure of meeting you yet, but bump into Josh quite regularly. I love your posts and have started writing a little myself on 3-D! (http://www.definitionmagazine.com/issue_pdfs/def29/3-D_HD.pdf).
In your piece above I noticed you write, “The concept of orthostereoscopy doesn’t have much importance in entertainment. ” Bugs! was an orthostereo film, as are most IMAX 3D films, particularly those shot with the Solido. I think they have great importance in entertainment; they certainly take loads of money at the box office. Did you really mean that?
Hope to see you at NAB – I am doing a 3D primer at 10am on Mondaty morning