Wednesday, May 15, 2013

On a recent trip to Princeton
University, I had the opportunity to visit the Thomas Edison National Historic Park, in West Orange, New Jersey (about an hour drive from the campus). At
this large laboratory complex, Edison invented the Kinetoscope, one of the first commercially successful motion
picture viewers (the one in the photo is from the George Eastman House in Rochester, NY). Edison's interest in moving pictures was sparked in 1888 by a visit
from Eadweard Muybridge who had already built a motion picture projector but
one that could only present a dozen or so images in succession (e.g., a
galloping horse). Edison's device was not a movie projector but instead a large
one-person viewing console in which about 40 ft of 35 mm film strip passed by a peephole.
The film was illuminated by stroboscopic flashes that were produced by a
spinning opaque disk with an open slit that was placed between the film strip
and a lamp. In this way, frames were flashed as instantaneous still images in
rapid succession.

At West Orange, Edison and William
Dickson (the true film expert at Edison's lab) constructed the
"Black Maria," a small studio where short films were shot for the Kinetoscope. Many of Edison's silent "flicks" were shot there, including the first U.S. copyrighted film, a 5-sec clip of an Edison employee, Fred Ott, sneezing. In the spring of 1894 in
New York City, the first movie parlor opened, and people paid a nickel to peer
into a Kinetoscope and watch a short clip (~20 sec) of a vaudeville act, such as a dancer or contortionist. The movie craze thus began, and Kinetoscope
parlors cropped up in cities around the country and across the Atlantic. Yet within a couple of years the
popularity of these peep shows faded as movie projectors were invented which could present the 35 mm films to large audiences.

It was of course the remarkable
depiction of naturalistic movement that enthralled these early moviegoers
as it does so today. The “illusion” of seeing movement from the rapid succession of
still images, or what psychologists call apparent motion, is based on a not completely understood set of processes studied early on by Max Wertheimer, the German Gestalt psychologist. In 1912, Wertheimer published experiments in which a vertical line is followed a
horizontal line at various lag times between presentations. If the lag time (or
inter-stimulus interval) was very fast (less than 3/100 sec) then the viewer perceived
both lines simultaneously, which appeared to form a right angle. In this
case, visual persistence of the first line overlapped with the presentation of
the second line so that the two were perceived as being presented
simultaneously. If the lag between stimuli was long (greater than 1/5 sec)
then the viewer simply saw two separate lines presented sequentially. However,
between these two lag times, particularly around 1/20 sec (or 50 msec), the
vertical line looked as if it moved and swung down to a horizontal position.

The perceptual mechanisms
underlying apparent motion are multifaceted and still studied by
psychologists and neuroscientist. Some brain regions are particularly sensitive
to movement, both real and apparent. In fact, at the initial entry point where
visual input enters the cortex—an area identified as V1—some neurons are activated when objects are moving at a specific direction. From V1, two main paths
process visual information. The dorsal path courses up to the parietal cortex
and is involved in spatial processing ("where" things are). The
ventral path courses down into the temporal cortex and is involved in object
processing ("what" things are). For example, regions in the inferotemporal (IT) cortex become active when we recognize objects, such as faces, cars, and furniture. The ventral and dorsal paths work together to integrate visual information allowing us to recognize objects placed in a spatial environment.

In the cortex, two regions
are particularly involved in motion perception. The first, MT (middle temporal region, also called MT+ or V5) is sensitive to directional
movement of a pattern or object, such as viewing a car moving across an
intersection. In an fMRI study, individuals watched two squares
blinking on and off with the lag time between presentations varying between 50
and 62 ms. Recall that in the Wertheimer study such time lags are often
perceived as a single object moving—in this case a square appearing to move from left
to right. With longer lags the apparent motion disappears, and the two squares simply
appear to blink on and off in sequence. When individuals perceived a square
to move from one position to the other, area MT was particularly active. The STS
(superior temporal sulcus) is another motion-related region. It becomes active
when individuals perceive bodily movements, including moving torsos, heads, and
even eyes.

It is still amazing to sit in a darkened
theater and experience an action-packed Hollywood blockbuster on the big screen. As we watch such
movies, multiple brain regions allow us to interpret movement from these
flashing still images and "see" objects flying around or "feel" as
if we are moving in the scene. With the advent of digital animation and other
forms of computer-generated imagery, filmmakers have developed graphic
techniques that make the completely artificial appear as natural as if we were
viewing a scene outside our house. The analysis of various methods of digitally
constructing realistic scenes, such as rendering shading and perspective and
mimicking natural movements through subtle blurring of motion, might help scientists understand better how we perceive the real
world. Natural viewing is somewhat akin to viewing a stop-action movie, as we are always fixating and then moving our eyes (about three
times a second), and each time we do so, it is as if we capture a single-shot frame
of the world. Thus, as much as science might help us understand
filmmaking, filmmaking itself might help us understand how the brain works.

Wednesday, May 1, 2013

Walter Murch, the preeminent
editor of such movies as Apocalypse Now
(1979), The Unbearable Lightness of Being
(1988), The English Patient (1996),
and Cold Mountain (2003), wrote an
incisive book on film editing called In the Blink
of an Eye. In it, he said that
editors must be preoccupied with "misdirection" and keep in mind the
following questions: "What is the audience going to be thinking at any
particular moment? Where are they going to be looking? What do you want them to
think about? And, of course, what do you want them to feel?" (Murch, 2001,
page 21). In the fast-paced rhythm of current Hollywood blockbusters where cuts
are interspersed every 3-5 seconds, film editors must ask themselves these
questions about 1200 times per movie.

The essential skill of a film
editor is to create shot transitions that are so smooth that we are blind to
them. In this way, outstanding editors keep us from being aware of their own
craft. We are familiar with great actors, directors, and even producers, but how
many award-winning editors do you know? (At least you now know one—Walter Murch.)
In my mind, film editing defines in large part what makes movies distinct from other
art forms. Murch's description of the editor's role is much like that of a
psychologist who must be attuned to the thoughts and feelings of others. His
notion of "misdirection" resembles the ploy of magicians who also must
be aware of the viewer's attention and thoughts.

The psychologist Tim Smith
has conducted some of the best research in psychocinematics. By studying eye
movement behavior, he has shown that movies are extraordinarily potent stimuli
in driving and directing our attention. As we view the world (including movies),
our eyes fixate on a spot for about a third of a second and then jump to
another spot, fixate, then jump again, and so on. These jumps are called
saccadic eye movements, which are rapid, taking only 3/1000 of second to
complete. Occasionally our eyes will track a moving object, but about 95%
of the time they are motionless and fixated at a point in space. During
any fixation, you can only clearly see a relatively small focused area, which
amounts to a span of three or four words as you read this text. Thus, the position
and sequence of eye movements offer a record of what your attention is focused
on at any given moment.

Smith and colleagues recorded
eye fixations of groups of subjects while they watched clips of feature films,
such Blade Runner and There Will Be Blood. When they analyzed
the data what became evident was that the subjects all tended to fixate on
the same spot and move in synch from moment to moment. The figure below shows a
frame from the movie There Will Be
Blood with the circles representing where subjects were fixated at the
moment of viewing that frame. The cluster of circles shows that there was a
strong overlap or coherence of eye fixations among subjects as they watched the
movie. Smith refers to this gaze attraction as attentional synchrony. In fact, as the subjects watched a film these clusters seemed to move together as if all were being guided by the
moving images (for a video demo see Tim
Smith's blog).

As with Uri Hasson's fMRI analysis of coherent brain activity while watching movies (see March
15 blog), Smith has shown that filmmakers have found ways to control our attentional
focus as we follow the plot. Thus, as the Murch quote above suggests, a good filmmaker
understands the cognitive demands of viewing movies and knows how to guide the
viewer so as to drive the storytelling in a forceful manner both perceptually
and emotionally. In some respect we allow the filmmaker to guide and move us, as if we are on a fantastic vehicle that transports us through
the movie. When the plot is riveting, we all seem to move together and enjoy
the same magical ride.