Friday, March 15, 2013

How
do our brains respond to movies? With the advent of functional magnetic resonance imaging (fMRI), it is now possible to have individuals watch a movie in a
scanner and record the brain regions that are active during the experience
(something that was considered science fiction fantasy only 20 years ago). Brain
circuits have been identified that pertain to various mental processes, such as
vision, memory, language, emotion, and decision making. Of course,
the whole brain must work together to give us the ability to perform complex
operations, such interpreting the plot of a feature film. Mostly likely, when
we watch movies we borrow many of the brain mechanisms we use in
everyday experiences. Indeed, we often are sucked into a dramatic film
almost as if we are in the scenes themselves. Recently, a few brain scientists
have explored our movie experience—from the ways movies drive visual processes
to ways they evoke brain responses associated with feelings. Neuroscientist Uri Hasson has advocated neurocinematics, which has the specific goal of understanding the
neural underpinnings of our movie experience.

Hasson
uses a statistical method called inter-subject correlations (ISC), in which he
calculates the degree to which a movie evokes the same or coherent brain
responses in a group of subjects. Interestingly, not all movies show the same level
of coherent brain activity. In one analysis, a clip from Bang! You're Dead, a TV episodic from Alfred Hitchcock Presents,evoked
coherent brain activity in over 65% of the cortex (green region in figure). A
clip from Sergio Leone's The Good, Bad,
and Ugly elicited coherent activity in 45% of the cortex (blue region).
Less mutual activity was observed in a TV episode of Curb Your Enthusiasm (red region) and a single-shot video clip of
people walking around Washington Square Park near New York University (orange
region)—these two clips offered minimal or no real plotline (Curb Your Enthusiasm is often shot
without a script). These findings suggest that some movies engage comparable brain
processes in all subjects, whereas others are less successful in doing so.

Not
only can neurocienematic studies offer intriguing findings about our movie experience,
they also can help elucidate the neural underpinnings of natural, everyday
viewing. Jack Gallant and colleagues showed
two hours of movie clips to subjects in a fMRI scanner and created a
mathematical model—a kind of dictionary—that translated brain responses in
terms of the textures, edges, and motion depicted at any given moment. Once
this dictionary was established, the experimenters obtained brain responses to other clips that had not been seen before. They then used the dictionary of prior scans to predict or "decode" what subjects were seeing in these new clips. The image on the left is a frame from
one of the test clips (Steve Martin in the 2006 remake of The Pink Panther). The image on the right is a reconstruction of
what the model "thinks" the subject is seeing based on the brain
activity to the clip. Remarkably, the model does an amazing job of decoding
the image purely on the basis of a brain scan!

Other
neuroimaging studies have ventured into conceptual and emotional processes
while viewing movies. Gallant
and colleagues have also developed models from fMRI data that define conceptual
"maps" that can predict what objects people are viewing. The
cognitive neuroscientist Jeff
Zacks has studied the way we segment events as we watch movies. I and
others have looked at brain responses while viewing emotionally laden film
clips (see Shimamura
et al., 2013). I contend, however, that merely recording brain activity while
watching movies is not enough, as it is important to consider the psychological
processes that are defined by such neural activity. We cannot fall into a
modern-day version of phrenology where bumps on the head are replaced by bright
spots on a brain scan. We need to go further and develop theories that describe the
functional dynamics of neural activity and how brain regions interact to enable
us to see, think, and feel. This is why I prefer the term
"psychocinematics" rather than the more restrictive
"neurocinematics" to describe the scientific inquiry into our movie
experience. In the end, we'll need minds, brains, and even more—sociology,
history, anthropology, and other disciplines to gain a thorough understanding of
the magic of movies.

Friday, March 1, 2013

What
makes a good story? Aristotle in The Poeticssuggested that a well-formed plot should be structured
as a complete tale with a beginning, middle, and end. Characters are to be
developed through recognition or realization, such as Luke Skywalker realizing
that Darth Vader is his father. The screenwriting guru Syd Field articulated a
three-act story arc for movies: Act I, the Set-up,
lasts about a quarter of a movie and introduces the characters offering
insights into theirnature, goals, and
predicament.Act II,
Confrontations, is the longest
segment, encompassing the middle half of a movie, which sets the protagonist
off ("the game's afoot!") and builds tension through a series of
challenges and conflicts. Act III, Resolution,
leads to the climax with the last scene serving as a final release of tension. Acts
are separated by plot points, which
move the action and "spins it around in another direction" (Fields, 2005,
pg 26). Since antiquity this storytelling scheme has been successful in
capturing our attention and driving our emotions. In Hollywood movies, it has been followed so rigorously that many consider it excessively formulaic (can a Hollywood action movie end without a final chase
scene?).

The
film theorist Kirsten Thompson analyzed the story arc of over a hundred movies and identified a turning point at the midpoint of many movies, which splits Fields' middle act into
two segments: "The Complicating Action" and "Development." This turning point is marked by a dramatic change in the direction of the
main character's goals and disposition. In Casablanca,
Thompson defined the end of her Act II (The Complicating Action) as the scene
where Ilsa tries to explain to a drunken Rick why she didn't meet him at the
train station in Paris. Rick ends the scene dejected after cruelly dismissing her
story ("Was it Lazlo or were there others in
between…or aren't you the kind that tells"). The scene ends with a fade to black at halftime (51 min into the 102
min movie). In Act III (Development), Rick must turn his former bitterness into
saving Ilsa from a desolate lifetime in Casablanca. With Thompson's four-act structure,
each section is of equal length (about 20-30 min) and marked by a prominent
turning point. These large-scale descriptions are of course only guidelines, and
Thompson has shown that movies may conform to a 3-, 4- or even 5-act structure,
often depending on the length of the movie.

The
narrative structure of a good movie is much like a rollercoaster ride which
begins with anticipation, creates waves of tension and release, and when we
reach the end we are satisfied by a thrilling experience. A movie's rhythmic
pacing involves the varying of tempo with stops and starts, questions and
answers, and successes and failures. Buck Henry, the screenwriter for The Graduate, said: "If it's just a
series of climaxes, you can go crazy. You have to find some way to moderate the
tempo so that it's not all one crescendo, or one diminuendo. There have to be
changes of pace to give the audience time to stop and start again" (Froug, 1999,
pg 209). In this way, the filmmaker flirts with the viewer by concealing and
revealing knowledge as the story progresses. There may be a hierarchy of
tensions and releases with small-scale predicaments woven into the fabric of
larger-scale conflicts and goals.

Cinemetrics,
which pertains to statistical analyses of film features, has been used to
capture the rhythmic pacing of films. Analyses by Barry Salt and James Cutting
have shown both large-scale and small-scale rhythms or waves defined by
clusters of short and long shot lengths. Shown in the graph below is an analysis by Barry Salt of the shot lengths across the movie Ride
Lonesome. Shot lengths varied from 2 to over 20 secs, though it is clear
that the ordering of these edits is not random as there are moments of
rapid cuts and clusters of shots with long durations. The red
line shows a larger-scale averaging of the shot durations which seems to
conform to a 3-act segmentation with each act starting with long shot durations (low points) and followed by a cluster (hump) of rapid shots. As the climax approaches, we
see a distinct series of very rapid shots with the final shots increasing in
duration. James Cutting and colleagues analyzed 150 feature films and showed
that "…the sequence of shot lengths in films has a waxing and waning
structure…that occurs simultaneously at different scales—that is, fluctuations
on the orders of tens of seconds, minutes, tens of minutes, and longer (Cutting et
al., 2013, pg 4).

When
we watch a movie, we are not generally conscious of its rhythmic pacing as we tend
to get sucked into the story and pay attention to the drama. Yet filmmakers are
well aware of the story arc and its rhythmic scheme, though of course not all
movies are successful, as much has to "work" with respect to the
acting, visuals, editing, and most importantly the storyline. A worthy
challenge for psychocinematics is to consider how alterations in the event
structure or shot length rhythm influence our attention and emotional
engagement.