The Critical Angle: Can a computer tell why we like Marvel movies more?

By using just about any metric you can come up with, the Marvel Cinematic Universe is quantitatively more successful than the DC Extended Universe, with greater overall box office receipts, better budget-to-sales ratios, critical reception, etc.

It’s even more successful in potentially made up methods.

Earlier this week, marketing website Adweek reported on a study done by a company called ZappiStore, which measured the facial expressions of people viewing MCU and DCEU movie trailers in an attempt to determine their “emotional engagement” to the properties. The test also asked respondents how much they liked the trailers and how likely they’d be to share them, with Marvel’s consistently coming out on top.

The study was conducted using an MIT-born (so it must be good, right?) technology called Affectiva, which claims to be able to detect and identify people’s emotions by measuring their facial expressions. The technique also purports to somehow gauge a person’s heart rate using nothing but a webcam — by simply looking at color changes in a person’s face. ZappiStore says test subjects reacted well to DC’s visual effects, whereas Marvel scored high marks with the characters and their humor, making viewers more emotionally engaged.

I really connect with the film’s … villain? No? Are you sure?

Okay, let’s get the simple stuff out of the way — even if Affectiva can do what its purveyors say it can do, doesn’t applying it to movie trailers only really show that people like those trailers more? A trailer is not an entire film, and ZappiStore apparently makes no attempt to justify this extrapolation. Everyone’s seen a good trailer for a movie that ultimately sucked. Remember the previews for Suicide Squad? And how that turned out? Yeah.

Focus groups in general are known to be problematic, since they’re done on a small sample size and are often not repeated. It’s rumored that focus groups asked for more Loki in Thor: The Dark World, and while it was nice Marvel Studios added that, it was seemingly at the expense of the film’s villain, leading to what might be Marvel’s worst big screen outing yet. Guardians of the Galaxy Vol. 2 made waves before it was released due to a “perfect score” among test audiences, yet in hindsight, few would rate it above the franchise’s debut installment.

Another necessary difficulty in such studies is the (un)reliability of self-reporting, which Affectiva has, if nothing else, seemed to mitigate. But are its third party assessments really any more reliable?

We don’t know, since the algorithm that Affectiva uses is not public knowledge. There’s really no way to figure it out either, MIT Ph.D. student Jonathan Frankle told Undark, because Affectiva utilizes “neural net” processing, in which nodes give feedback to each other, rather than executing a simple code. Data goes in, info comes out, and there’s not much of a way to question it.

Here’s what we do know: Using people’s facial features to tell us something about them isn’t new. In the 19th century, it was used to justify racism.

Physiognomy was the use of defined facial features to assess a person’s character or personality. Cesare Lombroso thought he could identify a “born criminal” based on his face, with worrying traits to watch for including a sloping forehead, high cheekbones and a flattened nose. Before that, the legendary Carl Linnaeus, father of taxonomy, had used physiognomic measurements to label the “African race” as sly, lazy, cunning and lustful.

Yes, Affectiva has data to back it up. A WHOLE LOT of data — analyzing the facial expressions of 6.3 million people in 87 countries, in an attempt to avoid cultural bias. Physiognomy had data too (not that much, obviously), and was considered scientific in its heyday. Functional MRI (which measures blood flow to brain areas in near real-time) is steeped in data, and has also been tried as a market research tool. This despite the fact the technique once suggested a dead salmon could think.

No one’s going to say there’s nothing we can learn about a person or their emotional state from their physical appearance or expressions, but history has shown that we can be pretty good at fooling ourselves into putting too much stock into a particular method or technology. One can imagine Warner Brothers seeing this study and immediately changing their DC movie MO, based on less-than-conclusive data. And let’s face it, the last thing the DCEU needs is to make more questionable decisions.
To help celebrate Darwin Day on the 12th, AiPT! Science will be loading up on skepticism all throughout the month of February! Stay tuned for more critical analysis of all things pop culture!