Shots - Health News

If Your Shrink Is A Bot, How Do You Respond?

Ellie is a computer simulation designed to engage real people in meaningful conversation and take their measure. The computer system looks for subtle patterns in body language and vocal inflections that might be clues to underlying depression or other emotional distress.

Ellie (right) is a computer simulation designed to engage real people, like the woman on the left, in meaningful conversation and take their measure. The computer system looks for subtle patterns in body language and vocal inflections that might be clues to underlying depression or other emotional distress.

YouTube

Her hair is brown and tied back into a professional-looking ponytail. She wears a blue shirt, tan sweater and delicate gold chain. It's the first time she has met the man sitting across from her, and she looks out at him, her eyes curious.

"So how are you doing today?" she asks cautiously, trying to build rapport.

"I'm doing well," he answers. His eyes blink.

"That's good," she continues. "Where are you from originally?"

"I'm from L.A.," he tells her, and this makes her smile slightly.

"Oh!" she says with surprise in her voice. "I'm from L.A. myself!"

She is from L.A. She was created in Los Angeles and "lives out her life" there on a computer screen in a lab at the University of Southern California. She's not a real woman but a virtual one, created to talk to people who are struggling emotionally, and to take their measure in a way no human can. Her makers believe that her ability to do this will ultimately revolutionize the way mental health care is practiced in this country. Her name is Ellie.

Rizzo and Morency spent months laboring over every element of Ellie's presentation and interaction with patients, experimenting with a range of different personalities, outfits and vocal mannerisms.

"Everything has been thought of," says Morency. For example, when patients talk, Ellie encourages them to continue talking with a well timed "uh-huh," just as real people do.

"We have recorded more than 200 of these uh-huhs," Morency says, "and these are so powerful. Because a simple 'uh-huh' and a silence — if they are done the right way — can be extremely powerful. So we spent a lot of time on these little details."

But the most important thing about Ellie is not her skill at gently probing all of the people her scientist brings into the lab to talk to her. Her real value, the reason she was built at all, is her skill at taking and analyzing thousands of measurements of those people.

Under the wide screen where Ellie's image sits, there are three devices. A video camera tracks facial expressions of the person sitting opposite. A movement sensor — Microsoft Kinect — tracks the person's gestures, fidgeting and other movements. A microphone records every inflection and tone in his or her voice. The point, Rizzo explains, is to analyze in almost microscopic detail the way people talk and move — to read their body language.

"We can look at the position of the head, the eye gaze," Rizzo says. Does the head tilt? Does it lean forward? Is it static and fixed?" In fact, Ellie tracks and analyzes around 60 different features — various body and facial movements, and different aspects of the voice.

The theory of all this is that a detailed analysis of those movements and vocal features can give us new insights into people who are struggling with emotional issues. The body, face and voice express things that words sometimes obscure.

"You know, people are in a constant state of impression management," Rizzo says. "They've got their true self and the self that they want to project to the world. And we know that the body displays things that sometimes people try to keep contained."

So, as Ellie gets the person in front of her to ruminate about when they were happy and when they were sad, the machines below her screen take measurements, cataloging how much the person smiles and for how long, how often they touch their head.

Morency says the machines record 30 measurements per second, or "about 1,800 measurements per minute." Literally every wince, pause and verbal stumble is captured and later analyzed.

Ellie was originally commissioned by the U.S. Department of Defense. After all of the deployment in Iraq and Afghanistan, the military was seeing a lot of suicides and wanted to find a way to help military therapists stop them. Soldiers don't always like to confess that they're having problems, but maybe their bodies would say what their words wouldn't.

This is why Ellie is being programmed to produce a report after each of her sessions — it's a kind of visual representation of the 60 different movements she tracks.

"For each indicator," Morency explains, "we will display three things." First, the report will show the physical behavior of the person Ellie just interviewed, tallying how many times he or she smiled, for instance, and for how long. Then the report will show how much depressed people typically smile, and finally how much healthy people typically smile. Essentially it's a visualization of the person's behavior compared with a population of depressed and nondepressed people.

If the person's physical behaviors are similar to someone who's depressed, then the person will be flagged.

The idea here is not for Ellie to actually diagnose people and replace trained therapists. She's just there to offer insight to therapists, Morency says, by providing some objective measurements.

"Think about it as a blood sample," he says. "You send a blood sample to the lab and you get the result. The [people] doing the diagnosis [are] still the clinicians, but they use these objective measures to make the diagnosis."

Real People Are Complicated

Now, obviously this work raises all kinds of issues, and even on a practical level, real obstacles remain. Jeff Cohn, a psychologist at the University of Pittsburgh, studies the relationship between physical movements and emotion and says signals from the face, voice and body are incredibly complicated to interpret.

"Individuals vary a lot in how expressive they are," Cohn explains. "You know, if I'm someone who is very expressive and I smile frequently, [even] when I'm depressed and smiling less, I may still smile more than you do if you're a tight-lipped, not very emotive individual."

This means, Cohn says, that using Ellie in the way blood tests are used — as proof positive of one diagnosis or another — will be really difficult.

"It strikes me as unlikely that face or voice will provide that information with such certainty," he says.

But Skip Rizzo, the psychologist working on Ellie, genuinely believes these technologies will eventually change the field of mental health. One of the central problems with humans, he says, is that they bring their own biases to whatever they encounter, and those biases often make it hard for them to see what's directly in front of them.

"You can get training to be a health care provider or psychologist," he says, "and try to put those things on hold and be very objective. But it's still a challenge. It's always going to be biased by experience. What computers [like Ellie] offer is the ability to look at massive amounts of data and begin to look at patterns, and that, I think, far outstrips the mere mortal brain."

This summer, Ellie is being tested. She's scheduled to sit down with dozens of veterans from Iraq and Afghanistan.

She'll ask them about their lives, encourage them to open up.

Then, silently, Ellie will measure their answers.

Copyright 2015 NPR. To see more, visit http://www.npr.org/.

Transcript

DAVID GREENE, HOST:

On a Monday, it's MORNING EDITION from NPR News. I'm David Greene.

STEVE INSKEEP, HOST:

And I'm Steve Inskeep.

Today in Your Health, we'll hear from scientists who want to identify people at risk for depression. They want to do it using the power of computers.

GREENE: In particular, they're using a virtual therapist. This computer turns its cameras on patient analyzing how they move and searching for clues about what they're really feeling.

INSKEEP: Now, the virtual therapist has a name, Ellie. The NPR reporter who brings you Ellie's story is named Alix Spiegel.

ALIX SPIEGEL, BYLINE: Her hair is brown, and tied back into a professional looking pony tail. She wears a blue shirt, tan sweater and delicate gold chain. From the screen - where she lives out her life - she looks out at the man sitting across from her with curious eyes. He's a real man sitting in a real chair and this is the first time that they've talked. So Ellie starts simply, trying to build rapport.

ELLIE: So how are you doing today?

UNIDENTIFIED MAN: I'm doing well.

ELLIE: That's good. Where are you from originally?

UNIDENTIFIED MAN: I'm from Los Angeles.

ELLIE: Oh, I'm from L.A. myself.

SPIEGEL: Ellie is from L.A. She lives there in a computer at the University of Southern California, the creation of a group of scientists who believe that what they're doing will eventually revolutionize mental health care. They've been working on Ellie for months now, experimenting with giving her different personalities, programming her to ask provocative open ended questions. Laboring, says project manager Louis-Phillippe Morency over every element of her interactions.

LOUIS-PHILLIPPE MORENCY: Everything has been thought of. Just a simple what we call back channel like uh-huh, we have recorded more than 200 of these uh-huh.

ELLIE: When was the last time you felt really happy?

UNIDENTIFIED MAN: Uh, when was the last time.

ELLIE: Hmm.

UNIDENTIFIED MAN: Probably a couple months ago.

MORENCY: And these are so powerful, because a simple uh-huh and a silence, if they are done the right way - can be extremely powerful. So we spend a lot of time on these little details.

SPIEGEL: But Ellie isn't really intended to replace actual human beings as a therapist. Her real value - the reason she was created at all - is to help actual therapists diagnose patients because Ellie can measure people in a way that most humans can't. Underneath the wide screen, where Ellie's image sits, there are three devises: a video camera which tracks a person's face, a Microsoft Kinect that tracks their body movement, and a microphone which records their voice.

The point of these, says Skip Rizzo, a psychologist who is working on Ellie, is to analyze in almost microscopic detail the way that people talk and move, to read their body language.

SKIP RIZZO: We can look at the position of the head - the eye gaze and the head gaze - does the head tilt, does it lean forward, is it static and fixed.

SPIEGEL: In fact Ellie tracks and analyzes 60 different body and face and vocal movements. The theory of this is that a detailed analysis of those movements can give therapists new insight into their patients, because micro changes in the body, face and voice express things that words sometimes obscure.

RIZZO: You know, people are in a constant state of impression management. They've got their true self and the self that they want to project to the world. And we know that the body displays things that sometimes people try to keep contained.

SPIEGEL: And so, as Ellie gets the person in front of her to ruminate about when they were happy, when they were sad, the machines below her screen measure. Cataloguing how much the person smiles and for how long, how often they touch their head. Morency says they record 30 measurements a second .

MORENCY: Yeah, 30 times 60 seconds. So about 1,800 measurement per minute.

SPIEGEL: Literally every wince, pause and verbal stumble is captured.

ELLIE: What advice would you have given yourself 10 or 20 years ago?

UNIDENTIFIED WOMAN: Um, to, uh to not believe. Uh, to not be so gullible. To not be so gullible.

ELLIE: Hmm.

SPIEGEL: Now, Ellie was commissioned by the military. After all the deployments in Iraq and Afghanistan, the military was seeing a lot of suicides and wanted to find a way to help military therapists catch those suicides before they happened. Soldiers don't always like to confess they're having problems but maybe their bodies would. This is why Ellie is being programmed to produce a report after each of her sessions; a kind of visual representation of the 60 different movements that she tracks.

MORENCY: For each indicator, we will display three things.

SPIEGEL: First the report will show the physical behavior of the person Ellie just interviewed - for instance, how many times they smiled and for how long. Then the report will show how much depressed people typically smile; and finally, how much healthy people typically smile. Essentially, says Morency...

MORENCY: It's a visualization of the persons behavior compare to a population of depressed and non-depressed.

SPIEGEL: If their physical behaviors are similar to someone who's depressed, the person will be flagged. And again, Morency insists it's not that Ellie will actually diagnose these people, she's just there, he says, to give insight by providing some objective measurements.

MORENCY: Think about it as a blood sample, you send a blood sample to the lab and you get the result. The person doing the diagnosis is still the clinician, but they use these objective measures to make the diagnosis.

SPIEGEL: Now obviously, this work has all kinds of issues. Even on a practical level there are real obstacles. Jeff Cohen is a psychologist at the University of Pittsburg who researches the relationship between physical movements and emotion, and he says signals from the face, body and voice are incredibly complicated to interpret.

JACK COHEN: Individuals vary a lot in how expressive they are. You know, if I'm someone who's very expressive and I smile frequently; when I'm depressed and smiling less, I may still smile more than you do, you know, if you're a tight lipped, not very emotive individual.

KIRK SIEGLER, BYLINE: This means Cohen says, that using Ellie in the way blood tests are used - as proof positive of one emotional state or another - will be really difficult.

COHEN: It strikes me as unlikely that face or voice will provide that information with such certainty.

SPIEGEL: But Rizzo - the psychologist working on Ellie - genuinely believes these technologies will eventually change mental health. If you can collect enough data from enough humans, he says, you'll be able to see human behavior in a new way. One of the central problems with humans, he says, is that they bring their own biases to whatever they encounter, and those biases often make it hard for them to see what's directly in front of them.

RIZZO: You can get training to be a health care provider or psychologists and try to put those things on hold and be very objective. But it's still a challenge. It's always going to be biased by our past experience. So what computers offer is the ability to look at massive amounts of data and begin to look for patterns and that I think far outstrips the mere mortal brain.

SPIEGEL: So are you saying that the computer can see people in a more objective way?

RIZZO: Yes. I would say that you're objectively picking up physical events in the world. And the ability to do that on such a grand comprehensive scale with a massive database of past observations, I think you can pull information out of that by using various methodologies in computing - that goes beyond a human brain.

SPIEGEL: This summer Ellie's being tested. She scheduled to sit down with about a hundred veterans from Iraq and Afghanistan. She'll ask them about their lives, encourage them to open up, then silently, Ellie will measure their answers.