Smartphones become capable of sensing human emotion

Smartphones are amazing. They tell us where we’re going, let us know if it’s going to rain, and even act like personal assistants. Now, a new research project out of the University of Rochester aims to make your phone capable of sensing your emotions just from measuring how you’re speaking — not based on what you’re saying.

This research, titled the Bridge Project, focuses on small changes in the human voice. Rather than using the traditional methodology of self-reporting and monitoring body language, this new method is based on automatic passive emotion detection. This technology can always be listening and monitoring a patient’s emotional state without any work on his or her part — providing a bigger picture on the patient’s entire status.

The basics of the system involve measuring twelve different aspects of speech, and then mapping the data onto six different emotions. Wendi Heinzelman, professor of electrical and computer engineering, said that the project analyzed completely emotionless phrases of speech, such as saying dates of the month. Impressively, they are able to reach an 81% accuracy rating with this model while previous attempts were only around 55%. By having actors read scripts with certain performances, the researchers are able to tweak their algorithm to associate certain pitches, volumes, and harmonics to a specific emotional state.

While this is undoubtedly an invaluable tool for psychologists and medical researchers, it also has huge potential for consumers. Take a look at Apple’s Siri. It’s designed to appear more human-like by offering humorous answers, apologizing, and using more realistic speech, like “Let’s hear some Beatles,” instead of something with less flare, like “Now playing: The Beatles, track one.” This gives us a better experience because it mimics human interaction. Now, think about this technology integrated into Siri. When you’re getting frustrated, it could offer simple hints on how to interact better. When you’re sad, it could throw in compliments.

In Dr. Oliver Sacks’s book The Man Who Mistook His Wife For A Hat, he tells a story about a group of patients suffering from aphasia — an inability to understand words. In the story, he details how very capable these patients are in detecting emotion through speech. In fact, they are able to use sound cues to effectively communicate with their loved ones and doctors despite not being able to understand the words directly. He even notes that it is extremely difficult to execute a lie in front of an aphasiac because they are so adept at picking up the hidden emotion. This story truly illustrates how much of our emotional states are expressed verbally, and just how useful this research really is.

Tagged In

My smartphone should be able to link with the other Smartphone and warn me as to the Girlfriend’s current Mood: Is she Angry? Pissed? Disgruntled? Moody? or is she ready to ambush me with “We have to Talk…” issues?

I call that Better Living Thru Science for MAN-kind!

preilly2

This program’s developers might be clever, but a smartphone isn’t. And just exactly which ‘patients’ are going to have their emotions constantly monitored, and will the doctor or hospital tell them that this is being done to them and why the data are being gathered?

preilly2

This program’s developers might be clever, but a smartphone isn’t. And just exactly which ‘patients’ are going to have their emotions constantly monitored, and will the doctor or hospital tell them that this is being done to them and why the data are being gathered?

It might be useful for Siri to self evaluate my reaction to its search answers and try harder.

http://ecelltech.blogspot.com/ e_cell

Hmm. I guess this is helpful to people who–admit it or not–are somewhat insensitive. But, doesn’t it appear silly if you put your phone close to someone just to know what their reaction is? You better hide it if they’re angry or they’ll smash your phone to pieces. Hah! I guess in this way, we get to spot if something is truthfully said or is just plain sarcasm.

ExtremeTech Newsletter

Subscribe Today to get the latest ExtremeTech news delivered right to your inbox.

Use of this site is governed by our Terms of Use and Privacy Policy. Copyright 1996-2016 Ziff Davis, LLC.PCMag Digital Group All Rights Reserved. ExtremeTech is a registered trademark of Ziff Davis, LLC. Reproduction in whole or in part in any form or medium without express written permission of Ziff Davis, LLC. is prohibited.