All Tech Considered

3:38 am

Mon October 28, 2013

A Look Into Facebook's Potential To Recognize Anybody's Face

Social media companies like Facebook won't talk about who can access face-tagging data. That silence is a problem, privacy advocates say.

iStockPhoto.com

Revelations about NSA spying have left people wondering about the privacy of their digital data. But what about the privacy of their faces?

The movies make facial recognition look easy: In the 1998 film Enemy of the State, a team of NSA agents simply freeze a surveillance tape, tap some keys and identify the face a few computer beeps later.

But Neeraj Kumar, an expert in computer vision at the University of Washington, says we're nowhere close to just grabbing anybody's face off a security camera and coming up with a name.

Facial recognition has become pretty good at one-to-one comparisons — for instance, checking your face against the photo on your company ID. The accuracy is up to 95 percent.

But that's not so good if you're trying to come up with a name, and you're comparing one photo against many possible matches.

"Each time you do a comparison, there's 5 percent chance that it's wrong," Kumar says. "And that adds up. In fact, it multiplies up. Very quickly, you find that a 95 percent accuracy leads to pretty terrible results when you're actually trying to answer the question of, 'Who is this person?' "

Universal Challenges

That's a hard question for a computer. It's even harder when you ask it to identify a face in what experts call an "image from the wild."

"You're looking at surveillance-type images from low-resolution cameras," says Manuel Cueva, a Los Angeles County sheriff's deputy who trains officers in their facial recognition system. "If the image is such poor quality, you may not get any results, period."

The software can get confused by shadows and weird angles. Even a goofy smile can throw it. So that's problem one.

Problem two in this quest for universal facial recognition is that the software needs to compare one picture to another. In LA County, for instance, officials will run the image against their booking photos — about 6 million mug shots.

But if you've never been arrested in LA County, their system won't name you. And that's usually the end of it.

"There really isn't a set standard that we follow to be able to extend our searches into other jurisdictions," Cueva says.

Some agencies and states are working to increase sharing of mug shots and department of motor vehicles photos. But as of right now, we simply don't have a universal database of faces.

Facebook has the largest biometric database in the world — "and it's all been formed by people voluntarily submitting pictures to Facebook and identifying who they belong to," she says.

Theoretically, every time you label faces by tagging a picture, you're chipping away at those two big challenges for universal facial recognition. First, you're helping to build a super-database of labeled faces. Second, you're uploading multiple versions of each person's face, which can improve a system's accuracy.

"If you had lots of photos of each person ... you could build a model for Martin, a model for me, a model for other people. Now you have a custom-tuned model for each person," Kumar, from the University of Washington, says.

Multiply that by a billion — a billion custom-tuned facial "models."

Facebook would not answer NPR's questions about what it does with facial recognition information; social media companies rarely talk about their internal systems.

Stepanovich wants social media companies to explain how facial models can be used. On Facebook, for example, you can't identify faces of people who aren't already your "friends," but she wonders if, behind the scenes, Facebook can do broader searches — say, at the request of the government.

"As we're seeing specifically over the past few months, no matter how much a company attempts to protect your privacy, if they're collecting information about you, that information is vulnerable to government search," Stepanovich says.

Facebook won't say whether this is technically possible; Google, which offers the competing Google Plus service, also won't comment on the record about the feasibility of broader face searches.

Kumar doubts anyone is doing universal searches of Facebook faces. He says the numbers are just too big.

However, if social media companies are able to narrow the search — say, if they can compare a photo with the facial models of everybody who "likes" NPR, or everybody who lives in Des Moines — then, Kumar says, you'd have the makings of a useful search tool.

Copyright 2013 NPR. To see more, visit http://www.npr.org/.

Transcript

RENEE MONTAGNE, HOST:

Bet you've never thought of your Facebook photos as mug shots of you, your family and friends. Maybe not now, but at some point that could be the case. Computer facial recognition technology used by police and government security agencies is getting better. And if a universal facial recognition system becomes a reality, we find that we all helped to build it.

NPR's Martin Kaste explains.

MARTIN KASTE, BYLINE: In the movies they make it look easy.

(SOUNDBITE OF MOVIE, "ENEMY OF THE STATE")

UNIDENTIFIED MAN: (Unintelligible) pull up the surveillance tapes for the last 20 minutes.

KASTE: Here's a scene from "Enemy of the State," where they ID one of the good guys.

(SOUNDBITE FROM MOVIE "ENEMY OF THE STATE")

UNIDENTIFIED MAN #2: Freeze it and process it for face recognition.

KASTE: That's a movie that's already 15 years old. So can they do this now? Can they just grab anybody's face off a security camera and come up with a name? Neeraj Kumar should know. He's an expert in computer vision at the University of Washington.

NEERAJ KUMAR: No. We're nowhere close to that.

KASTE: Kumar says facial recognition has become pretty good at one-to-one comparisons - for instance, checking your face against the photo on your company ID. The accuracy there is up to 95 percent. But that's not so good if what you're doing is trying to come up with a name, and you're comparing one photo against lots of possible matches.

KUMAR: So each time you do a comparison, there's a five percent chance that it's wrong.

KASTE: And that adds up.

KUMAR: And that adds up. In fact, it multiplies up. Very quickly you find that a 95 percent accuracy leads to pretty terrible results when you're actually trying to answer the question of who is this person.

KASTE: That's a hard question for a computer, especially when you're asking it to ID a face in what experts call an image from the wild.

CUEVA: If the image is such poor quality, you may not get any results, period.

KASTE: The software gets confused by shadows and weird angles. Even a goofy smile can throw it. So that's problem one. Problem two in this quest for universal facial recognition is that you need to compare your picture to something else. In LA County, for instance, they'll will run the image against their booking photos, mug shots, about six million of them.

But if you've never been arrested in LA County, their system won't name you. And that's usually where it ends.

CUEVA: There really isn't a set standard that we follow to be able to extend our searches into other jurisdictions.

KASTE: There are some efforts to increase the sharing of mug shots and DMV photos between agencies and states. But as of right now, we really don't have a universal database of faces. Or do we?

AMIE STEPANOVICH: Go to Facebook.

KASTE: Oh, yeah, Facebook. This is Amie Stepanovich, director of the domestic surveillance project at the Electronic Privacy Information Center in Washington, D.C.

STEPANOVICH: Facebook has, in the world, the largest biometric database, and it's all been formed by people voluntarily submitting pictures to Facebook and identifying who they belong to.

KASTE: Every time you tag a picture, when you're labeling the faces in that photo from your birthday party, you're chipping away at these two big challenges of universal facial recognition. First, you're helping to build a super-database of labeled faces. And second, you're uploading multiple versions of a person's face. Researcher Neeraj Kumar says that can improve a system's accuracy.

KUMAR: If you had lots of photos of each person, you could build a model, you know, a model for Martin, a model for me, a model for other people. Now you have a custom-tuned model for each person.

KASTE: Multiply that by a billion, a billion custom-tuned facial models. Facebook would not answer NPR's questions about facial recognition. Social media companies rarely talk about their internal systems, but they're surely aware of their huge database's potential. Last year Facebook bought Face.com. One of that company's founders had published a paper titled "Leveraging Billions of Faces to Overcome Performance Barriers in Unconstrained Face Recognition."

STEPANOVICH: It's uncertain what can be done with this information.

KASTE: Amie Stepanovich wants the social media companies to explain how the facial models can be used. On the websites you can't identify the faces of people who aren't already in your friends network, but she wonders if behind the scenes Facebook can do broader searches - say, at the request of the government.

STEPANOVICH: As we're seeing specifically over the last few months, no matter how much a company attempts to protect your privacy, if they're collecting information about you, that information is vulnerable to government search.

KASTE: Again, Facebook won't say whether this is technically possible. The competition, Google Plus, also won't comment on the record about the feasibility of broader face searches. At the University of Washington, Neeraj Kumar doubts that anyone is doing universal searches of Facebook faces. He says the numbers there would just be too big.

KUMAR: If Facebook has a billion images, then maybe they can narrow it down to a million for you, so have fun to enter a million.

KASTE: However, if social media companies are able to narrow the search - say, if they can compare a photo with the facial models of everybody who likes NPR, or everybody who lives in Des Moines, then, Kumar says, you'd have the makings of a useful search tool, a tool government agencies might find hard to resist. Martin Kaste, NPR News. Transcript provided by NPR, Copyright NPR.