Facebook’s facial recognition software is now as accurate as the human brain, but what now?

This site may earn affiliate commissions from the links on this page. Terms of use.

Facebook’s facial recognition research project, DeepFace (yes really), is now very nearly as accurate as the human brain. DeepFace can look at two photos, and irrespective of lighting or angle, can say with 97.25% accuracy whether the photos contain the same face. Humans can perform the same task with 97.53% accuracy. DeepFace is currently just a research project, but in the future it will likely be used to help with facial recognition on the Facebook website. It would also be irresponsible if we didn’t mention the true power of facial recognition, which Facebook is surely investigating: Tracking your face across the entirety of the web, and in real life, as you move from shop to shop, producing some very lucrative behavioral tracking data indeed.

The DeepFace software, developed by the Facebook AI research group in Menlo Park, California, is underpinned by an advanced deep learning neural network. A neural network, as you may already know, is a piece of software that simulates a (very basic) approximation of how real neurons work. Deep learning is one of many methods of performing machine learning; basically, it looks at a huge body of data (for example, human faces) and tries to develop a high-level abstraction (of a human face) by looking for recurring patterns (cheeks, eyebrow, etc). In this case, DeepFace consists of a bunch of neurons nine layers deep, and then a learning process that sees the creation of 120 million connections (synapses) between those neurons, based on a corpus of four million photos of faces. (Read more about Facebook’s efforts in deep learning.)

Once the learning process is complete, every image that’s fed into the system passes through the synapses in a different way, producing a unique fingerprint at the bottom of the nine layers of neurons. For example, one neuron might simply ask “does the face have a heavy brow?” — if yes, one synapse is followed, if no, another route is taken. This is a very simplistic description of DeepFace and deep learning neural networks, but hopefully you get the idea.

Sylvester Stallone, going through DeepFace’s forward-facing algorithm. Notice how the slight tilt/angle in (a) is corrected in (g). (d) is the “average” forward-looking face that is used for the transformation. Ignore (h), it’s unrelated.

Anyway, the complexities of machine learning aside, the proof is very much in the eating: DeepFace, when comparing two different photos of the same person’s face, can verify a match with 97.25% accuracy. Humans, performing the same verification test on the same set of photos, scored slightly higher at 97.53%. DeepFace isn’t impacted by varied lighting between the two photos, and photos from odd angles are automatically transformed (using a 3D model of an “average” forward-looking face) so that all comparisons are done with a standardized, forward-looking photo. The research paper indicates that performance — one of the most important factors when discussing the usefulness of a machine learning/computer vision algorithm — is excellent, “closing the vast majority of [the] performance gap.”

Facebook tries to impress upon us that verification (matching two images of the same face) isn’t the same as recognition (looking at a new photo and connecting it to the name of an existing user)… but that’s a lie. DeepFace could clearly be used to trawl through every photo on the internet, and link it back to your Facebook profile (assuming your profile contains photos of your face, anyway). Facebook.com already has a facial recognition algorithm in place that analyzes your uploaded photos and prompts you with tags if a match is made. I don’t know the accuracy of the current system, but in my experience it only really works with forward-facing photos, and can produce a lot of false matches. Assuming the DeepFace team can continue to improve accuracy (and there’s no reason they won’t), Facebook may find itself in the possession of some very powerful software indeed. [Research paper: “DeepFace: Closing the Gap to Human-Level Performance in Face Verification“]

What it chooses to do with that software, of course, remains a mystery. It will obviously eventually be used to shore up the existing facial recognition solution on Facebook.com, ensuring that every photo of you on the social network is connected to your account (even if they don’t show a visible tag). From there, it’s hard to imagine that Zuckerberg and co will keep DeepFace purely confined to Facebook.com — there’s too much money to be earnt by scanning the rest of the public web for matches. Another possibility would be branching out into real-world face tracking — there are obvious applications in security and CCTV, but also in commercial settings, where tracking someone’s real-world shopping habits could be very lucrative. As we’ve discussed before, Facebook (like Google) becomes exponentially more powerful and valuable (both to you and its share holders) the more it knows about you.

Tagged In

It won’t do you any good. Unless you live in a cave and never come out your photo will be taken and you won’t even realize it in most cases.

eonvee375

+1, at least i dont pretend that i dont have FaceBook for other reason than that im antisocial…

Ivor O’Connor

Hmmm. I don’t use FaceBook for anything other than logging in to places. No friends. However I’m also antisocial.

AD Fkin Ate

Its happening now big things.. They moved a huge step forward.. Back in feburary and now using it to their full advantage.. Couple people already been busted for marijuana possession because of a picture they were tagged in and had no knowledge heh .

Ivor O’Connor

That is just the start. Giving that power to the police is like giving a big milatary to the politicians.

Jake Soden

hi it is only for your own safety i don’t have anything to hide do you if you don’t just be glad that it is the really bad guys that have to be careful relevant big time bad guy, irrelevant pettey low crimes/ non crime lords

Jake Soden

i made the software but not the one for fb my own to get the goverment and play them at there own sick twisted game

brekinapez

Too late. Like Ivor said, your face is already captured somewhere somehow and will be used to someone else’s profit.

Singh1699

It will but no more information is being put on there. My face doesn’t mean much, hopefully it can be put next to other great men one day.
Enemies will no longer have direct access to ideas and information though.

Hunter Lucas

But they’ll have all your phone conversations, texts, emails, etc. as well. Your only alternative is to go live as a hobo in the woods. And even then, we’ll have drones flying all over the place pretty soon. There’s going to be no place left to hide.

dickass

It’s been a year. Still no drones you fucking moron.

Bryan Field

Sshh. They are invisible and silent

Pedro Pereira

Since you are born you are doomed.
It’s a big conspiration and a theory without any relevant information that can wistand what I’m saying, so no proof is garanteed… but from the moment you step in a supermarket as baby, and you go to make you identaty card its bye bye for the “anonymous person”. Everyone has their info somewhere in an intranet where somewhere there can be a security breach to reach those kind of databases… eventually you’ll never be safe. Pretty much every info relating to you that was written in paper, might already be digitalized and in a computer somewhere in the world.

Ran Don

Do you have a whole separate closet for all your aluminum hats

dc

That’s not very impressive. I can never remember anyone’s name.

eonvee375

its just that Dan is as fat as the little girl

normal for the kid, not so much for the dude…

John Flerianos

pay closer attention. they really do look a lot alike!

ephemeris

Your really stuck if all you do is landscapes.

preilly2

I can imagine a future in which significant numbers of people won’t leave the house unless they’re wearing some sort of disguise whilst out shopping or walking around town. This does sound far-fetched at present, but someday people may just have had enough of every movement of their lives being recorded. They’re going to power down their devices when they go out and hide their features for their own protection. The authorities will react by making ‘deliberate facial concealment’ a crime. It sounds dystopian, but stranger things have already happened.

Can this be used to help clone/ make look-alikes of people? Identity theft?

Jill_the_Pill

No, it cannot. That is not a worry in our lifetimes. Worry instead that you will be identified participating in something that is not now illegal but may become so in the future: journalism, protest, art, that kind of thing.

Jill_the_Pill

” there’s too much money to be earnt by scanning the rest of the public web for matches.”

I am really a big fan of face recognition system. Thanks for sharing this article with me. Also I would like to know more about the other artificial intelligence technologies. If you can tell me than I will be very grateful to you.

Stores who issue ‘banning orders’ to those caught shoplifting within their premises may soon use facial recognition technology to assist in enforcing the order. Yes, shoplifters may very well begin to wear disguises, but it won’t be that difficult to spot those wearing some sort of face mask.

ScottDrysdale

What political leader, given the opportunity, would choose to ignore the opportunity to make effective use of this powerful tool? How much faster would Hitler have rounded up all the jews in WW2 by means of FB c/w DeepFace facial recognition? Where is society going with this?

This site may earn affiliate commissions from the links on this page. Terms of use.

ExtremeTech Newsletter

Subscribe Today to get the latest ExtremeTech news delivered right to your inbox.