In this paper, we demonstrate a simple face spoof attack targeting the face recognition system of a widely available commercial smart-phone. The goal of this paper is not proclaim a new spoof attack but to rather draw the attention of the anti-spoofing researchers towards a very specific shortcoming shared by one-shot face recognition systems that involves enhanced vulnerability when a smiling reference image is used.

Introduction

One-shot face recognition (OSFR) or single sample per person (SSPP) face recognition is a well-studied research topic in computer vision (CV) [8]. Solutions such as Local Binary Pattern (LBP) based detectors [1], Deep Lambertian Networks (DLN) [9] and Deep Supervised Autoencoders (DSA) [4] have been proposed in recent times to make the OSFR system more robust to changes in illumination, pose, facial expression and occlusion that they encounter when deployed in the wild. One very interesting application of face recognition that has gathered traction lately is for mobile device unlocking [6]. One of the highlights of Android 4.0 (Ice Cream Sandwich) was the Face Unlock screen-lock option that allowed users to unlock their devices with their faces. It is rather imperative that we mention here that this option is always presented to the user with a cautioning clause that typically reads like *Face recognition is less secure than pattern, PIN, or password.

The reasoning behind this is that there exists a plethora of face spoof attacks such as print attacks, malicious identical twin attack, sleeping user attack, replay attacks and 3D mask attacks. These attacks are all fairly successful against most of the commercial off-the-shelf face recognizers [7]. This ease of spoof attacks has also attracted attention of the CV researchers that has led to a lot of efforts in developing liveness detection anti-spoofing frameworks such as Secure-face [6]. (See [3] for a survey.)

Recently, a large scale smart-phone manufacturer introduced a face recognition based phone unlocking feature. This announcement was promptly followed by media reports about users demonstrating several types of spoof attacks.

In this paper, we would like to explore a simple print attack on this smart-phone. The goal of this paper is not proclaim a new spoof attack but to rather draw the attention of the anti-spoofing community towards a very specific shortcoming shared by face recognition systems that we uncovered in this investigation.

2. Methodology and ResultsFigure 1. Example of two neutral expression faces that failed to spoof the smart-phone’s face recognition system.Figure 2. Example of 2 smiling registering faces that successfully spoofed the smart-phone’s face recognition system.The methodology we used entailed taking a low quality printout of the target user’s face on a plain white US letter paper size (of dimension 8.5 by 11.0 inches) and then unlocking the device by simply exposing this printed paper in front of the camera. Given the poor quality of the printed images, we observed that this simple print attack was duly repulsed by the detector system as long as the attacker sported neutral facial expressions during the registration phase. However, when we repeated the attack in such a way that the attacker had an overtly smiling face when (s)he registered, we were able to break in successfully with high regularity.

In Figure 1, we see two examples of neutral expression faces that failed to spoof the smart-phone’s face recognition system when the registering image had a neutral facial expression. A video containing the failed spoofing attempt with a neutral facial expression can be viewed here.

In Figure 2, we see the same two subjects’ images that successfully spoofed the phone’s face recognition system when the registering (enrollment) image was overtly smiling. The face training demo videos are available here. The video of the successful spoof can be viewed here.

2.1. Motivation for the attack and discussion

It has been well known for a long time in the computer vision community that faces displaying expressions, especially smiles, resulted in stronger recall and discrimination power [10]. In fact, the authors in [2] termed this the happy-face advantage, and showcased the variation in detection performance for varying facial expressions. Through experimentation, we wanted to investigate the specific onshot classification scenario when the registering enrollment face had a strong smile that resulted in the discovery of this attack. As for defense from this attack, there are two straightforward recommendations. The first recommendation would be to simply display a message goading the user to maintain a passport-type neutral facial expression. The second would entail having a smile detector such as [5] as a pre-filter that would only allow smile-free images as a reference image.

Today, UnifyID, a service that can authenticate you based on unique factors like the way you walk, type, and sit, announced the final 16 fellows selected for its inaugural Artificial Intelligence Fellowship for the Fall of 2016. Each of the fellows have shown exemplary leadership and curiosity in making a meaningful difference in our society and clearly has an aptitude for making sweeping changes in this rapidly growing area of AI.

Of the company’s recent launch and success at TechCrunch Disrupt, claiming SF Battlefield Runner-Up (2nd in 1000 applicants worldwide), UnifyID CEO John Whaley said, “We were indeed overwhelmed by the amazing response to our first edition of the AI Fellowship and the sheer quality of applicants we received. We also take immense pride in the fact that more than 40% of our chosen cohort will be women, which further reinforces our commitment as one of the original 33 signees of the U.S. White House Tech Inclusion Pledge.”

The final 16 fellows hail from Israel, Paris, Kyoto, Bangalore, and cities across the U.S. with Ph.D., M.S., M.B.A., and B.S. degrees from MIT, Stanford, Berkeley, Harvard, Columbia, NYU-CIMS, UCLA, Wharton, among other top institutions.

Aidan Clark triple major in Math, Classical Languages and CS at UC Berkeley

This highly selective, cross-disciplinary program covers the following areas:

Deep Learning

Signal Processing

Optimization Theory

Sensor Technology

Mobile Development

Statistical Machine Learning

Security and Identity

Human Behavior

Our UnifyID AI Fellows will get to choose from one of 16 well-defined projects in the broad area of applied artificial intelligence in the context of solving the problem of seamless personal authentication. The Fellows will be led by our esteemed Fellowship Advisors, renown experts in machine learning and PhDs from CMU, Stanford, and University of Vienna, Austria.

I just got back from HackMIT, and what a crazy, intense experience it was. For those who don’t know, HackMIT is a 24-hour hackathon with over 1,000 students from all over the country and the world, all hacking on some very cool stuff. I was on the judging panel as well as acted as a mentor, helping students debug issues with a wide variety of technologies like node.js/Express, cocoapods and Swift 3, Ethereum smart contracts, Angular and Javascript, 502 errors on HTTP requests, and a bunch of other issues. A few students came up to me after they recognized UnifyID from our TechCrunch video and wanted to take photos together.

I met a lot of great students from all over the US, Europe, and South America. I also gave a tech talk where we demonstrated our implicit authentication technology in action with a volunteer from the audience. Being a technical crowd, I was able to dive deep into the technical aspects with some of the actual data in a Jupyter notebook. People were amazed by some of the unique aspects to human movement and how much information you can get from the accelerometer and gyroscope in your phone!

HackMIT had tons of free food/drinks/activities. They had no soft drinks because they were encouraged to avoid unhealthy drinks, but they had plenty of Red Bull (?). And unlimited Soylent, too. Plus food/snacks at all hours of the day and night, like fresh smoothies at midnight and hot waffles with chocolate in the morning. And crazy activities like the 2am shakedown and the 7 minute workout outside in the wee hours of the morning.

Many/most teams stayed up all night hacking. There was a wide variance in hacking ability but the top teams were truly astonishing in what they were able to build in 24 hours. All of the top ten were amazing and it was hard to choose.

The ultimate winner was “WindowShare”. They built an awesome cross-platform tool where you can drag any window between computers and it seamlessly copies the program’s file and opens it on the other machine. Like if you open a text file in TextEdit on a Mac, you can drag the window over and the contents appear in a Notepad window on the Windows machine. Likewise for images and Chrome windows/tabs. They also implemented remote mouse so you could move your mouse on the other screen as well and control it without messing up the original mouse. They implemented in Java with JNI and socket communication.

The runner-up was a book-reading bot that used the phone, OCR, and text-to-speech to read (physical) books aloud. It also used a motorized mechanism including a computer fan to reliably turn pages.

We also added a honorable mention: “Fretless”, an MIT team that built a Guitar Hero like contraption that hooks to your violin. It takes a MIDI file and lights up where you are supposed to press your fingers so you can learn how to play real songs.

All of the top ten projects were amazing and the teams got a ton done in 24 hours! To everyone who participated, I say “Hack on!”