How Does Face ID Work?

Apple’s new Face ID technology appears only in the flagship iPhone X. This replaces Touch ID on that device, instead unlocking the phone via facial recognition. This has sparked some concern with users who have grown used to the speed and reliability of Touch ID and seen similar face recognition technologies fail, as well as concerns among privacy advocates and security researchers. By a large, facial recognition technology has been disappointing, slow and easily fooled. But as with any markets, devices and technologies, Apple can often provide superior execution on an established technology. The hope is that Face ID is the best version of facial recognition that we’ve ever seen.

How does Face ID work?

Face ID is powered by Apple’s TrueDepth camera. If you’re a gamer, you might recognize that this camera is essentially an Xbox Kinect mounted on top of the phone. It’s even powered by the same technology. The Israeli company that pioneered infrared dot mapping technology used in the Kinect, PrimeSense, was purchased by Apple in 2013.

The TrueDepth camera is an infrared camera, using light at a lower wavelength then that visible by humans. When the user enrolls in Face ID, this infrared camera scans their face. It then creates a three-dimensional depth map with more than 30,000 points of measurement. The iPhone X uses these points to create a mathematical model of the face. That mathematical model is then stored securely on the device. On future unlock attempts, the camera re-scans the user’s face and compares the result. If it’s sufficiently similar, the device unlocks.

What about sunglasses, beards, and darkness?

The infrared light used by Face ID is invisible to humans, but it functions much the way normal light does. However, because its frequency is lower than regular light, it has a slightly easier time passing through some objects. This includes things like sunglasses which are typically balanced to block visible and ultra-violet (UV) light. This light exists at the opposite, high-frequency end of the spectrum of visible light. As a result, most sunglasses allow infrared light to pass. This permits Face ID to catch a glimpse even when sunglasses obscure your face.

Because iPhone X’s much-touted A11 Bionic chip supports machine learning, Face ID can adapt to gradual changes in your appearance. If you grow a beard, cut your hair or put on glasses, Face ID should keep up with the changes. We’ll have to see how successful this feature is in practice, but if it works as advertised, that will be a major advancement in face recognition’s utility.

Infrared light also allows Face ID to work in the dark. Because the camera provides its own light source in the form of an infrared beam, the system doesn’t require external sources of light to function. It can also use the flood illuminator (an infrared flash) to balance and normalize lighting on your face. And because this infrared light is invisible to humans, you won’t be blinded by the light. Thanks to this tech, the iPhone X will be able to ID you in total darkness or, more importantly, uneven illumination.

Is Face ID secure?

According to Apple, Face ID avoids common spoofing attacks that made other facial unlocking methods insecure. Thanks to it’s depth-sensitive infrared camera, attackers can’t fool it with a photograph. Even three-dimensional masks were apparently unsuccessful in fooling Face ID, according to Apple’s kind of creepy wall of masks.

The methods of avoiding three-dimensional spoofing haven’t been released. However, we can make some educated guesses. Infrared light can penetrate the skin, allowing the TrueDepth camera to scan subdermal features that would be difficult for mask-makers to replicate. We can also assume that the 30,000 points of facial recognition make it difficult to create a mask with sufficient fidelity.

Like Touch ID’s fingerprints, the mathematical models of recognized faces are stored within the iPhone X’s Secure Enclave. They’re encrypted, inaccessible to other apps and attackers alike. Faces exist only on the device, and they’re not stored on Apple’s servers or synced to iCloud.

Will Face ID replace Touch ID?

Apple says that Face ID’s error rate is 1 in 1,000,000. This means that Face ID is estimated to unlock for the wrong person once per one million unlock attempts. Touch ID had an error rate of 1 in 50,000, making Face ID more accurate by several orders of magnitude. And just like Touch ID, Face ID can also authenticate Apple Pay. Face ID also avoids some of Touch ID’s more annoying failure cases, like wet hands and gloves. Does this mean that Face ID will replace Touch ID eventually? The wise move is to wait for the reviews. If Face ID is as easy and reliable as Apple claims, it just might.