Quickly Building a Face Recognizer

Have you ever wondered how companies like Facebook automatically tag millions of user images?

Or did you find yourself in a situation where you want to automate tagging people in images… perhaps with tens of thousands of images?

Or maybe you just want to build a simple TV show celebrity classifier for your fan site?

Well now there’s a quick and efficient way of doing this today. And it scales seamlessly!

De-mystifying: How does facial recognition work?

Simple explanation: Face recognizer uses deep learning technology to detect faces in images, and then use facial features like eyebrows, lips, noses, etc. to tell people apart from each other. It usually can tell people apart from each other when you feed the model at least 10 images per person.

In-depth explanation: The algorithm first tries detecting all of the faces in a given image. Each face is represented with facial landmarks. These landmarks are essentially mathematical feature points that represent facial features like the eyes, eyebrows, nose, lips, etc.

We then use a special trick called perceptual hashing to remove duplicate facial features. Essentially, if you get the exact same features for a person, you don’t need to add that again for training. Doing this prevents overfitting to specific features in training.

Later, we use these facial features with their respective labels (people names) to train a new model. Since training is pretty fast (usually under a few mins), we do this after every action. (like adding or removing images, etc.)

And voila! We have a face recognizer for the people we’ve trained on. Now you can pass a new image, and the classifier will tell you who’s in the image with other information like bounding box coordinates and a confidence value.

Example use-case

For example, if you’re in a position where you have a ton of photos of users that aren’t tagged in, you can potentially create a classifier from scratch. The important point to emphasize here though is that you can do this unsupervised. This means that you don’t have to label the data yourself, which means you’ll save valuable time.

You would first start off by using the dlib/FaceDetection algorithm to filter out images that don’t have only one face in any given image. There’s a really good chance that the majority of those single face images are actually the user himself/herself. After filtering all of these images, you would just automatically label the faces with the owner of those images. Since we need at least 10 images per person, in a sizable social media site, you should be able to create a custom classifier for most of your users.

You can later use this classifier to automatically suggest user-tagging to your users, or just use it to determine real-life relationships between users by determining who’s in the photos.