Build a Face Detection App Using Node.js and OpenCV

Human beings have an innate ability to detect, process and recognize faces — we’re born with it. Computers can do it, too — it just takes some clever algorithms, reams of code and some training of the algorithms.

Face detection is the process of identifying faces in digital images. It shouldn’t be confused with facial recognition — i.e., trying to work out who someone is from a photograph — but it’s the first part of the process. Facial recognition is a huge topic for another time, but face detection is the subject of this article.

Applications of Face Detection

There are numerous applications of face detection. Some modern biometrics systems detect faces and then use facial recognition to compare those faces to images in their databases, in order to try and identify someone without having to resort to good old-fashioned passwords. Some cameras use face detection for auto focussing. And like so many things, it also has applications in marketing.

For the purposes of this tutorial, we’ll replicate a feature that you may well have used yourself if you’re a Facebook user. When you upload a photo of your friends, Facebook often shows it back to you with any faces highlighted, in order to prompt you to “tag” people in it. We’re going to build something similar.

A Little Background

Before we dig into the code, let’s look at some of the tools and concepts we’re going to be using.

OpenCV and the Viola-Jones Object Detection Algorithm

OpenCV (Open Source Computer Vision) is an open-source library of hundreds of computer vision algorithms. Although OpenCV is written in C++, we can use it in Node.js applications thanks to the opencv package.

Face detection is simply a sub-set of feature (object) detection, but the algorithm is geared towards the challenges involved in detecting faces specifically.

Of course when we talk about feature detection in this context, it’s nothing to do with the sort of feature detection provided by libraries such as Modernizr and yepnope!

First presented in a 2004 article by Paul Viola and Michael J. Jones, this approach has become the de facto standard for face detection.

You’ll find some additional resources on the framework listed under Further Reading later on in this tutorial.

Cascades and Classifiers

An important aspect of the Viola-Jones algorithm is a cascade of classifiers, which is described as “a cascade of boosted classifiers working with haar-like features”. In practical terms, what this means is that it’s a set of visual features that OpenCV has been “trained” to look for in an image, in order to identify a particular type of object — in our case, faces. You’ll find more information about cascades and classifiers in the documentation. A cascade designed specifically for identifying faces is provided for us, as we’ll see when we look at the implementation.

Installation

Before we can start playing with face detection, we need to install a few pre-requisites.

The easiest (and recommended) way to get up-and-running is to use Vagrant. You’ll find the necessary configuration and provisioning script in the repository which accompanies this article. If you use this approach, there’s no need to go through these installation steps.

Installing OpenCV

Linux (Debian-based systems)

OpenCV has a number of pre-requisites itself, which we can install using apt-get:

At time of writing, this installs version 2.4.8, although the latest 2.x version is 2.4.11 and there is currently a version 3.0.0. However, there are currently issues with the Node.js wrapper on version 3.0.0., so this version is just fine.

Building from Source

If you want to build from source, start by installing the dependencies listed above, then download and extract the files from the downloads page.

As noted above, there are currently issues with the 3.0.0. in conjunction with the Node.js module, so it’s best to download version 2.4.11.

I’m referencing the Bootstrap framework to prettify the application slightly, but this is optional. Either download the files yourself, or you’ll find them in the repository which accompanies this article.

Before we can get into the nitty-gritty of face detection, we need to build a simple file upload mechanism. This will allow users to upload an image, which we’ll resize to a sensible size and then show it back to them.

All we’re doing here is grabbing an uploaded file, ensuring it’s an image, checking it’s of a minimum size, and if it is, we’re resizing it to 960px. The face detection code has been left out for now. We’ll come to that in a moment.

I won’t go into too much detail about this process, as it’s not the primary focus of the article — but if you want to understand what’s going on, check out the multer and easyimage documentation.

Next we need to read the image using the OpenCV library. Behind the scenes, this converts the image into a matrix of pixels, which it can then run the feature detection algortithm on.

The method we use to do this has the following signature:

cv.readImage(filepath, function(err, im){
// do something with the matrix referred to by the im variable
});

Because we’re using the async module, we can simply pass it the callback as its second argument. The first argument is the destination dst; that is, the result of the resizing process. So the function in question looks like this:

All being well, the faces variable will contain an array of hashes, one for each face it’s found. Each hash will contain x and y co-ordinates (0,0 being the top-left corner of the image), along with a width and height — thus defining the area in the image that the face is thought to cover.

In our final callback — the second argument to async.waterfall() — we’re rendering an error template if something went wrong, otherwise we render the result, which we pass the filename of the uploaded image, along with our array of faces data.

What we’re doing here is wrapping the image in a <div>, which we’ve assigned position: relative, then rendered a link for each face. Each link is shown as an absolutely positioned block, and we use the face data to define its position and dimensions.

Now run the application:

node index.js

Note that you may see the following warning:

libdc1394 error: Failed to initialize libdc1394

Because libdc1394 isn’t required for our purposes, you can simply disable it as noted in this Stackoverflow answer with the following command:

sudo ln /dev/null /dev/raw1394

Use this with caution, as it may impact other applications installed on your system

Now visit the application in your browser. If you’re using Vagrant, you’ll find it here:

http://192.168.10.10:8080/

All being well, you should see the upload form:

Here’s an example of the results of a successful face detection attempt:

You can grab the image shown in the screenshot from here, or try with your own images. One thing to note is that this method requires that the faces be in full view, front-facing and upright.

Summary and Further Reading

That concludes our brief introduction to face detection, during which we’re built the basis of a clone of Facebook’s photo tagging widget.

If you’d like a really in-depth guide to the implementation of the Viola-Jones object detection framework, you might be interested in this thesis by Ole Helvig Jensen at the Technical University of Denmark. You might also find this YouTube video useful background.

There’s a lot more to explore in OpenCV; a good start is the documentation and in particular the tutorials; note, though, that these are primarily C++ focussed.

What other applications can you come up with? Let me know in the comments.

Lukas is a freelance web and mobile developer based in Manchester in the North of England. He's been developing in PHP since moving away from those early days in web development of using all manner of tools such as Java Server Pages, classic ASP and XML data islands, along with JavaScript - back when it really was JavaScript and Netscape ruled the roost. When he's not developing websites and mobile applications and complaining that this was all fields, Lukas likes to cook all manner of World foods.