Introduction

The facial recognition has been a problem worked on around the world for many persons; this problem has emerged in multiple fields and sciences, especially in computer science, others fields that are very interested In this technology are: Mechatronic, Robotic, criminalistics, etc. In this article I work in this interesting topic using EmguCV cross platform .Net wrapper to the Intel OpenCV image processing library and C# .Net, these library’s allow me capture and process image of a capture device in real time. The main goal of this article is show and explains the easiest way how implement a face detector and recognizer in real time for multiple persons using Principal Component Analysis (PCA) with eigenface for implement it in multiple fields.

Background

Facial recognition is a computer application composes for complex algorithms that use mathematical and matricial techniques, these get the image in raster mode(digital format) and then process and compare pixel by pixel using different methods for obtain a faster and reliable results, obviously these results depend of the machine use to process this due to the huge computational power that these algorithms, functions and routines requires, these are the most popular techniques used for solve this modern problem:

Techniques:

Traditional

Some facial recognition algorithms identify faces by extracting landmarks, or features, from an image of the subject's face. For example, an algorithm may analyze the relative position, size, and/or shape of the eyes, nose, cheekbones, and jaw. These features are then used to search for other images with matching features. Other algorithms normalize a gallery of face images and then compress the face data, only saving the data in the image that is useful for face detection. A probe image is then compared with the face data. One of the earliest successful systems is based on template matching techniques applied to a set of salient facial features, providing a sort of compressed face representation. Recognition algorithms can be divided into two main approaches, geometric, which looks at distinguishing features, or photometric, which is a statistical approach that distill an image into values and comparing the values with templates to eliminate variances. Popular recognition algorithms include Principal Component Analysis with eigenface, Linear Discriminate Analysis, Elastic Bunch Graph Matching fisherface, the Hidden Markov model, and the neuronal motivated dynamic link matching.

text taken from [1]

An example of EigenFaces:

image taken from [4]

3-D

A newly emerging trend, claimed to achieve previously unseen accuracies, is three-dimensional face recognition. This technique uses 3-D sensors to capture information about the shape of a face. This information is then used to identify distinctive features on the surface of a face, such as the contour of the eye sockets, nose, and chin. One advantage of 3-D facial recognition is that it is not affected by changes in lighting like other techniques. It can also identify a face from a range of viewing angles, including a profile view. Even a perfect 3D matching technique could be sensitive to expressions. For that goal a group at the Technion applied tools from metric geometry to treat expressions as isometries.

image taken from [2]

Skin texture analysis

Another emerging trend uses the visual details of the skin, as captured in standard digital or scanned images. This technique, called skin texture analysis, turns the unique lines, patterns, and spots apparent in a person’s skin into a mathematical space Tests have shown that with the addition of skin texture analysis, performance in recognizing faces can increase 20 to 25 percent. It is typically used in security systems and can be compared to other biometrics such as fingerprint or eye iris recognition systems.

text taken from [1]

EmguCV

Emgu CV is a cross platform .Net wrapper to the Intel OpenCV image processing library. Allowing OpenCV functions to be called from .NET compatible languages such as C#, VB, VC++, IronPython etc. The wrapper can be compiled in Mono and run on Linux / Mac OS X.

text taken from [3]

In my own words EmguCV is an awesome Wrapper, this let make very interesting things and tasks of computer vision, this library set let do an unlimited amount of wonderful projects in this field, EmguCV have many functions that let us work with CPU and GPU increases the performance dramatically with the latest mentioned.

This Awesome SW project let work and do:

Optical Character Recognition(OCR)

Face Detection

Pedestrian Detection

Kinect projects

3D reconstruction

SURF feature detector ...between many others interesting tasks.

EmguCV basics: How i start to work?

If you don't had work never with this wrapper, you want see how add refereces to project or solve troubles look this god article/tutorial by C_Johnson:

Passing to FrameGrabber event (main part of prototype) we use the most important methods and objects: DetectHaarCascade And EigenObjectRecognizer and perform operations For each face detected in one frame:

haarObj: Haar classifier cascade in internal representation scaleFactor: The factor by which the search window is scaled between the subsequent scans, for example, 1.1 means increasing window by 10%

minNeighbors: Minimum number (minus 1) of neighbor rectangles that makes up an object. All the groups of a smaller number of rectangles than min_neighbors-1 are rejected. If min_neighbors is 0, the function does not any grouping at all and returns all the detected candidate rectangles, which may be useful if the user wants to apply a customized grouping procedure

flag: Mode of operation. Currently the only flag that may be specified is CV_HAAR_DO_CANNY_PRUNING. If it is set, the function uses Canny edge detector to reject some image regions that contain too few or too much edges and thus cannot contain the searched object. The particular threshold values are tuned for face detection and in this case the pruning speeds up the processing.

minSize: Minimum window size. By default, it is set to the size of samples the classifier has been trained on (~20x20 for face detection)

How to Train the Prototype?

I do this part the most easy possible, the prototype detect faces constantly (Each frame) and you can add this detected face in the image database with one label respectably, the face trained image will show in the imageBoxFrameGrabber and the process will be finished!!

Keep in mind: The face recognition algorithms based in PCA (Principal Component Analysis) do multiple comparisons and matches between a face detected and the trained images stored in binary database for this reason And for improve the accurate of recognition you should add several images of the same person in different angles, positions and luminance conditions, this training do this prototype solid and very accurate.

Example:

Code of training button (This perform the adding of training faces and labels for each):

How to improve the performance for slower CPUs?

All image processing algorithms demand many computational power, in this case the internals process carried on for the CPU with this sw prototype are so hard for slower o monocore CPUS, the Easy way for improve the performance of this Demo is modify the parameters that use the DetectHaarCascade method, these allow decrement the number of iteration, critic sections and Comparisons of the real time image captured for the Webcam improving notoriously the application performance.

Keep in mind: reduce the values of these parameters will affect the efficiency of recognition Algorithms.

First option:

For a faster operation on real video images the settings are: scale_factor=1.2, min_neighbors=2, flags=CV_HAAR_DO_CANNY_PRUNING, min_size=<minimum> (for example, ~1/4 to 1/16 of the image area in case of video conferencing).

Keep in mind: you can copy these files in Windows/System32/ folder and forget your problems of dependencies for this and other project that use Emgu or openCV(Emgu.CV.dll, Emgu.CV.UI.dll, Emgu.Util.dll SHOULD go always in the bin or .exe folder) or Download all ready project(Optimized version) and files here:

Greetings and Regards
It's me you saw a real-time face detection and recognition. Was excellent. It may help me, so I can use your proposed method, if the input is an image. Not a picture in real time from a webcam.
In other words: offline time
Thanks a lot

Hi sergio,
i would like to know how can i insert more data, instead of by adding name.
can i insert more data on a profile? such as enter the picture`s information, eg. name, age, and gender.
please advice, thank you.

sorry for bothering. i able to adding in the data.
but i still don`t understand "NamePersons[t-1] = name"
what`s the meaning of [t-1]?
can you kindly explain? i wish to understand and learn more.
thanks a lot.

You can change the size of image captures(320x240), comparing(100x100) and training(320x240) in the code.

the default is 320x240 you can increase these valor to 1280x720(720p) or 1920x1080(1080p) the amount of pixels increase the data to compare but decrease the performance dramatically in high resolutions because are millions of pixels to process in each frame.

In other hand only the type of sensor(high quality, low noise) can increase the accuracy of matching.