Introduction

Eyes are the most important features of the human face. So effective usage of eye movements as a communication technique in user-to-computer interfaces can find place in various application areas.

Eye tracking and the information provided by the eye features have the potential to become an interesting way of communicating with a computer in a human-computer interaction (HCI) system. So with this motivation, designing a real-time eye feature tracking software is the aim of this project.

The purpose of the project is to implement a real-time eye-feature tracker with the following capabilities:

RealTime face tracking with scale and rotation invariance

Tracking the eye areas individually

Tracking eye features

Eye gaze direction finding

Remote controlling using eye movements

Instructions to Run and Rebuild TrackEye

Installation Instructions

Extract TrackEye_Executable.zip file. Before running TrackEye_636.exe, copy the two files SampleHUE.jpg and SampleEye.jpg to the C:\ folder. These two files are used for CAMSHIFT and Template-Matching algorithms.

There are no other steps to be followed by the user to run the software. There are no DLL dependencies as the software was built with the DLLs statically included.

Background

So far there has been a lot of work on eye detection and before the project, the previous methods were carefully studied to determine the implemented method. We can classify studies related to eye into two main categories as listed below:

Special Equipment Based Approaches

These type of studies use the necessary equipment which will give a signal of some sort which is proportional to the position of the eye in the orbit. Various methods that are current in use are Electrooculography, Infra-Red Oculography, Scleral search coils. These methods are completely out of our project.

Image Based Approaches

Image based approaches perform eye detections on the images. Most of the image based methods try to detect the eyes using the features of the eyes. Methods used so far are knowledge-based methods, feature-based methods (color, gradient), simple template matching, appearance methods. Another interesting method is “Deformable template matching” which is based on matching a geometrical eye template on an eye image by minimizing the energy of the geometrical model.

Implementation of TrackEye

The implemented project is on three components:

Face detection: Performs scale invariant face detection

Eye detection: Both eyes are detected as a result of this step

Eye feature extraction: Features of eyes are extracted at the end of this step

Face Detection

Two different methods were implemented in the project. They are:

Continuously Adaptive Means-Shift Algorithm

Haar Face Detection method

Continuously Adaptive Mean-Shift Algorithm

Adaptive Mean Shift algorithm is used for tracking human faces and is based on robust non-parametric technique for climbing density gradients to find the mode (peak) of probability distributions called the mean shift algorithm. As faces are tracked in video sequences, mean shift algorithm is modified to deal with the problem of dynamically changing color probability distributions. The block diagram of the algorithm is given below:

Haar-Face Detection Method

The second face detection algorithm is based on a classifier working with Haar-Like features (namely a cascade of boosted classifiers working with Haar-like features). First of all it is trained with a few hundreds of sample views of a face. After a classifier is trained, it can be applied to a region of interest in an input image. The classifier outputs a "1" if the region is likely to show face and "0" otherwise. To search for the object in the whole image, one can move the search window across the image and check every location using the classifier. The classifier is designed so that it can be easily "resized" in order to be able to find the objects of interest at different sizes, which is more efficient than resizing the image itself.

Eye Detection

Two different methods were implemented in the project:

Template-Matching

Adaptive EigenEye Method

Template-Matching

Template-Matching is a well-known method for object detection. In our template matching method, a standard eye pattern is created manually and given an input image, the correlation values with the standard patterns are computed for the eyes. The existence of an eye is determined based on the correlation values. This approach has the advantage of being simple to implement. However, it may sometimes be inadequate for eye detection since it cannot effectively deal with variation in scale, pose and shape.

Adaptive EigenEye Method

Adaptive EigenEye Method is based on the well-known method EigenFaces. However as the method is used for eye detection we named it as “EigenEye Method”. The main idea is to decompose eye images into a small set of characteristics feature images called eigeneyes, which may be thought of as the principal components of the original images. These eigeneyes function as the orthogonal basis vectors of a subspace called eyespace. However we know that the eigenface method is not scale invariant. To provide the scale invariance we can resize the eye-database once with the information gathered by the face detection algorithm (EyeWidth / FaceWidth ? 0.35), we can provide scale-invariant detection using only one database.

OpenCV Functions for Object Tracking and Detection

OpenCV Library offers a lot of image processing and object tracking & detection libraries. The main function used in these projects and their usage are given below:

Thanks your codes. I compiled the source code and recieved "cvaux.h missing " error. So I added the include files to project and compiled agian. But recieved other errors from cvaux.h head file. Can help me about this?
Thanks a lot!

Both video link address:
http: //v.youku.com/v_show/id_XODAxOTcxMjY4.html
http://v.youku.com/v_show/id_XODAxOTY3MDI4.html
currently do combine OPENCV2.3.0 the implementation of the eye-tracking, you can see the effect, tracking rate of 82.5% -98%

Both video link address: http: //v.youku.com/v_show/id_XODAxOTcxMjY4.html (without glasses) http://v.youku.com/v_show/id_XODAxOTY3MDI4.html (wearing glasses) currently do combine OPENCV the implementation of the eye-tracking, you can see the effect, tracking rate of 82.5% -98%

According to the "Charles Song" said, I download the VS2008 version of the Track Eye, and OpenCV_b3.1.exe, configured environment, compile and run the code, the program can run up, but there is still a lot of BUG, I want to explain, this program can only be run on windows Xp, higher than the XP version of the win7 no, OpenCV_b3.1.exe cv.dll, will directly report to memory errors, the XP is ok. Provides the ideas of this program, we can refer to the next. Now I'm in the basis of further write an algorithm to improve the capture rate of the eye, and the use of GPU to increase efficiency. I would like to thank the selfless dedication!

As I said before, this program as a reference, but still need to solve a lot of problems, the library is really old, I have used opencv2.3.1 in the re write a part inside I need to function, and the eye mark inside the very unstable, I improved a lot on this basis, including our own to obtain hard clipping algorithm! The hope behind can more exchanges!

Both video link address: http: //v.youku.com/v_show/id_XODAxOTcxMjY4.html (without glasses) http://v.youku.com/v_show/id_XODAxOTY3MDI4.html (wearing glasses) currently do combine OPENCV the implementation of the eye-tracking, you can see the effect, tracking rate of 82.5% -98%

I tried your sample.avi and everything works well.
But as long as I use a random .avi file in my computer, this error comes out (even if I change its name to Sample.avi)?
So it seems that this application only works well for the avi file you provided. Why?
Thanks

Where can i download the OpenCV Vb3.1.? I've been googling it for such a time just to download. Does anybody still have a copy of it? I am using the latest version of OpenCV but it has errors during compiling..