Face Detection in Java - Haar Cascade with JJIL (how-to)

Published Wed, 12 Jan 2011 • 61 comments

A requirement came up on a recent project to automatically crop displayed profile images of people to just the "face" area for a thumbnail.

This seems like a job for a face detection algorithm. Searching for appropriate open-source Java implementations didn't yield too many results, however I was successful with JJIL - Jon's Java Imaging Library, which is open sourced under the LGPL licence.

JJIL is targeted at Java ME / Android platforms and doesn't have much documentation or a particularly intuitive API (not complaining, as clearly some stellar work has gone into it, kudos to Jon Webb, it's creator).

In the end I muddled through, detecting faces in an image in a standard Java project, but given it took me a while to get everything working, I thought I'd write a quick guide to help out others that are trying to achieve similar results.

Getting the Right JARs

First things first, I had trouble getting the published JAR files to work happily together. There seems to be some sort of version mismatch issue between the core and J2SE versions.

This is a build of the current trunk JJIL code, and the Java SE additions (jjil-j2se).

This build is guaranteed to work with the code examples below.

Basic Process

I'm going to try to explain the basic process of detecting the faces in terms of input and output data.

The key file provided by JJIL for easy face (and other body part) detection is Gray8DetectHaarMultiScale.java

This operation is applied to an 8-bit greyscale input image, in combination with a pre-defined Haar Cascade profile. The profile determines which areas of the image are "detected". So you would want (for example) a profile to detect the frontal face features. JJIL provides several profiles out of the box.

The output image is a mask of the area of image where faces are detected (white) and the areas where no face was detected (black). This isn't tremendously useful, as we'd usually rather just have the rectangular areas in coordinate form - I'll address this later, after walking through the process.

Read an image from disk (.JPG, etc.)

Convert it into a jjil.core.Image

Generally we'll have an RGB image (colored image) and so need to convert it to 8-bit greyscale, which is what the Gray8DetectHaarMultiScale class requires.

Create a new instance of Gray8DetectHaarMultiScale with the Haar profile for detecting faces (or other body part if that's what you're looking for).

Apply Gray8DetectHaarMultiScale to our 8-bit grey image.

Retrieve result from Gray8DetectHaarMultiScale.

The figure below shows, the source image, overlayed with the resulting mask, from step #6

So as you can see, the masks correctly identify the faces the two people in the image.

The Code

Here's a small Java class that demonstrates how to read an image and apply the process described above.

Note: all the code in the article can be downloaded as a full project at the end of the article.

Getting Face Rectangles Instead

It would be more useful in many cases, to get a collection of Rectangles, in coordinate form, instead of an image mask.

There is a version of DetectHaarMultiScale in the JJIL project SVN, which implements a "getRectangles" method to retrieve this data. Unfortunately the source is incompatible with the rest of the library in SVN, so it may be WIP or an abandoned version of the code.

To get around this, I created my own version of Gray8DetectHaarMultiScale, which you can download here - Gray8DetectHaarMultiScale

If you run this, you will note that it actually detects 3 faces in the image. This is common, as the way the Haar algorithm works, is by resizing the image to different scales and running a fixed size matrix over the image. It is possible for the same face to be detected at different scales and so you end up with rectangles within rectangles. It is a pretty trivial matter to remove these "extra" rectangles though by just checking if they are fully contained by another and ignoring them accordingly.

Download the Project

To save you some time, here's the full example as a Netbeans project that you can download and run, play with, etc. Have fun!

Could you please elaborate on this part ? "If you run this, you will note that it actually detects 3 faces in the image. This is common, as the way the Haar algorithm works, is by resizing the image to different scales and running a fixed size matrix over the image. It is possible for the same face to be detected at different scales and so you end up with rectangles within rectangles. It is a pretty trivial matter to remove these ?extra? rectangles though by just checking if they are fully contained by another and ignoring them accordingly."

I ran this example on various images which I have, every time, It gives me output as "3 faces found "

I appreciate your help.

Thanks,Gaurav

Tres Finocchiaroon Mon, 21 Nov 2011 at 10:05

Excellent tutorial. I'm still a bit confused on the txt files (I assume they're "tweaks" to help detection depending on the photo type).

The only one that seems to work predictably for me is HCSB.txt. The rest are hit and miss.

Also, I've noticed that faces at angles aren't detected easily.

I'm using this to thumbnail photos from drivers licenses scans. Thanks for the work to get this to J2SE and the excellent tutorial!

-Tres

Richard Nicholson Mon, 21 Nov 2011 at 18:02

@Gaurav The code in the sample does not remove the "duplicate" matches, which is why you get 3 images found.

To implement this you need to loop through the matches and ensure that the rectangle for each one is not wholly contained within another in the set. Once you remove the ones which are "contained" within others you have the final list of matches.

Gauravon Tue, 22 Nov 2011 at 04:27

Hi Richard,

Thanks for your reply. Can you please tell me how to approach to remove those duplicates and find exact answer. I tried with some more images which has single face. Sometimes It gave me output as 0 faces found.I am trying to modify this code so that i achieve following task.Input: Image fileOutput: # of faces found in images.

Thanks,Gaurav

Piyushon Mon, 19 Dec 2011 at 12:49

Hello Richard,how can i construct an image out of detected face....so i could use it for further processing such as matching them with a set of images...kind of Facial Recognition....

hello sir i am tarun and my last sem of b.tech is going on .my faculty give me face detection as my final year project but i am not able to make so please help me i am in trouble i have to give my presentation of 10th feb as my 5th presentation but my code is not runnig please help me.my exam is from 15th feb and i am going to give an exam of GATE on 12th feb so now what i doi have to make a module of face detection and other module can be easily prepared by me.so please help me...................

Daniel Dario Morales Salason Sun, 12 Feb 2012 at 00:09

Hi, nice How-to!

I wonder how can I overlay the first image with the result, I mean I want the original picture with the rectangles in the faces, How can I do that? when you create Gray8Rgb and then push the resulting image we get a black backgroung with white rectangles but if you overlay the original pic with that result you cannot see the original picture in the background with the rectangles in the foreground, you see only the resulting pic with the background in black with the white rectangles. Can you help with this? thanks!

Greetings!

androidiston Sun, 12 Feb 2012 at 17:25

Thank you for you help so much.

Could you please tell me how i can make this work on Android? I tried using the detect face in jjil but it keeps giving the application suddenly stopped.

Should i also used the HCSB.txt in android? where do i place it on the phone?

and can i use RgbImage instead of RgbImageJ2SE?

thanks again

lkf2002on Tue, 21 Feb 2012 at 11:34

conv.toFile((RgbImage)g2rgb.getFront(), output.getCanonicalPath());

how do i show the mask image in a java panel without to create the file first ?

is it possbile to covert (RgbImage)g2rgb.getFront() to a planar image or buffered image?

lkf2002on Tue, 21 Feb 2012 at 11:40

i know how to complete the face detection, but i dun know how to show the mask image in a java panel without to create the file first .

To complete the face detection, we need 2 files, one is the original file, another one is the mask file ( black background and white squre).

apply image subtraction, done!

i forget wether the 1st image - 2nd image or the 2nd one subtract the first one

plx help me , urgent.

$aM EaGlEon Sat, 17 Mar 2012 at 04:27

Thanks a ton for sharing this article.......Its very helpful for me........

I will be very thankful if someone will share something related to capture facial features for comparing two faces.........My aim is to capture face from one image and identify it in other image for that purpose i need to capture facial features.......

Thanks in Advance....

Galon Fri, 20 Apr 2012 at 08:55

I still can't figure out how to get the resulting face - I cant seem to find how to subtract the 2 images, or just take the rectangle recognized from the original image.. I would greatly appreciate it if somehow could help me with that!

Markon Fri, 11 May 2012 at 09:28

I copied several jpeg's to the ~JJILExample\build\classes\jjilexample\test.jpg fileand re ran `java -jar JJILExample.jar'. The first attempt was a file with one face,then with two then with zero (a landscape). I always get the result "Three facesfound" and when I look at the c:\temp\result.jpg file, it always has two rectanglesat the same location as in the original test picture. What am I doing wrong?

sandaon Wed, 13 Jun 2012 at 01:57

Thanks for help.could you please tell me how can I get original image with rectangles on the faces?

Rashid Padikkalon Mon, 23 Jul 2012 at 04:16

sir actually i'm a fresher in java. now i'm doing a project cost sensitive face recognition.. so i need a code for recognize the face.. could u help me sir..? i think u r a great programmer. that's why i'm asking to you... waiting for your reply... please reply me to [email protected]

Alexandruon Sun, 28 Oct 2012 at 13:40

Great article!One quick typo:"Unfortunately the source in incompatible"*is

Ranjion Tue, 8 Jan 2013 at 05:08

Can you tell me how can I get original image with rectangles on the faces? or the coordinates of those rectangles ?

piyush kumar lodayaon Fri, 29 Mar 2013 at 11:52

thnx a lot ..was recently working on face dtection ... found your post very lucid and thanks for giving edited jar files... would love to read more

sousouon Thu, 25 Apr 2013 at 18:16

i want some help concerned this code so please did you know how can i convert it to Anndroid (eclipse) and thank you

Kacharyon Wed, 10 Jul 2013 at 10:17

This one is quite useful! Specially, JJIL and the rectangle from you.

Nirmala Valghudeon Wed, 7 Aug 2013 at 04:41

Thanks a lot sir,but this jar file does not display .Sir this is my kindly request to you can you guide me about how can crop face image in hair,eyes,nose,etc.after the button click in a selected image.and also how can we match them that means Croped image and full image.
I am working on a project Criminal Face Detecting System.I want to crop criminal face and match with eyewitness's criminal face.Sir,please guide me.

Karthikon Thu, 10 Oct 2013 at 02:22

Hi Richard the article which you provide is very nice but how could I implement this on android. As, I am facing the problems in android how to detect the face and compare two images with facial expressions on it.
If possible could you send the code or can you give me any suggestion on it.
Thanks.

prasad nikumbhon Thu, 12 Dec 2013 at 10:34

hey richard tour code does not provide only faced square pixels.i mean the portion that hihglights the square.so can u provide me the same. becoz i need to separate those pixels out