Object detection with OpenCV

An interesting usage of the classification algorithms is object detection in live images. In this tutorial we will demonstrate how to detect a simple object using the open source library OpenCV. After a short description of OpenCV, we will see the steps needed to produce a model file using the OpenCV Cascade Classifier. This step is also called training the model. The model file will be then used to detect an object in the live images taken from a USB web camera. The model creation and the object detection code were tested on a laptop running Ubuntu 14.04 LTS. Basic C/C++ programming knowledge is required for this tutorial. A USB web camera is also needed.

OpenCV Library

OpenCV (Open Source Computer Vision) is a open source library which contains function for real-time image processing. The library is written in C++ and distributed under BSD License. It can be run under Linux, Windows, OS X, iOS, Android and has wrappers for Python, Java, C# and Ruby. It can be used for: object detection, motion detection, augmented reality, facial recognition or gesture recognition.

Trainig the model

In this section we will create a XML model file, which will be later used to detect the objects. For no special reason I choose a brezel (engl. prezel) as detection subject. The model creation process involves gathering a large amount of images containing the subject – we will call these positive images. We will also need a large a large amount of images not containing at all the searched object- negative images. Using the two folders and some description files we will run the OpenCV Cascade Classifier algorithm in order to produce the model file.

These are actual steps:

Step 1: I started by taking 201 JPG images of the brezel using a mobile phone.

Step 2: I installed GIMP for Ubuntu then I added David’s Batch Processor to the installation using the command:

1

sudo apt-getinstall gimp-plugin-registry

Step 3: I created a batch process to convert the original images to grayscale and to resize them to 60 x 80 pixels. These 201 images will be used in the next steps to produce additional training images

Step 5: Created a folder named positives and moved all the images to this folder.

Step 6: Created the positives.txt file. This is a descriptor file for the positives folder. Each line in this file contains the path to a positive image, the number of objects (brezels) in this image, the coordinates of the bounding rectangle containing the object.

1

2

find./positives-iname"*.jpg">positives.txt

sed-i's/.jpg/.jpg 1 0 0 60 80/'positives.txt

Step 7: Downloaded the negative image files from this tutorial. Moved all negative images in a folder called negatives.

Step 8: Created the negatives.txt file. This is a descriptor file for the negatives folder.

1

find./negatives-iname"*.jpg">negatives.txt

Step 9: Prepare a training dataset from 1200 positive images, using the following command:

Depending on the hardware the training can last a few hours. On my computer I got the model after 3:41 hours:

1

Training untilnow has taken0days3hours41minutes55seconds.

The resulting model file can be found found in the data folder and is named cascade.xml. I took this file and renamed it to brezel.xml. As a reference you can have a look at my trained model available in this archive brezel.zip.

Using the trained model to detect objects

In this section we will see how to create a simple C++ Project, add the needed libraries to the project, add the model and do the actual object detection. As development tool I used Eclipse Luna (4.4.0) with CDT (C/C++ Development Tools).

Following, are the steps needed to setup and run the project:

Step1: Start Eclipse and create a new project: File > New > Other > C++ Project.

Step2: Name the project BrezelDetectionProject, the press Finish.

Step3: Right click on the project and choose to create a new folder named src.

Step 4: Right click the src folder and create a new .cpp file, BrezelDetector.cpp.

Step 5: Add the following code to the .cpp file.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

#include <highgui.h>

#include <iostream>

#include <stdio.h>

#include <cv.h>

usingnamespacestd;

usingnamespacecv;

usingnamespacestd;

intmain(){

cvNamedWindow("Brezel detecting camera",1);

// Capture images from any camera connected to the system

CvCapture*capture=cvCaptureFromCAM(CV_CAP_ANY);

// Load the trained model

CascadeClassifier brezelDetector;

brezelDetector.load("src/brezel.xml");

if(brezelDetector.empty()){

printf("Empty model.");

return0;

}

charkey;

while(true){

// Get a frame from the camera

Mat frame=cvQueryFrame(capture);

std::vector<Rect>brezels;

// Detect brezels

brezelDetector.detectMultiScale(frame,brezels,1.1,30,

0|CV_HAAR_SCALE_IMAGE,Size(200,320));

for(inti=0;i<(int)brezels.size();i++){

Point pt1(brezels[i].x,brezels[i].y);

Point pt2(brezels[i].x+brezels[i].width,

brezels[i].y+brezels[i].width);

// Draw a rectangle around the detected brezel

rectangle(frame,pt1,pt2,Scalar(0,0,255),2);

putText(frame,"Brezel",pt1,FONT_HERSHEY_PLAIN,1.0,

Scalar(255,0,0),2.0);

}

// Show the transformed frame

imshow("Brezel detecting camera",frame);

// Read keystrokes, exit after ESC pressed

key=cvWaitKey(10);

if(char(key)==27){

break;

}

}

return0;

}

Step 6: The project does not know where to find the OpenCV libraries. To fix that go to Project > Properties. In the C/C++ Build section, go to Settings, in the right pane select Tool Settings. In the GCC C++ Compiler section go to Includes and add to Include paths (-l) the folder where OpenCV is installed. For me the the path is /usr/include/opencv.

Conclusion

We saw how we can use OpenCV to train a model for object detection and how to use this model to detect objects in live images. This is just a small use case of this computer vision library. Generally speaking, the computer vision has a wide application field and can handle areas like

No, it is not necessary to have a white background in the image containing the object. I took the pictures with different background colors in order to obtain a better model. The more diverse the background, the better.

It seems that you are trying to allocate too much memory. The width and height for the images are too large (540 x 960). As you can see, I used 60 x 80 images.
Also the number of positive and negative images is too low for the training to produce meaningful results.

Please try decreasing the image size and increasing the number of samples – positives and negatives.

i want to identify one animal from a photo. what kind of feature identify tht animal to processing?

Leonard Giura4 January 2016

Hi Nithy,

In order to identify any kind of object (the animal for example) in an image, you have to train the model using many positive images. Positive images are the images that contain the targeted object. The more images and diverse positions of the object in the images, the better. With the trained model you can then identify the object. So, the first step would be to gather images with the animal you want to identify.

Thank you!
I experimented with several image sizes and this was optimal in my case. If you choose a bigger size you will have a good model, but the learning will take longer. This size was sufficient to obtain a good model, yet not to spend too much time with the learning process. Finding the right image size was a trial and error process.

Does your Brezel Detector works for this particular Brezel(you used in your training). I mean, have you been using the same brezel object for training as well as detecting? Or your code works for the class ‘Brezel’

I used the same brezel for training and detecting, the model is not suited for all types of brezels. The quality of the model depends on the quality and quantity of the training images. For a more precise model I should have used images with different types of brezels in various positions and angles. For the purpose of my demo though, the current model was good enough.

The code in the .cpp file is all the code you will need to run the current demo. It is free for use. The most important part though, is the model file (brezel.xml) which is not shared yet. For your specific problem you should train a new model according to the steps I described in the tutorial.
For your last question: it depends on the training images. As I trained my model with images taken from the top, it would detect the brezel only from the top. Had I used training images with brezels taken at different angles, the application would most probably have detected the brezel in different angles.