7Structure de base de OpenCVOpenCV 2.x is a C++ library as opposed to OpenCV 1.xSignificant changes in module structure since version 2.2Modules: core, imgproc, video, calib3d, features2d, objdetect, highgui, gpu...

8OpenCV core functionalityAll the OpenCV classes and functions are placed into the cv namespacecore - compact module defining basic data structures and basic functions used by all other modulesBasic image class cv::Mat26/10/11

9cv::Mat memory managementOpenCV handles all the memory automaticallyMemory is freed automatically when neededWhen a Mat instance is copied, no actual data is really copiedTo make a real copy, use Mat::clone26/10/11

19HighGUI: Creating Interfaces IStart off by creating a program that will constantly input images from a camera#include <cv.h>#include <highgui.h>int main(){CvCapture* capture = 0;capture = cvCaptureFromCAM(0);if(!capture)printf("Could not initialize capturing...\n");return -1;}cvNamedWindow("video");This code creates a capture structure pointing to camera #0 and creates a window named “video”26/10/11

20HighGUI: Creating Interfaces IICreate two variables holding the values of the trackbars we’ll createint bright=128, contrast=26;And now we actually create the trackbars:cvCreateTrackbar("brightness", //name of the trackbar"video", //name of the window&bright, //pointer to a variable that will hold the value of the trackbar)255, //maximum value of the trackbar (minimum is always 0)NULL); //A callback function (which is called whenever the position of the trackbar is changed)cvCreateTrackbar("contrast", "video", &contrast, 50, NULL);Start the infinite loop requesting for frames:while(true){IplImage* frame = 0;frame = cvQueryFrame(capture);if (!frame)break;26/10/11

21HighGUI: Creating Interfaces IIIbright is in range [0,255], thus subtract 128 to have a convenient range to reduce to increase brightness.Modify image contrast and brightness by adding to every pixel bright value and scaling by contrastcvAddS(frame, cvScalar(bright-128,bright-128,bright- 128), frame);Display the image in the window “video” until the Esc key (ASCII = 27) is pressedcvShowImage("video", frame);int c = cvWaitKey(20);if ((char)c==27)break;}cvReleaseCapture(&capture);return 0;26/10/11

23OpenCV: image filteringIn this tutorial you will learn how to apply diverse linear filters to smooth images using OpenCV functions such as:BlurGaussian blurMedian blurBilateral filter26/10/11

24Theory Smoothing (blurring) is a simple and frequently used operationThere are many reasons for smoothing, e.g. noise suppressionTo perform a smoothing operation we will apply a filter to our image. The most common type of filters are linear, in which an output pixel’s value (i.e. 𝑔(𝑖,𝑗)) is determined as a weighted sum of input pixel values (i.e. 𝑓(𝑖+𝑘,𝑗+𝑙)) :𝑔 𝑖,𝑗 = 𝑘,𝑙 𝑓 𝑖+𝑘,𝑗+𝑙 ℎ(𝑘,𝑙)ℎ(𝑘,𝑙) is called the kernel, which is nothing more than the coefficients of the filter.It helps to visualize a filter as a window of coefficients sliding across the image.26/10/11

26Gaussian Filter IProbably the most useful filter (although not the fastest). Gaussian filtering is done by convolving each point in the input array with a Gaussian kernel.1D Gaussian kernel26/10/11

27Gaussian Filter II Pixel located in the middle has the biggest weight.The weight of its neighbors decreases as the spatial distance between them and the center pixel increases.2D Gaussian kernelwhere 𝜇 is the mean (the peak) and 𝜎 represents the variance (per each of the variables 𝑥 and 𝑦)26/10/11

28Median filterThe median filter run through each element of the signal (in this case the image) and replace each pixel with the median of its neighboring pixels (located in a square neighborhood around the evaluated pixel).The median of a finite list of numbers can be found by arranging all the observations from lowest value to highest value and picking the middle one.26/10/11

29Bilateral FilterConsidered filters main goal were to smooth an input image. However, sometimes the filters do not only dissolve the noise, but also smooth away the edges. To avoid this (at certain extent at least), we can use a bilateral filter.In an analogous way as the Gaussian filter, the bilateral filter also considers the neighboring pixels with weights assigned to each of them.These weights have two componentsThe first component is the same weighting used by the Gaussian filterThe second component takes into account the difference in intensity between the neighboring pixels and the evaluated one.26/10/11

30Usage examples Box filterblur(src, dst, Size( filt_size_x, filt_size_y), Point(-1,-1));src: Source imagedst: Destination imageSize( w,h ): Defines the size of the kernel to be used ( of width w pixels and height h pixels)Point(-1, -1): Indicates where the anchor point (the pixel evaluated) is located with respect to the neighborhood. If there is a negative value, then the center of the kernel is considered the anchor point.Gaussian blurGaussianBlur( src, dst, Size(filt_size_x, filt_size_y ), 0, 0 );Size(w, h): The size of the kernel to be used (the neighbors to be considered). and have to be odd and positive numbers otherwise the size will be calculated using the and arguments.sigma_x: The standard deviation in x. Writing 0 implies that is calculated using kernel size.sigma_y: The standard deviation in y. Writing 0 implies that is calculated using kernel size.26/10/11

35Face detection IIBasic idea: slide a window across image and evaluate a face model at every locationChallenges:Sliding window detector must evaluate tens of thousands of location/scale combinationsFaces are rare: 0–10 per imageFor computational efficiency, we should try to spend as little time as possible on the non-face windowsA megapixel image has ~106 pixels and a comparable number of candidate face locationsTo avoid having a false positive in every image, our false positive rate has to be less than 10-626/10/11

36Object classificationTakes as input 𝑛-dimensional vector of parameters 𝑣Training step: given a number of samples of each class and corresponding feature vectors -> builds 𝑛-dimensional space partitioningClassification: using acquired partitioning, make decision to which class object belongs to based on its feature vectorFeature vector is any reasonable set of parameters that makes good separation of the objects of different class26/10/11

41Fast computation with integral imageThe integral image computes a value at each pixel (x,y) that is the sum of the pixel values above and to the left of (x,y), inclusiveSum of values inside the rectangle sum = A – B – C + D(x,y)DBCA26/10/11

42Feature selectionFor a 24x24 detection region, the number of possible rectangle features is ~160,000!At detection time, it is impractical to evaluate the entire feature setCan we create a good classifier using just a small subset of all possible features?How to select such a subset?26/10/11

43Boosting Learn a single simple classifier Classify the dataLook at where it makes errorsReweight the data so that the inputs where we made errors get higher weight in the learning processNow learn a 2nd simple classifier on the weighted dataCombine the 1st and 2nd classifier and weight the dataaccording to where they make errorsLearn a 3rd classifier on the weighted dataFinal classifier is the combination of all T classifiersThis procedure is called “Boosting” – works very well in practice26/10/11

51Classifier cascadeWe start with simple classifiers which reject many of the negative sub-windows while detecting almost all positive sub-windowsPositive response from the first classifier triggers the evaluation of a second (more complex) classifier, and so onA negative outcome at any point leads to the immediate rejection of the sub-windowFACEIMAGESUB-WINDOWClassifier 1TClassifier 3FNON-FACEClassifier 2

52Classifier cascadeA detection rate of 0.9 and a false positive rate on the order of 10-6 can be achieved by a 10-stage cascade if each stage has a detection rate of 0.99 ( ≈ 0.9) and a false positive rate of about 0.30 (0.310 ≈ 6×10-6)26/10/11

54OpenCV face detectionOpenCV already have cascade classifiers and Haar features implementedPre-trained cascades for face detection as well as eyes and some more are also included! (Look in "data/haarcascades/ “)26/10/11