We wish to mark points along the curve where the magnitude is biggest. We can do this by looking for a maximum along a slice normal to the curve (non-maximum suppression). These points should form a curve. There are then two algorithmic issues: at which point is the maximum, and where is the next one? Khurram Hassan-Shafique Non-Maximum Suppression• Suppress the pixels in ‘Gradient Magnitude Image’ which are not local maximum

• If the gradient at a pixel is above ‘High’, declare it an ‘edge pixel’

• If the gradient at a pixel is below ‘Low’, declare it a ‘non-edge-pixel’• If the gradient at a pixel is between ‘Low’ and ‘High’ then declare it an ‘edge pixel’ if and only if it is connected to an ‘edge pixel’ directly or via pixels between ‘Low’ and ‘ High’

Khurram Hassan-Shafique Hysteresis Thresholding

M M  Threshold  25

High  35 Low  15Khurram Hassan-ShafiqueResultado de algoritmo de histerese– Subpixel Localization One can try to further localize the position of the edge within a pixel by analyzing the response to the edge enhancement filter– One common approach is to fit a quadratic polynomial to the filter response in the region of a maxima and compute the true maximum.

Camillo J. Taylor Examining the matrix– One way to decide on the presence of a corner is to look at the eigenvalues of the 2 by 2 matrix C. • If the area is a region of constant intensity we would expect both eigenvalues to be small • If it contains a edge we expect one large eigenvalue and one small one • If it contains edges at two or more orientations we expect 2 large eigenvalues

Camillo J. Taylor Finding corners

– One approach to finding corners is to find locations where the

smaller eigenvalue is greater than some threshold– We could also imagine considering the ratio of the two eigenvaluesComputing Image Gradients Corner Analysis– The ellipses indicate the eignvalues and eigenvectors of the C matrices Corner Detection• Basic idea: Find points where two edges meet—i.e., high gradient in two directions• “Cornerness” is undefined at a single pixel, because there’s only one gradient per point – Look at the gradient behavior over a small window• Categories image windows based on gradient statistics – Constant: Little or no brightness change – Edge: Strong brightness change in single direction – Flow: Parallel stripes – Corner/spot: Strong brightness changes in orthogonal directions Corner Detection: Analyzing Gradient Covariance• Intuitively, in corner windows both Ix and Iy should be high – Can’t just set a threshold on them directly, because we want rotational invariance

• Analyze distribution of gradient components over a window to differentiate between types from previous slide:

This figure displays results of feature detection over the dinosaur test sequence with the algorithm set to extract the 6 most "interesting" features at every image frame.It is interesting to note that although no attempt to extract frame-to-frame feature correspondences was made, the algorithm still extracts the same set of features at every frame.This will be useful very much in feature tracking. One More..

• Office sequence• Office Tracking Harris Detector: Some Properties

• Rotation invariance

Ellipse rotates but its shape (i.e. eigenvalues)

remains the same

Corner response R is invariant to image rotation

Harris Detector: Some Properties

• Partial invariance to affine intensity change

 Only derivatives are used => invariance

to intensity shift I  I + b  Intensity scale: I  a I

R Rthreshold

x (image coordinate) x (image coordinate)

Harris Detector: Some Properties• But: non-invariant to image scale!

All points will be Corner !

classified as edges Harris Detector: Some Properties • Quality of Harris detector for different scale changes -- Correspondences calculated using distance (and threshold) -- Improved Harris is proposed by Schmid et al -- repeatability rate is defined as the number of points repeated between two images w.r.t the total number of detected points.  Repeatability rate: # correspondences # possible correspondences

Fine/Low Coarse/High Scale Invariant Detection• The problem: how do we choose corresponding circles independently in each image? Scale Invariant Detection • Solution: – Design a function on the region (circle), which is “scale invariant” (the same for corresponding regions, even if they are at different scales)

Example: average intensity. For corresponding regions

(even of different sizes) it will be the same. – For a point in one image, we can consider it as a function of region size (circle radius)

s1 region size/scale s2 region size/scal

s1 region size/scale s2 region size/scale

• The ratio of the scales, at which the extrema were found for corresponding points in two rescaled images, is equal to the scale factor between the images. • Characteristic Scale: Given a point in an image, compute the function responses for several factors sn The characteristic scale is the local max. of the function (can be more than one). • Easy to look for zero-crossings of 2nd derivative than maxima. Scale Invariant Detection • A “good” function for scale detection: has one stable sharp peak

A normal distribution in a variate with mean  and variance 2 is a

statistic distribution with probability function Harris-Laplacian• Existing methods search for maxima in the 3D representation of an image (x,y,scale). A feature point represents a local maxima in the surrounding 3D cube and its value is higher than a threshold.• THIS (Harris-Laplacian) method uses Harris function first, then selects points for which Laplacian attains maximum over scales.• First, prepare scale-space representation for the Harris function. At each level, detect interest points as local maxima in the image plane (of that scale) – do this by comparing 8-neighborhood. (different from plain Harris corner detection)• Second, use Laplacian to judge if each of the candidate points found on different levels, if it forms a maximum in the scale direction. (check with n-1 and n+1) Scale Invariant Detectors • Experimental evaluation of detectors w.r.t. scale change

K.Mikolajczyk, C.Schmid. “Indexing Based on Scale Invariant Interest Points”. ICCV 2001 Scale Invariant Detection: Summary • Given: two images of the same scene with a large scale difference between them • Goal: find the same interest points independently in each image • Solution: search for maxima of suitable functions in scale and in space (over the image)

• Now we go on to: Affine transform (rotation + non-uniform scale) Affine Invariant Detection• Take a local intensity extremum as initial point• Go along every ray starting from this point and stop when extremum of function f is reached

points along the ray

• We will obtain approximately

corresponding regions

Remark: we search for scale

Affinely Invariant Regions”. BMVC 2000. Affine Invariant Detection• Algorithm summary (detection of affine invariant region): – Start from a local intensity extremum point – Go in every direction until the point of extremum of some function f – Curve connecting the points is the region boundary – Compute geometric moments of orders up to 2 for this region – Replace the region with ellipse

Affinely Invariant Regions”. BMVC 2000. Affine Invariant Detection : Summary• Under affine transformation, we do not know in advance shapes of the corresponding regions• Ellipse given by geometric covariance matrix of a region robustly approximates this region• For corresponding regions ellipses also correspond