A collection of sloppy snippets for scientific computing and data visualization in Python.

Friday, June 28, 2013

Shape Matching Experiments

Often, in Computer Vision tasks we have a model of an interesting shape and we want to know if this model matches with a target shape found through the analysis of the edges of an image. The target shape is never identical to the model we have, because it could be a representation of the model in a different pose or it could match only partially our model, so we have to find a proper transformation of the model in order to match the target. In this post we will see how to find the closest shape model to a target shape using the fmin function provided by Scipy.
We can represent a shape with a matrix like the following:

where each pair (xi,yi) is a "landmark" point of the shape and we can transform every generic point (x,y) using this function:

In this transformation we have that θ is a rotation angle, tx and ty are the translation along the x and the y axis respectively, while s is a scaling coefficient.
Applying this transformation to every point of the shape described by X we get a new shape which is a transformation of the original one according to the parameters used in T.
Given a matrix as defined above, the following Python function is able to apply the transformation to every point of the shape represented by the matrix:

Now, we want to ﬁnd the best pose (translation, scale and rotation) to match a model shape X to a target shape Y.
We can solve this problem minimizing the sum of square distances between the points of X and the ones of Y, which means that we want to find

where T' is a function that applies T to all the point of the input shape. This task turns out to be pretty easy using the fmin function provided by Scipy:

In the code above we have defined a cost function which measures how close the two shapes are and another function that minimizes the difference between the two shapes and returns the transfomation parameters.
Now, we can try to match the trifoil shape starting from one of its transformations using the functions defined above:

and the value of the parameters of the transformation at each iteration:

pl.plot(allsol)
pl.show()

From the graphs above we can observe that the the minimization process found its solution after 150 iterations. Indeed, after 150 iterations the error become very close to zero and there is almost no variation in the parameters.
Now we can visualize the minimization process plotting some of the solutions in order to compare them with the target shape:

In the graph above we can see that the initial solutions (the green ones) are very different from the shape we are trying to match (the dashed blue ones) and that during the process of minimization they get closer to the target shape until they fits it completely.
In the example above we used two shapes of the same form. Let's see what happen when we have a starting shape that has a different form from the target model: