OpenCV: Image Processing using Morphological Filters

Morphological filtering is a theory developed in the 1960s for the analysis and processing of discrete images. It defines a series of operators which transform an image by probing it with a predefined shape element. The way this shape element intersects the neighborhood of a pixel determines the result of the operation. This article presents the most important morphological operators. It also explores the problem of image segmentation using algorithms working on the image morphology.

Eroding and dilating images using morphological filters

Erosion and dilation are the most fundamental morphological operators. Therefore, we will present them in this first recipe.

The fundamental instrument in mathematical morphology is the structuring element. A structuring element is simply defined as a configuration of pixels (a shape) on which an origin is defined (also called anchor point). Applying a morphological filter consists of probing each pixel of the image using this structuring element. When the origin of the structuring element is aligned with a given pixel, its intersection with the image defines a set of pixels on which a particular morphological operation is applied. In principle, the structuring element can be of any shape, but most often, a simple shape such as a square, circle, or diamond with the origin at the center is used (mainly for efficiency reasons).

Getting ready

As morphological filters usually work on binary images, we will use a binary image produced through thresholding. However, since in morphology, the convention is to have foreground objects represented by high (white) pixel values and background by low (black) pixel values, we have negated the image.

How to do it...

Erosion and dilation are implemented in OpenCV as simple functions which are cv::erode and cv::dilate. Their use is straightforward:

The two images produced by these function calls are seen in the following screenshot. Erosion is shown first:

Followed by the dilation result:

How it works...

As with all other morphological filters, the two filters of this recipe operate on the set of pixels (or neighborhood) around each pixel, as defined by the structuring element. Recall that when applied to a given pixel, the anchor point of the structuring element is aligned with this pixel location, and all pixels intersecting the structuring element are included in the current set. Erosion replaces the current pixel with the minimum pixel value found in the defined pixel set. Dilation is the complementary operator, and it replaces the current pixel with the maximum pixel value found in the defined pixel set. Since the input binary image contains only black (0) and white (255) pixels, each pixel is replaced by either a white or black pixel.

A good way to picture the effect of these two operators is to think in terms of background (black) and foreground (white) objects. With erosion, if the structuring element when placed at a given pixel location touches the background (that is, one of the pixels in the intersecting set is black), then this pixel will be sent to background. While in the case of dilation, if the structuring element on a background pixel touches a foreground object, then this pixel will be assigned a white value. This explains why in the eroded image, the size of the objects has been reduced. Observe how some of the very small objects (that can be considered as "noisy" background pixels) have also been completely eliminated. Similarly, the dilated objects are now larger and some of the "holes" inside of them have been filled.

By default, OpenCV uses a 3x3 square structuring element. This default structuring element is obtained when an empty matrix (that is cv::Mat()) is specified as the third argument in the function call, as it was done in the preceding example. You can also specify a structuring element of the size (and shape) you want by providing a matrix in which the non-zero element defines the structuring element. In the following example, a 7x7 structuring element is applied:

The origin argument cv::Point(-1,-1) means that the origin is at the center of the matrix (default), and it can be defined anywhere on the structuring element. The image obtained will be identical to the one we obtained with the 7x7 structuring element. Indeed, eroding an image twice is like eroding an image with a structuring element dilated with itself. This also applies to dilation.

Finally, since the notion of background/foreground is arbitrary, we can make the following observation (which is a fundamental property of the erosion/dilation operators). Eroding foreground objects with a structuring element can be seen as a dilation of the background part of the image. Or more formally:

The erosion of an image is equivalent to the complement of the dilation of the complement image.

The dilation of an image is equivalent to the complement of the erosion of the complement image.

There's more...

It is important to note that even if we applied our morphological filters on binary images here, these can also be applied on gray-level images with the same definitions.

Also note that the OpenCV morphological functions support in-place processing. This means you can use the input image as the destination image. So you can write:

cv::erode(image,image,cv::Mat());

OpenCV creates the required temporary image for you for this to work properly.

Opening and closing images using morphological filters

The previous recipe introduced the two fundamental morphological operators: dilation and erosion. From these, other operators can be defined. The next two recipes will present some of them. The opening and closing operators are presented in this recipe.

How to do it...

In order to apply higher-level morphological filters, you need to use the cv::morphologyEx function with the appropriate function code. For example, the following call will apply the closing operator:

How it works...

The opening and closing filters are simply defined in terms of the basic erosion and dilation operations:

Closing is defined as the erosion of the dilation of an image.

Opening is defined as the dilation of the erosion of an image.

Consequently, one could compute the closing of an image using the following calls:

// dilate original imagecv::dilate(image,result,cv::Mat());// in-place erosion of the dilated imagecv::erode(result,result,cv::Mat());

The opening would be obtained by inverting these two function calls.

While examining the result of the closing filter, it can be seen that the small holes of the white foreground objects have been filled. The filter also connects together several of the adjacent objects. Basically, any holes or gaps too small to completely contain the structuring element will be eliminated by the filter.

Reciprocally, the opening filter eliminated several of the small objects in the scene. All of the ones that were too small to contain the structuring element have been removed.

These filters are often used in object detection. The closing filter connects together objects erroneously fragmented into smaller pieces, while the opening filter removes the small blobs introduced by image noise. Therefore, it is advantageous to use them in sequence. If our test binary image is successively closed and opened, we obtain an image showing only the main objects in the scene, as shown below. You can also apply the opening filter before closing if you wish to prioritize noise filtering, but this can be at the price of eliminating some fragmented objects.

It should be noted that applying the same opening (and similarly the closing) operator on an image several times has no effect. Indeed, with the holes having been filled by the first opening, an additional application of this same filter will not produce any other changes to the image. In mathematical terms, these operators are said to be idempotent.

Detecting edges and corners using morphological filters

Morphological filters can also be used to detect specific features in an image. In this recipe, we will learn how to detect lines and corners in a gray-level image.

Getting started

In this recipe, the following image will be used:

How to do it...

Let's define a class named MorphoFeatures which will allow us to detect image features:

Using this class in a main function, you then obtain the edge image as follows:

// Create the morphological features instanceMorphoFeatures morpho;morpho.setThreshold(40);// Get the edgescv::Mat edges;edges= morpho.getEdges(image);

The result is the following image:

The detection of corners using morphological corners is a bit more complex since it is not directly implemented in OpenCV. This is a good example of the use of non-square structuring elements. Indeed, it requires the definition of four different structuring elements shaped as a square, diamond, cross, and an X-shape. This is done in the constructor (all of these structuring elements having a fixed 5x5 dimension for simplicity):

// Get the cornerscv::Mat corners;corners= morpho.getCorners(image);// Display the corner on the imagemorpho.drawOnImage(corners,image);cv::namedWindow("Corners on Image");cv::imshow("Corners on Image",image);

The image of detected corners is then, as follows.

How it works...

A good way to help understand the effect of morphological operators on a gray-level image is to consider an image as a topological relief in which gray-levels correspond to elevation (or altitude). Under this perspective, bright regions correspond to mountains, while the darker areas form the valleys of the terrain. Also, since edges correspond to a rapid transition between darker and brighter pixels, these can be pictured as abrupt cliffs. If an erosion operator is applied on such a terrain, the net result will be to replace each pixel by the lowest value in a certain neighborhood, thus reducing its height. As a result, cliffs will be "eroded" as the valleys expand. Dilation has the exact opposite effect, that is, cliffs will gain terrain over the valleys. However, in both cases, the plateaux (that is, area of constant intensity) will remain relatively unchanged.

The above observations lead to a simple way of detecting the edges (or cliffs) of an image. This could be done by computing the difference between the dilated image and the eroded image. Since these two transformed images differ mostly at the edge locations, the image edges will be emphasized by the differentiation. This is exactly what the cv::morphologyEx function is doing when the cv::MORPH_GRADIENT argument is inputted. Obviously, the larger the structuring element is, the thicker the detected edges will be. This edge detection operator is also called the Beucher gradient. Note that similar results could also be obtained by simply subtracting the original image from the dilated one, or the eroded image from the original. The resulting edges would simply be thinner.

Corner detection is a bit more complex since it uses four different structuring elements. This operator is not implemented in OpenCV but we present it here to demonstrate how structuring elements of various shapes can be defined and combined. The idea is to close the image by dilating and eroding it with two different structuring elements. These elements are chosen such that they leave straight edges unchanged, but because of their respective effect, edges at corner points will be affected. Let's use the simple following image made of a single white square to better understand the effect of this asymmetrical closing operation:

The first square is the original image. When dilated with a cross-shaped structuring element, the square edges are expanded, except at the corner points where the cross shape does not hit the square. This is the result illustrated by the middle square. This dilated image is then eroded by a structuring element that, this time, has a diamond shape. This erosion brings back most edges at their original position, but pushes the corners even further since they were not dilated. The left square is then obtained, which, as it can be seen, has lost its sharp corners. The same procedure is repeated with an X-shaped and a square-shaped structuring element. These two elements are the rotated version of the previous ones and will consequently capture the corners at a 45-degree orientation. Finally, differencing the two results will extract the corner features.

Summary

This article explored the concept of mathematical morphology. We took a look at eroding and dilating images, opening and closing images, and detecting edges and corners using morphological filters.

Alerts & Offers

Series & Level

We understand your time is important. Uniquely amongst the major publishers, we seek to develop and publish the broadest range of learning and information products on each technology. Every Packt product delivers a specific learning pathway, broadly defined by the Series type. This structured approach enables you to select the pathway which best suits your knowledge level, learning style and task objectives.

Learning

As a new user, these step-by-step tutorial guides will give you all the practical skills necessary to become competent and efficient.

Beginner's Guide

Friendly, informal tutorials that provide a practical introduction using examples, activities, and challenges.

Essentials

Fast paced, concentrated introductions showing the quickest way to put the tool to work in the real world.

Cookbook

A collection of practical self-contained recipes that all users of the technology will find useful for building more powerful and reliable systems.

Blueprints

Guides you through the most common types of project you'll encounter, giving you end-to-end guidance on how to build your specific solution quickly and reliably.

Mastering

Take your skills to the next level with advanced tutorials that will give you confidence to master the tool's most powerful features.

Starting

Accessible to readers adopting the topic, these titles get you into the tool or technology so that you can become an effective user.

Progressing

Building on core skills you already have, these titles share solutions and expertise so you become a highly productive power user.