How to use an Unsharp Mask to Improve the Accuracy of Saliency Detection

Stephanie Kim

2 years ago

In our introduction to saliency detection post, we showed how to harness the power of the human brain using a saliency algorithm to detect the most distinct and noticeable objects in an image.

This algorithm can be used in a variety of fields, such as user experience testing, neuroscience research, and even product placement in advertisements or commercials.

SalNet is a saliency detection microservice hosted on Algorithmia, which allows you to easily integrate saliency detection into your product via a serverless API.

Most images do very well using SalNet, however some older, fuzzier images might need a little help from another algorithm called Sharpen Image. This algorithm uses an unsharp mask to sharpen the edges in the image, which increases the ability of the saliency detection algorithm to detect the most relevant shapes in an image.

In this recipe we’ll go through how to use the Sharpen Image algorithm to clean the image before running it through Sal Net, the saliency detection algorithm. This way you can pass in less than perfect images through the saliency algorithm and get great results.

Step 1: Install the Algorithmia Client

This tutorial is in Python. But, it could be built using any of the supported clients, like Scala, Ruby, Java, Node and others. Here’s the Python client guide for more information on using the Algorithmia API.

Install the Algorithmia client from PyPi:

pip install algorithmia

You’ll also need a free Algorithmia account, which includes 5,000 free credits a month – more than enough to get started with saliency detection.

Step 2: Sharpen Your Images

Now that we have our account credentials, we can call the first algorithm that will sharpen our image in order to prepare the photo for the saliency algorithm. In this example we are showing how to pass in an image that is located in a data collection on Algorithmia. You could also use S3 or Dropbox.

In the above code notice there is a parameter called path_dict that when we call our function, we’ll pass a dictionary in as our argument. This dictionary has two keys, the “image” key and the “location” key. The “image” key’s value will hold our file path to our original photo and “location” will be the path that our sharpened image will go into.

The “image” path should be the location of your original image in your data collection. To get this path, go to the Hosted Data page (requires sign in). In the collection your original photo is in, you’ll find its path under the filename: data://:username/:collection/:filename

You’ll want to set the “location” path to somewhere other than the temporary folder (data://.algo/temp/), because our next algorithm will need access to that image.

For details about permissions and special directories such as the .algo directory, see the Hosted Data Guide.

Even though we’ll wait to run it until the whole script is finished, here is a sneak peak of what we’ve done so far:

Original

After Sharpen Image Algorithm

Step 3: Apply Saliency to Find Notable Objects

Now that we have our sharpened image, let’s put them through the saliency algorithm:

The above code calls the Sal Net algorithm, passing in the dictionary that holds our input and output for our first algorithm Sharpen Image. The Sal Net algorithm’s output will be held in the algorithm’s temporary folder (.algo/deeplearning/SalNet) in your Hosted Data.

After running the algorithm, you’ll print out the paths for the Sharpen Image file, the Sal Net image path and the matrix of the saliency results in a JSON file: