Filtering web images effectively

Jun 25, 2012 By Rob Knies

Youre looking for a photo of a flower. Not just any photoit needs to be horizontal in shape. And not just any flowerit needs to be a purple flower.

What do you do? You could perform a conventional image search on the web. There are lots of flowers out therelots of shapes, lots of colors. Poke around for a while, and you just might find what you need.

Alternatively, you can use the filter bar in Bing Image Search, which has been augmented by work from Microsoft Research Asia. You type in a textual query: flower and filter for purple, photograph, and wide, and voilà, a collection of horizontal shots of purple flowers pops up.

The color filter is thanks, in large part, to research by Jingdong Wang and Shipeng Li. They are in Providence, R.I., from June 16 to 21, attending the Institute of Electrical and Electronics Engineers 2012 Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2012), during which they are presenting their paper Salient Object Detection for Searched Web Images via Global Saliency, written in collaboration with Peng Wang, Gang Zeng, Jie Feng, and Hongbin Zha of the Key Laboratory on Machine Perception at Peking University.

That paper is part of a substantial conference contribution from Microsoft Research, which had no fewer than 41 peer-reviewed papers accepted for CVPR 2012. Andrew Blake, Microsoft distinguished scientist and managing director of Microsoft Research Cambridge, is program chair for the event, and Jian Sun of Microsoft Research Asia and Richard Szeliski of Microsoft Research Redmond are serving as area chairs.

A salient object is the primary component in an image. Imagine a photo of a horse in a field under a blue sky. The grass might be green, and the sky might be gorgeous, but in all likelihood, the salient object in the shot is the horse.

Our goal is to develop an effective and efficient technique to locate the salient object, Wang explains. Particularly, we also predict if an image contains a salient object, which has been rarely studied before.

The detection of salient objects in images has beenfor a long time. Such an ability has broad applicationsimage cropping, adaptive image displays on mobile devices, extracting dominant colors within images, removing images lacking an object of interest. But localizing salient objects remains a challenge.

The problem is that objects have a variety of visual characteristics, making it difficult to differentiate salient objects from an image background simply by appearance. And while low-resolution thumbnail images, which proliferate on the web, are recognizable by humans, previous detection methods make it difficult to get the reliable image segmentation those methods require for success.

The Microsoft Research Asia researchers, though, use a learning approach called a random forest, an algorithm that clusters data points into groups, to predict the existence and the position of a salient object in an image.

The key, Wang explains, is to describe an image using a global saliency description and to conduct a classification stage, to check the existence of the salient object, and a regression stage, to check the location.

If a salient object can be identified, using the researchers techniques, its dominant color is extracted. If a salient object cant be identifiedwhich could be possible, for instance, in a landscape shotthe dominant color of the entire image is extracted.

To validate their results, Li and Wang constructed a large image database consisting of hundreds of thousands of manually labeled web images from Bing image search and deployed their algorithm to identify the existence, predict the location, and identify the color of the salient object in thumbnail images.

We are pleased to see that the results justified our hypothesis that salient objects essentially share common patterns, Wang says, even though the objects are generally different. That motivated us to investigate machine-learning tools to solve the problems.

Related Stories

Creating a high-quality realistic color image from a grayscale picture can be challenging. Conventional methods typically require the users input, either by using a scribbling tool to color the image ...

Photographs of moving objects are almost always a little blurry  or a lot blurry, if the objects are moving rapidly enough. To make their work look as much like conventional film as possible, game and ...

Youre interested in purchasing a car youve seen on the web. Its the right make, model and vintage. It seems to be in great shape, and its just the right color. The price seems reasonable. So whats ...

Like many kids, Antonio Torralba began playing around with computers when he was 13 years old. Unlike many of his friends, though, he was not playing video games, but writing his own artificial intelligence ...

Recommended for you

Using complex computational algorithms based on repeated sampling, a University of Arkansas graduate student helped spearhead a project to develop a computer simulator that provides customized electron beam ...

A team of researchers at Google has been looking into ways to change the way links are retrieved by its famous search engine—instead of ranking them based on popularity, the researchers are looking into ...

Visitors to interactive virtual worlds want the ability to significantly affect the outcome of a story, but authoring these digital experiences is extremely complex. A new platform developed by Disney Research ...

If you live in a city, you know that a fair amount of your movement around town is social in nature. But how much, exactly? A new study co-authored by MIT researchers uses a novel method to infer that around ...

Social networks such as Facebook, LinkedIn and Twitter play an increasingly central role in our lives. Centrality is also an important concept in the theory of social networks. Centrality of an individual, called a "node" ...

User comments : 0

Please sign in to add a comment.
Registration is free, and takes less than a minute.
Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.

Javascript is currently disabled in your web browser. For full site functionality, it is necessary to enable Javascript.
In order to enable it, please see these instructions.