Unofficial news and tips about Google

December 2, 2008

Find Similar Images Using Live Search

Microsoft's image search engine added another feature that uses image analysis: for each result, you can find similar images. The related images have nothing to do with are connected to the original query, so Live Search restricts the results to images similar to your selection.

"With Live Search, you can now use images, rather than additional keyword queries, to refine a search and discover more content," explains Live Search's blog.

The next obvious step would be to upload an image and find other similar images on the web. TinEye finds different versions of an image, but the scope of the results should be more encompassing.

When will Chrome be doing the same stuff as IE and Firefox - i love Chrome but it frustrates me that i have to change browser, sometime to use Google's own stuff :( like the google earth plug-in.Thanks for the tip anyway Alex.

Apparently Microsoft doesn't use image features alone for similarity, but also the textual context of images. Try [gibson les paul custom], then use any of the guitar images to search for similar images, and you will see Gibson guitars only. Same for [fender stratocaster], similar images shows Fender Stratocasters only, but there are many other guitar makes that look very similar to a Fender Strat.

One image feature they seem to use is color. Search for [fender jazz bass], then use the one with the red background to search for similar images, and all the images will have some major portion in red. Interestingly, there are red bass guitars on a non red background also.

I would have to rate this as lame. It just doesn't as advertised. To me, it appears to take your original search query (example: toys) and then primarily select a subset of images from that category that are vaguely similar in color, background, etc. and return those images (again). I selected a toy car from the original Toy search and the resulting images were just a random selection of various "toys" from the original Toys query with a few car pics mixed in.

Another example: I searched ‘Eiffel Tower’ and got a lot of tower pics of course. I then selected from those results a pic of a map of Paris (no tower in it) and chose the Similar Images option. Guess what? It returns pics of the Eiffel Tower again…wow, not really similar to the map pic!

For the idea of the 'next obvious step' to upload a pic and find similar images – it may seem obvious, but I don't see it happening anytime soon, at least not with the tech at this stage. I think the results would useless at this point. Since this functionality appears to prioritize using the initial category or search query of images to return the majority of subsequent results, an uploaded pic would have no similar classification or starting point to begin with other than maybe a few queues from the pics characteristics (color, layout) or the file name itself.

Hopefully, Google's Neven Vision tech is still making progress and will provide more intelligent image matching algorithms that actually analyze the picture elements to a greater degree and not just match color patterns or use an existing result subset. I think their face matching tool in Picasa is pretty good, but not great.