Tuesday, May 22, 2007

Universal search, Google, and A9

David Bailey (ex-Amazon, now at Google) and Johanna Wright discuss Google's new universal search effort -- bringing image, video, news, and other vertical search results into the web search results -- on the Official Google Blog.

Gary Price makes the point that others have walked down this road before, at least a bit down it, but Google's universal search does sound to me like a broader effort, an attempt to surface data from verticals on many more queries and perhaps even eliminate the need to go separately to the verticals.

From David and Johanna's post:

Finding the best answer across multiple content types is a well-known hard problem in the search field.

Until now, we've only been able to show news, books, local and other such results at the top of the page ... [but] often we end up not showing these kinds of results even when they might be useful.

If only we could smartly place such results elsewhere on the page when they don't quite deserve the top, we could share the benefits of ... [results from verticals] much more often.

Although it's just a beginning ... now you'll be able to get more information Google knows about directly from within the search results. You won't have to know about specialized areas of content.

One thing I find interesting about this is comparing this universal search effort at Google with the federated search done by A9.

A9 allowed searching many databases, but searchers had to manually select which data sources to use, and results were not merged. A9 punted on the hard problems with federated search, query routing and relevance rank of the merged results.

Now, Google appears to be taking some of these problems on, trying to determine which of their verticals should be searched given likely user intent with a query and trying to determine which results are mostly likely to be relevant from the disparate data sources.

Of course, Google's universal search is not federated search. Google prefers their own copy of data and is unlikely to hit external sources. But, the problems of query routing and relevance rank of merged results share much in common with the problems that need to be solved for Google's universal search.

It is curious that Udi Manber, former CEO of A9, is now leading Google core search. Perhaps he is finishing the work at Google that he never finished at A9.

2 comments:

At XTech last week I demonstrated the federated search we've built in the Talis Platform (you can see a video of the talk here: http://cubicgarden.blip.tv/file/234421/). It's a bit like opensearch, but instead of sending the search (and the complexity involved in mapping different search syntaxes) we send the URI of the original set of search results (an RSS feed). The receiving service then annotates the results with additional information and returns the entire result set. This can in turn be fed to another service for further annotation.

The example I use is a search engine that searches book data. I send the URI of a book search to another search engine that contains book jackets which fetches the book search, then matches ISBNs to get jacket images for the books in the search. I then pass this to a search engine that has knowledge of which libraries hold which books to annotate the search results with location information. Ditto for wikipedia articles about the author, or reviews or ratings etc. It's federated and (because we're using RSS 1.0 RDF) trivially parallelisable with automatic merging of results.