Super search engine could come from HP/Autonomy deal

Hewlett-Packard and Autonomy, the British pattern-recognition software house it acquired for a cool $10.2 billion just a couple of weeks ago, are moving quickly to create a phone and computer-based search system that merges the technologies of both companies, Autonomy's founder Mike Lynch told New Scientist this week.

The mega-buyout has had mixed reactions - with some complaining it's a severe loss to the UK technology base and others saying HP is clutching at straws - but at a computer science level, Lynch believes the move will finally make messy, unstructured data searchable in mainstream applications on smartphones, computers or online via cloud computing services.

By the end of the year, he hopes HP's Vertica text-search database, which allows for detailed searching of rigidly structured text like emails and documents, will be operating in tandem with Autonomy's Idol system, which pinpoints relationships between informal, unstructured text, audio, images and video.

"Many people think of information as black and white, true or false, and that you can write a rule and put it in a database so you can equate things and query them," says Lynch.

But the real world is not that simple: if you wanted to search for voicemails and video clips related to a document, you'd be lost. But Idol extracts the probability that human behaviours caught on video, say, are related to the words uttered in a voicemail and/or a report - making it searchable.

"It will seamlessly understand phone calls and voice messages, for instance. So you could ask the system to find you a voicemail in which you told someone what time you'd meet them," says Lynch. Or simply pointing a phone at an image could bring up highly-likely-to-be-related recordings or web links.

"It will also help scientists take very large datasets and get meaning out of them," says Lynch.