FAST

I believe there are two ways search will improve significantly in the future. First, since talking is easier than typing, speech recognition will allow longer and more accurate input strings. Second, search will be informed by much more persistent user information, with search companies having very detailed understanding of searchers. Based on that, I expect:

A small oligopoly dominating the conjoined businesses of mobile device software and search. The companies most obviously positioned for membership are Google and Apple.

The continued and growing combination of search, advertisement/recommendation, and alerting. The same user-specific data will be needed for all three.

Enterprise search is greatly disappointing.My main reason for saying that is anecdotal evidence — I don’t notice users being much happier with search than they were 15 years ago. But business results are suggestive too:

Web search, while superior to the enterprise kind, is disappointing people as well. Are Google’s results any better than they were 8 years ago? Google’s ongoing hardwork notwithstanding, are they even as good?

Consumer computer usage is swinging toward mobile devices. I hope I don’t have to convince you about that one. 🙂

In principle, there are two main ways to make search better:

Understand more about the documents being searched over. But Google’s travails, combined with the rather dismal history of enterprise search, suggest we’re well into the diminishing-returns part of that project.

Understand more about what the searcher wants.

The latter, I think, is where significant future improvement will be found.

Lynda Moulton, to put it mildly, disagrees with the Gartner Magic Quadrant analysis of enterprise search. Her preferred approach is captured in:

Coveo, Exalead, ISYS, Recommind, Vivisimo, and X1 are a few of a select group that are marking a mark in their respective niches, as products ready for action with a short implementation cycle (weeks or months not years).

By way of contrast, Lynda opines:

Autonomy and Endeca continue to bring value to very large projects in large companies but are not plug-and-play solutions, by any means. Oracle, IBM, and Microsoft offer search solutions of a very different type with a heavy vendor or third-party service requirement. Google Search Appliance has a much larger installed base than any of these but needs serious tuning and customization to make it suitable to enterprise needs.

Attivio CEO Ali Riaz was previously CFO and COO of FAST. He tried to avoid involvement in the recent expose’ of his former employer. For his troubles he got a parking lot ambush, a big photograph, and some unflattering coverage. Read more

A Norwegian newspaper did an expose’ on FAST, dated June 28. Helpful search industry participants quickly distributed English translations to a variety of commentators, including me. TechCrunch posted a scan of part of the article.

The gist is that FAST followed a pattern very common in the packaged enterprise software industry: Read more

The old FAST search-as-BI dream might become pretty appealing to Microsoft/Yahoo.

In a non-search point, Microsoft is strong in games and Yahoo is strong in fantasy sports. Look for some synergies.

There sure would be a whole lot of non-Windows technology inside Microsoft. 🙂

Basically, Microsoft is a company that’s a lot more sophisticated in its thinking about user interfaces and experiences than Yahoo is. That’s where the really interesting competitive innovation would be most likely to occur.

Following up on my priorposts about Microsoft’s impending acquisition of FAST, they’ve now had the conference call. By custom and indeed antitrust law, such calls are very light on content. But here are a few tidbits and takeaways, all from Jeff Raikes of Microsoft:

Jeff talked solely about FAST as adding to enterprise search, and rightly contrasted that with web search.

However, he deflected questions about web search with “We aren’t talking about that much detail right now” rather than with a firm “Well, we aren’t allowed to use FAST that way.”

Specifically, enterprise search is all about integration with SharePoint (portal).

Jeff said Microsoft’s current search could handle millions or maybe tens of millions of documents, but thought there was demand for FAST’s ability to handle billions.

He positioned FAST as an application development platform, giving an example of structured search (the actual word was “pivot”) in consumer electronics. … Well, at least he’s looking in the right direction.

Microsoft has certainly had a number of false starts in search. At the 1997 Verity user conference, a Microsoft employee told me of his confidence Microsoft would surpass Verity in enterprise search the next year. Yeah, right.

In September, 2003, a nice woman wrote me to tell me she had joined Microsoft and would personally write the ranking engine for MSN search. That worked out great too.

Now Microsoft has a multi-faceted enterprise search strategy. Guy Creese seems mightilyimpressed. Should we, for once, be impressed too?

Frankly, yes. So far as I can tell, most traditional text search products have atrophied, including Verity before it was bought by Autonomy. And I’m skeptical about Autonomy’s Bayesian-everything approach. Oracle and Google, in different ways, consistently fail to round out their products. So if FAST’s technology can ever be fleshed out and stabilized, it indeed could be a market leader or even dominator. Read more

As you’ve probably heard by now, Microsoft is buying enterprise search vendor FAST (Fast Search & Transfer). FAST wasn’t always focused on enterprise search; in fact, FAST built alltheweb.com. And when FAST sold alltheweb.com to Inktomi, it agreed not to reenter the web search business itself. Inktomi was subsequently bought by Yahoo, a company not much inclined to do Microsoft any favors in the web search arena.

FAST is annoying me a bit these days. It’s nothing serious, but travel schedule screw-up’s, an annoying embargo, and a screw-up in the annoying embargo have all hit at once. So I’ll keep this telegraphic and move on to other subjects.

They’re doing fast queries without using a lot of RAM.

They’re doing the usual text search thing of indexing across multiple “databases,” only now it’s applied to, well, databases. (Not that there’s much new about that particular aspect. Actually, there seems to be a bit of kludge in that they export the databases to some kind of simple text files.)

They’re doing some level of concept identification ala the text mining guys. (They don’t call it “entity extraction” because the results aren’t dumped into a database anywhere, but instead are just used on the fly.) Of course, the text mining/search convergence goes both ways.

They bought a BI/dashboard tool and are using it both to analyze query logs and also to do normal BI/dashboard kinds of things.

They have big references for this stuff, at least the single-web-site query aspect. Well, actually, the customer names are confidential. Oh well.