Microsoft Introduces New Live Search Index, Adds Features In Effort To Close ‘Relevancy Gap’ And Improve User Experience

Compete and Nielsen show Microsoft gaining search share over the summer, while comScore and Hitwise show the company losing share. Regardless of these numbers, however, Microsoft is confident that the search engine it’s reintroducing tonight in conjunction with its Searchification event will be a substantial improvement of the current version of Live search.

Vanessa Fox was at Searchification and has an excellent post with lots of detail about specifics in the announcements today, including some technical detail. I was briefed ahead of that event, and here I’ll provide an overview, focus on selected specifics, and provide some screenshots to illustrate the changes being introduced.

There are two major upgrades happening this evening:

Bigger Index/Coverage Of The Web: The introduction of a new, larger search index and related “core” search improvements. Microsoft says these will make results more relevant and the overall experience more competitive with Google

Expansion Of Answers/UI Changes: The second part has to do with expansion of Microsoft Answers, user interface tweaks, query refinement tools, and filters. Answers (see screens below) is very similar to Yahoo Shortcuts in particular, but also to Google’s OneBox results that still show despite Universal Search or Ask’s Smart Answers.

Tied in to the expansion of Answers is a vertical initiative that touches on four content areas: Local/Maps, Entertainment (including video) Shopping, and Health. The company feels these are areas of high consumer interest and that it has strategic assets that will help differentiate its offerings. In some cases there are new “products,” while in other areas there are upgrades and additional enhancements to existing products. (Local is such an area, with new “one-click directions” and the elimination of the two box search field in favor of a single box. Vanessa goes into detail here as well.)

Microsoft told me that Live Search has “70 million users in the U.S.,” representing 38 percent of all search engine users, who use Live Search at least once a month. The challenge is gaining more frequency and engagement. This is very similar to what Ask said at the time of its “3D” launch — that they were seeking to boost frequency among existing users as their primary objective.

Leading up to the relaunch of Live Search today, Microsoft solicited and received feedback from its users as well as searchers generally about satisfaction levels with Live Search, as well as search engines across the board. What the company discovered was that its users were dissatisfied with Live Search coverage (breadth and depth of content/results) and relevance. However, Microsoft said search engine users overall were generally dissatisfied with the ability of search engines to discern query intent and help users refine their searches. Accordingly, Microsoft is introducing dynamic filters and query refinement tools to aid in that process — as well as using various techniques to better determine user intent. (Other engines are using similar approaches to address this cluster of challenges.)

Armed with data about its users’ satisfaction and frustration levels, the Microsoft search team focused behind the scenes on improving core relevance and coverage. The new search index is four times larger than the old index it replaces. In addition, new data centers have been added and the system has been built to scale dramatically as content volume and types (think video) grow.

Microsoft is using new ranking algorithms that take user behavior and click stream analysis, among other factors, into account. And I was told that in blind testing with users the company had now “closed the relevancy gap” with Google. Users broadly will, of course, be the judge of the validity of this contention.

Microsoft Answers is intended to condition users to look to the top of the page for rich information, but also to point to more specific vertical experiences where users can get more depth if they’re interested.

Within the four “strategic verticals,” there have been a number of specific improvements based on user feedback. As stated above, in Maps a “two box” search field has been replaced by a simpler single box. In addition, Microsoft has introduced alternate routing based on real-time traffic information when traffic is bad. The system calculates the best route based on available traffic data. User generated content (“collections”) on Live Search Maps has also been expanded. Microsoft actually was the first of the major mapping providers to enable simple mashups for non-developers. However, it didn’t really push and promote that capability initially and Google stole the thunder there with My Maps.

In shopping, Microsoft has created various dynamic filters based on metadata and product attributes. The company has also crawled user reviews and brought that information into search results. In the area of health, Microsoft, building on its Medstory acquisition, has partnered with authoritative sources to provide trusted data in search results.

Here are some screenshots for different categories of queries that illustrate the changes users will start to see:

Local/Maps:

Shopping (category level):

At the level of individual products, much more content (user reviews, metadata filters, etc.) have been added to results:

Entertainment/celebrity search:

Video search (which includes a new preview feature by simply mousing over the thumbnail):

Microsoft says it’s going to do new releases and upgrades monthly. And though I haven’t yet had an opportunity to try out most of the things being described in this post, I believe, from everything I have heard and seen, that Microsoft has achieved its goal of creating a much-improved search experience vs. what it previously offered.

I would imagine the company will see gains in frequency and engagement among existing users. Ask, since the introduction of “3D,” has apparently achieved its goal of increasing usage frequency among its target audiences, but the third party data doesn’t show any meaningful change in market share (however, the ACSI data show big gains for Ask).

Stepping back, what’s interesting is that all the major search engines are doing variations of the same thing: building more content and structure into search results. They’re no longer simply presenting lists of links. Instead, they’re building out a much more visually and content rich presentation on the first page of search results. They’re also adding tools and functionality to help users get to desired information much more quickly with fewer clicks back and forth between sites.

Regarding search market share, if anyone is going to have more than fractional gains it’s going to come over time. But Microsoft is one of the few companies that has the resources — and the apparent commitment — to really challenge Google in search.

Users are the “bottom line,” however, and they will judge whether Microsoft has indeed closed the relevance gap and whether these new features and improvements are compelling.

For related coverage, see the roundup growing at Techmeme. Also see Mary Jo Foley’s summary, where she notes Microsoft is still pondering a change away from the Live Search brand. Also see Microsoft’s official blog post here: Introducing…the new Live Search.

SMX Advanced is the only conference designed exclusively for experienced paid search advertisers and SEOs. You'll participate in experts-only sessions and network with fellow internet marketing thought leaders. Check out the tactic-packed agenda!

About The Author

Greg Sterling is a Contributing Editor at Search Engine Land. He writes a personal blog, Screenwerk, about connecting the dots between digital media and real-world consumer behavior. He is also VP of Strategy and Insights for the Local Search Association. Follow him on Twitter or find him at Google+.