Search at the Speed of Thought

A guiding principle in information technology has been to enable people to perform tasks at the “speed of thought”. The goal is not just to make people more efficient in our use of technology, but to remove the delays and distractions that make us focus on the technology rather than the tasks themselves.

For example, the principle motivation for the faceted search work I did at Endeca was to eliminate hurdles that discourage people from exploring information spaces. Most sites already offered user the ability to perform this exploration through advanced or parametric search interfaces–indeed, I recall some critics of faceted search objecting that it was nothing new. But there’s a reason that most of today’s consumer-facing sites place faceted search front and center while still relegating advanced search interfaces to an obscure page for power users. Faceted search offers users the fluidity and instant feedback that makes exploration natural for users. Once you’re used to it, it’s hard to live without it, whether your looking for real estate (compare Zillow.com to housing search on craigslist), library books (compare the Triangle Research Libraries Network to the Library of Congress), or art (compare to art.com to artnet).

Why is faceted search such a significant improvement over advanced or parametric search interfaces? Because it supports exploration at the speed of thought. If it takes you several seconds–rather than a single click–to refine a query, and if you have to repeatedly back off from pages with no results (aka dead ends), your motivation to explore a document collection fades quickly. But when that experience is fluid, you explore without even thinking about it. That is the promise (admittedly not always fulfilled) of faceted search.

Microsoft Live Labs director Gary Flake offered a similar message in his SIGIR 2010 keynote. He argued that we needed to replace our current discrete interactions with search engines into a mode of continuous, fluid interaction where the whole of data is greater than sum or parts. While he offered Microsoft’s Pivot client as an example of this vision, he could also have invoked the title of a book that Bill Gates wrote in 1999: Business @ the Speed of Thought. Indeed, anyone who has ever worked on data analysis understands that you ask fewer questions when you know you’ll have to wait for answers. Speed changes the way you interact with information.

We know your time is valuable, so when you’re seeking an answer on the web you want it right away – and we aim to please. We may be the only people in the world who can say our goal is to have people leave our website as quickly as possible. By shaving excess bits and bytes from our pages and increasing the efficiency of our serving environment, we’ve broken our own speed records many times over, so that the average response time on a search result is a fraction of a second. We keep speed in mind with each new product we release, whether it’s a mobile application or Google Chrome, a browser designed to be fast enough for the modern web. And we continue to work on making it all go even faster.

People have made much of Google VP Marissa Mayer’s estimate that Google Instant will save 350 million hours of users’ time per year by shaving two to five seconds per search. That’s an impressive number, but I personally think it understates the impact of this interface change. Rather, I’m inclined to focus on a phrase I’ve seen repeatedly associated with Google Instant: “search at the speed of thought”.

What does that mean in practice? I see two major wins from Google Instant:

1) Typing speed and spelling accuracy don’t get in the way. For example, by the time you’ve typed [m n], you see results for M. Night Shyamalan, a name whose length and spelling might frustrate even his fans. A search for [marc z] offers results for Facebook CEO Mark Zuckerberg. Admittedly, the pre-Instant type-ahead suggestions already got us most of the way there, but the feedback of actual results offers not just guidace but certainty.

2) Users spend less–and hopefully no time–in a limbo where they don’t know if the system has understood the information-seeking intent they have expressed as a query. For example, if I’m interested in learning more about the Bob Dylan song “Forever Young“, I might enter [forever young] as a search query–indeed, the suggestion shows up as soon as I’ve typed in “fore”. But a glance at the first few instant results for [forever young] makes it clear that there are lots of songs by this title (including those by Rod Stewart and Alphaville–as well as the recent Jay Z song “Young Forever” that reworks the latter). Realizing that my query is ambiguous, I type the single letter “d” and instantly see results for the Dylan song. Yes, I could have backed out from an unsuccessful query and then tried again, but instant feedback means far less frustration.

Google Instant also makes it a little easier for users to explore the space of queries related to their information need, but exploration through instant suggestions is very limited compared to using related searches or the wonder wheel–let alone what we might be able to do with faceted web search. I’d love to see this sort of exploration become more fluid, but I recognize the imperative to maintain the simplicity of the search box. Good for us HCIR folks to know that there’s still lots of work to do on search interface innovation!

But, in short, speed matters. Instant communication has transformed the way we interact with one another–both personally and professionally. Instant search is more subtle, but I think it will transform the way we interact with information on the web. I am very proud of my colleagues’ collective effort to make it possible.

74 responses so far ↓

Google Instant is not really search at the speed of thought, as reading Mark Zuckerberg in a snippet text is not yet a valid search / information seeking outcome (at least if you are not only interested in spelling).

I think you’ve hit the nail on the head with the concept of speed as a way of helping users focus on their task rather than the technology. There have been criticisms from some users that Google’s instant results can be a bit disconcerting, but I’m hopeful that this is only initial resistance that will fade as people get used to (and come to expect) this instant behaviour. I’m particularly interested to see if/when/how instant search will start showing up in ecommerce and enterprise search settings.

Lars, my apologies for skipping steps. I should have elaborated on the information need that motivates [marc z] as a query, e.g., I just heard about some big budget movie and want to learn more about the guy. My point is that I am guided to the information I need even if I don’t know / am unsure of how to express it precisely — and that the feedback of instant results couples this guidance with confirmation.

Tyler, I’ve read some of those criticisms. I’ve also read about user studies that suggesting that most users are not distracted by intermediate results. I’m glad that those who are can easily turn off the feature. But like you I’m hopeful that this is simply initial resistance to change.

And yes, it will be interesting to see where else this functionality shows up. I can certainly see promise in ecommerce, especially for use cases like model or part number search. But what I’d really like to see is an effective integration with faceted search.

Speed is certainly important. I’m sure you are old enough Daniel to remember having to go to the library and look for articles related to your thesis topic? You’d got there, find that the article you spent 5 minutes (or more!) tracking down is irrelevant… and then you would be tempted to browse nearby articles in the hope that one of them was relevant.

I was amazed last Spring to discover that many professors still coach their students to do exactly this. Students, apparently, still go in the library to find and copy research papers.

Obviously, this is intensely distracting. How can you keep a focus in such conditions?

Of course, the more senior people didn’t have to do this as they would collect research papers in their office… I still had piles of research papers some weeks ago… my wife just threw them out.

I don’t think that this instant access to documents makes me a much better researcher. Indeed, when it comes down to it, while you are writing a paper, you probably need 2-3 references at the most…

But what it allowed me to jump into research topics very quickly. So the entrenched researcher has a less of an edge from a documentation point of view.

Interesting analysis, but I wonder what % of traffic is straight to Google.com vs., say, the web browser address or search bars? Or from another site? I usually get to Google search results that way and have seen Google Instant maybe twice since it launched. Maybe if they get the suggestions working in the Chrome address bar…?

This is another awesomely cool engineering feat of crazy-fast caching and networking coupled to really good first-best guesses.

Given that you’re showing search results for best-guess auto-completions, I found it a bit confusing when I first completed a query (in my mind) that didn’t match the best-guess auto-completion.

For instance, after I completed typing my search [MC sample] (a notion from statistics), Google Instant’s top result is a screenshot from the YouTube video “I like small butts” by MC Sampler, whereas the results I get from hitting return are what I would’ve expected.

I only then noticed the light-grey auto-complete text in the search box with that little extra “r” at the end (hey, I’m on an old LCD monitor with sketchy gamma settings for images).

Interestingly, the behavior changes when you get into lower frequency queries. At that point, it slows to a (relative) crawl and gives you results that you’d get from querying from where you’re at (at least as far as I can tell).

I don’t see how to improve the behavior (not that I’d expect to given the engineering put into Google search!). The design makes sense in terms of both caching demands and in the sense that partial queries with decent auto-completions are more often than not meaningless.

Daniel, I’ll admit my age–I remember using card catalogs and asking authors to snail-mail me papers. I do think that instant access to documents makes me more productive, but I realize that more efficient access to information only gets us so far. There’s no substitute for hard work, and perhaps the abundance of readily available information can even be a distraction. Still, I can’t see any of us giving it up.

Rob, one step at a time!

Bob, I think the case you bring up is an interesting one–where the query doesn’t need auto-completion and indeed that auto-completion could cause confusion. That said, I’m not sure how well the results for [MC sample] satisfy your information need either. But yes, there are definitely legitimate tail queries that are prefixes to head queries. As for the behavior for long queries deep in the tail, I still find the “relative crawl” quite responsive–and I like the experience better than hitting enter and waiting for the results. Certainly it’s no worse than having to hit enter and wait for the results.

My only comment is that — as amazing an engineering feat that it was to get something like this working — it seems to primarily serve to enhance (aka entrench) the “known-item search” paradigm.

Sure, you can get instant feedback on whether the known item that you were looking for is easily findable using the query that you chose. But is that by itself an enabler of exploratoriality? Your ability to obtain knowledge that the known item you were looking for isn’t findable using your query happens faster, and I can see the argument that this allows you to ask more questions and cover more ground in the same amount of time.

But the fundamental interaction between user and search engine isn’t changed. Quantitative difference, not qualitative. Non?

I agree that it primarily helps with elaborating a known-item search intent, and that there’s much more to search than known-item search. But for web search I think this is a big deal. In particular, it means that fewer known-item searches break down into exploratory searches simply because the user needs help elaborating the query.

But for exploratory search intents, this at best scratches the surface–as I noted, it doesn’t go as far as Google’s related searches or wonder wheel features, and even those hardly go far enough.

In particular, it means that fewer known-item searches break down into exploratory searches simply because the user needs help elaborating the query.

I can believe that. And/but you think this new interface allows this to happen moreso than is already happening with Google Suggest? Not that it’s happening quicker, i.e. total elapsed time. But that Google Instant allows it to happen with fewer iterations, fewer refinements. Because the latter would be interesting. Do you have data, even informally, that you can talk about? Does the number of iterations go down? If so, by how much, on per-user average? 0.005 iterations? 0.3 iterations? 1.0? 2.9? Something that the user would significantly notice?

it doesn’t go as far as Google’s related searches or wonder wheel features, and even those hardly go far enough.

My take on this is that for cognitively-demanding searches (exploratory, recall-oriented, etc.), reducing the speed of response by 300 milliseconds is irrelevant because people spend most of their time thinking about the problem and about the documents they found, not waiting for the search engine to retrieve the documents, which was plenty fast to begin with.

I concede that faster response doesn’t help much with exploratory and recall-oriented searches. But I think it does help ensure that precision-oriented searches don’t become cognitively demanding just because the user isn’t sure how to express the query.

Dinesh, the observation that queries have been getting longer over the years implies that users see a utility in doing that, a utility that arises from more appropriate search results.

Google already has a query suggestion feature that can reduce the among of typing and can help by suggesting useful expansion terms. All Instant does is evaluate the first such suggestion immediately. This is useful only if the top-ranked suggestion is the correct one, but may drive people to accept it as the correct one even if it isn’t. Thus Instant may decrease the diversity of search results, pointing users only to the most common interpretation of their information need. Is that a good thing? Perhaps for some people, some of the time, yes. In general, no.

Gene: I disagree. In fact, if the top-ranked suggestion is the wrong one, there’s a chance that the use will realize that from the results but not from the suggestion. In that case, Instant helps in a way that Suggest cannot. It only reduces diversity to the extent that users themselves satisfice.

Gene: I understand that users don’t always know what’s best for them, but at the same time I’m inclined to let them make those choices. And for precision-oriented needs, satisficing is probably an optimal use of their time and effort.

My feeling is that I’m also not against users making their own choices. But it’s kinda like farm subsidies and the U.S. food system. Bear with me for a moment. Because of subsidies, too much corn gets grown in this country. And because of an overabundance of cheap corn, food manufacturers put in in everything, including ketchup. And then the consumer goes to the store and makes their own choices about what to buy.. and ends up buying/supporting the corn supply chain.

So is the consumer really choosing the corn? Well, yeah, kinda. But.. not really. It’s because corn is pretty much the only option that they have, that it’s in everything, that they keep picking it.

The analogy I’m trying to draw is that Google massively subsidizes satisficable, known-item searches with its adwords program. Consumers don’t pay for it directly.. Google subsidizes it, through massive engineering efforts such as the one behind Google Instant.

Then, when the consumer ends up satisficingly using Instant, I guess you can say that this has been a choice, and that more and more consumers do it. But (my feeling is) that’s mostly because it’s so readily, easily, cheaply available. Like corn. ‘

Forgive me; I haven’t spend too much time fleshing out this analogy…it’s not a perfect one. But do you see where I’m going with it? It’s basically your “Google is McDonald’s” comparison, taken one step further.

I wonder.. if consumers had to pay the true cost of every search, and it were not heavily subsidized by adwords, would consumer behavior really be the same?

And doesn’t Google have an ethical responsibility to not cornify, McDonaldsify our information supply? The “information corn” that Google produces.. isn’t that kinda.. evil?

But here’s a question: has Google changed searchers’ information diets? Did people lose interest in recall-oriented information needs once they started using Google? Did they lose their curiosity to explore? I’m pretty sure the answer to these questions is no. My belief is that our distribution of information needs hasn’t changed much in the past decade, but that advances in technology enable us to meet a subset of those needs far more efficiently today than a decade ago.

Specifically, I don’t think that improvements in the effectiveness and efficiency of known-item search have come at the expense of exploratory search–any more than the success of McDonald’s comes at the expense of 3-star restaurants. They are different markets.

And I think this is somewhat orthogonal to the ad-supported model. Even if the ad-supported model disappeared and users had to pay for searches (on all search engines), I have no reason to believe they would search differently.

Indeed, I suspect that the unsubsidized cost of satisfying an exploratory search query is, on average, higher than that of satisfying a known-item search. Not sure what that does to your argument.

As I quoted in the post, Google has a core believe that “fast is better than slow”. So, regardless of how you perceive the value of the feature, it shouldn’t be surprising that Google invests in speed. “Fast enough” is subjective. I may be biased, but I don’t believe there’s any ulterior motive here.

But here’s a question: has Google changed searchers’ information diets?

Yes, it did. In the same way that access to cheap corn and sugar changed Americans’ diets, post WWII. To continue in the food analogy and to answer your question: People’s basic dietary needs, in and of themselves, have not changed in hundreds of thousands of years. People still need salt, fiber, protein, fat, simple (instant) sugars, complex carbs, etc. Now, however, there is a glut of simple sugar, because of subsidies and market forces in which the consumer does not have to pay directly for externalities (e.g. the consumer doesn’t pay for/see the cost of pollution that comes from cheap oil, which oil allows the production of so much corn).

As a result, even though our dietary *needs* haven’t changed, our diets themselves have. To go all Michael Pollan on you, we’ve become addicted to fat, sugar, and salt. The reward centers in the brain light up when we get these things, because over the past hundreds of thousands of years they were so difficult to come by. Now that they’re easy (“Instant”), we gorge ourselves on them and that has led to diabetes, overweight, and heart issues. So yes, to continue the imperfect analogy, I think the instant sugarification of information is having a similar effect. To the detriment of balanced, healthy eating.

Did people lose interest in recall-oriented information needs once they started using Google? Did they lose their curiosity to explore? I’m pretty sure the answer to these questions is no.

If by lose you mean 100% gone.. of course not. But in the same way that our average diet has changed, and we’re more likely as a culture to eat a bag of doritos than to eat a fresh carrot, when hungry for a snack… then yes, I think we have lost interested in the more “complex carbohydrates” of an exploratory information need.

My belief is that our distribution of information needs hasn’t changed much in the past decade, but that advances in technology enable us to meet a subset of those needs far more efficiently today than a decade ago.

And I maintain that advances in technology that have indeed allowed us to meet a subset of those needs far more efficiently today than a decade ago have caused a unhealthy shift in our consumption patterns. Again, with the sugar analogy: The brain has always rewarded the consumption of sugar, it’s true. But evolutionarily that was because of scarcity. Now that technology has changed that scarcity balance, our bodies have not yet caught up. Same with information.

Specifically, I don’t think that improvements in the effectiveness and efficiency of known-item search have come at the expense of exploratory search–any more than the success of McDonald’s comes at the expense of 3-star restaurants. They are different markets.

I’m going to resist the return to the McDonald’s analogy for the moment, and stick with the “few steps beyond” raw foodstuff analogy.. the sugars and fats and salts and proteins and fiber and such. And what I argue is that, from that perspective, the market for simple sugars and the market for complex carbs are not two separate markets, like McD’s vs Au Bon Pain. Because it’s not like there is a separate need in our bodies for fat and sugar, vs. for fiber and protein. We need it all. It’s all the same market.

What we have, currently, though, is an imbalance in the supply of those items within the single marketplace. And that imbalance causes diabetes and heart disease.

And I think this is somewhat orthogonal to the ad-supported model. Even if the ad-supported model disappeared and users had to pay for searches (on all search engines), I have no reason to believe they would search differently.

I wish we could test this somehow. I don’t think it’s orthogonal at all. I think it’s directly related to the subsidy of ad-support. Consumers aren’t paying for externalities directly, the same way they’re not paying for oil externalities. But I cannot prove non-orthogonality beyond basic analogy and anecdote. Which is why our discussions are interesting

Indeed, I suspect that the unsubsidized cost of satisfying an exploratory search query is, on average, higher than that of satisfying a known-item search. Not sure what that does to your argument.

What I would say in this case is that the unsubsidized cost of satisfying an “Instant” search query is much higher than satisfying a known-item search, too. Maybe even more expensive than satisfying an exploratory need. So without subsidies, if people had to pay the true cost of getting that “Instant” simple sugar bloodstream injection, they might very well make different choices. Right? In the same way that people made different dietary choices before the (subsidized, cheap) ubiquity of cheap corn.

For every iota of engineering that gets invested in Google instant, that is an time and money that don’t get invested in satisfying more complex needs. And the easier it becomes for people to get access to that instant fix/rush, the more they crave it, and the more out of balance they get. It’s a self-perpetuating cycle, that I see Google Instant worsening. It’s not a neutral thing.

Don’t get me wrong: Google Instant, by itself, is not “bad” any more than sugar itself is bad. Of course its not. What bothers me is the increasing imbalance that it creates.

As I quoted in the post, Google has a core believe that “fast is better than slow”. So, regardless of how you perceive the value of the feature, it shouldn’t be surprising that Google invests in speed.

To me, a lot of what Google does in terms of speed is like trying to go 90 mph between stoplights in a residential zone. Versus going 45 mph on an unobstructed country lane. In the former, you get these rushes of speed, and instantaneous outcomes, and it “feels” fast. But because you’re waiting for 2 minutes at every stoplight, the person going 45 mph on the country lane might actually get their first.

So which one is “faster”?

I feel sometimes like, because the user perceived the former to be faster (the 90 mph residential stoplighted bursts), that’s what Google focuses on. Even though the overall time to satisfy an information need might be faster by going 45 mph and offering search interfaces that are more deliberative and exploratory.

To paraphrase Bob: As Information Retrieval researchers, we should keep in mind perceived speed versus speed.

@ daniel
I don’t want put you on the spot. Anecdotal evidence suggests that people don’t like web surfing or web searching on mobile devices and Apps have made it possible to focus on the task and let the App perform searches in the background. Mobile users want faster response times. They also want minimal text entry ie. touch and go. Hence, I was wondering if one of the prime motivations behind Instant was to satisfy mobile users as they can enter just a few characters and select a query from the autosuggestion list. In this case ‘fast’ is not about returning the results faster but entering the query faster on a small factor device not designed for long text entry.

Back to the diet metaphor, I’d like to see a concrete example where people used to explore a domain but only now are satisfied to use the first sufficiently relevant result they find. I actually suspect that making individual precision-oriented queries faster increases exploration for those who have any interest in it, since it lowers the cost.

As for the prioritization of engineering investment, I believe it is user-driven, rather than vice versa. The tail isn’t wagging the dog. Perhaps users would take more of an interest in exploratory search if the tools were better. But I don’t think that investment in precision-oriented search is making them less interested in exploration.

As for the prioritization of engineering investment, I believe it is user-driven, rather than vice versa.

I don’t disagree. Just like more people reach for the doritos (fat, sugar, salt) than reach for the carrot. But that doesn’t justify or “make good” the national junk food industry. Or mean that one should only produce junk food, because that’s where the “user driven” arrow is pointing.

Perhaps users would take more of an interest in exploratory search if the tools were better.

That’s what I feel. If you were to actually cook a delicious meal from scratch and you might actually get more people to eat that, instead of the lik-m-aid fun dip. But if you never give them that choice, they will never stray from the lik-m-aid. And if you instead start manufacturing pixie stix, because that’ll let you get your sugar fix even faster than lik-m-aid and users go for those even more, then you’re only exacerbating the problem.

But I don’t think that investment in precision-oriented search is making them less interested in exploration.

That’s the opposite of what I was saying. I am saying that investment in precision oriented searching takes away (robs?) investment in exploratory searching. So that the users never get a chance to satisfy their appetites on the good, from-scratch meal.

The more time (and money) you spend manufacturing corn syrup, the less time (and money) you can spend making that real meal. Is that not an obvious conclusion to draw?

I actually suspect that making individual precision-oriented queries faster increases exploration for those who have any interest in it, since it lowers the cost.

Gene already addressed this issue above. Given that most of one’s time is spent reading, assimilating, and synthesizing information (on the order of minutes or even hours), I really don’t see how an extra 2-5 seconds speedup really helps the user explore any quicker or easier.

@ jeremy
My guess is that we (in the IR community) and those who use search in their professional lives do not represent the majority of users. As such, a facility that helps to formulate the query is going to make the average user happy. Reading ahead and understanding the ‘read ahead’ is much faster than thinking, formulating and typing the query.

It talks to my earlier point about the continual drive towards convenience. I am biased because I do like autosuggestion and am fascinated by Google’s introduction of Instant.

My problem with the junk food analogy is that, to the best of my knowledge:

1) Helping users perform precision-oriented / known-item searches more efficiently isn’t bad for them. In fact, for many users these needs may be the most valuable.

2) Users aren’t doing less exploratory search because of this increased efficiency for precision-oriented / known-item searches. In fact, I expect that they are exploring more because, even if the support isn’t explicit, exploration is easier if the search experience is more fluid.

Also, I’m not sure how much we can do to modify user’s information-seeking tastes. We certainly can’t get folks to eat their vegetables in real life:

I love exploratory search. But that doesn’t mean that exploratory search is better for users than precision-oriented / known-item search — especially in the context of web search. The critique of junk food starts with its ill health effects. What is the analogous harm?

Perhaps a better analogy: making dictionaries (the ultimate precision-oriented / known-item search tool) cheaper and more efficient to access doesn’t make people read fewer books (which feels analogous to exploratory search). In fact, it may help people read more books, since the dictionary becomes a better enabling tool.

But that doesn’t mean that exploratory search is better for users than precision-oriented / known-item search — especially in the context of web search.

As I said above: “Don’t get me wrong: Google Instant, by itself, is not “bad” any more than sugar itself is bad. Of course its not. What bothers me is the increasing imbalance that it creates.”

I don’t think I ever said that one was better for users than the other. I said that only one, without the other, is bad. One can eat a bag of doritos, or a corn syrup-laden soda, from time to time and suffer no ill health effects. If one’s diet *only* consists of doritos or soda, that’s a different story.

Right?

And what Goog is offering is all doritos, only doritos, all the time. And now with Google Instant, it’s the new ranch-flavored doritos, i.e. more of the same.

But sure, let’s go with your dictionary vs. books analogy. Wouldn’t you agree that the best scenario is one in which users toggle back and forth between dictionaries and books? They read a book and come across a new word, which they then can look up in the dictionary. While looking that word up in (let’s say OED = Google), that OED is so good that it lists usages of that word over time, and they find that Lord Tennyson used it in a book of poetry 130 years ago. So they then go to that book, and see how Tennyson creatively (exploratorily) strung those words together.

Yes, I agree, that’s the ideal scenario.

But remember, in your analogy, books (or poems) = exploratory search. Not the product of a precision-oriented search. Exploratory search gives the ability to see how various concepts and words and thoughts and ideas are strung together, by algorithmically surfacing those connections to the user.

And right now, very little of that is happening. To continue in your analogy: Using Google (OED) today is like reading the dictionary, and then *never* reading a book. And you ask how that is unhealthy? It’s unhealthy, because one stays ignorant, and never gets exposed to the non-obvious, non-intuitive connections that arise via exploratory search. One stays in one’s own little world view, and never grows. And w/r/t the society in which we live, a healthy democracy depends not on canned, bias-confirming answers. It depends on active engagement, of the sort one gets during search exploration.

In fact, I expect that they are exploring more because, even if the support isn’t explicit, exploration is easier if the search experience is more fluid.

Yes, you said this above, and Jeff also brought up the same point on the FXPAL blog. I need some evidence, or at least a couple of anecdotes, if I’m to even consider believing the veracity of that notion. You may be right, but as I again said above: “Given that most of one’s time is spent reading, assimilating, and synthesizing information (on the order of minutes or even hours), I really don’t see how an extra 2-5 seconds speedup really helps the user explore any quicker or easier.” Or more.

I guess this goes back to my earlier question about the statistics of the number of iterations users engage in when using Google Instant. Do you have/are you allowed to provide those stats? Users who you have classified as having exploratory search information needs, who used to do 3 rounds of searching.. do they now do 5 rounds of searching, with Google Instant? Or do they now do 2 rounds of searching? Or still 3?

Please, if you can share any info along those lines (assuming that someone at Goog is thinking/measuring this type of thing, and not just measuring the amount of time spent doing a single query), it would greatly aid this discussion.

@ jeremy
Maybe Google wasn’t supposed to be like this (based on its idealist early days) but it is not surprising that it has gone down this route.

If Google is the McDonalds or Krafts of the search industry then you’d think there was a place for a Whole Foods version. An organic version that goes at 45mph with sophisticated tools to perform deeper/meaningful searches.

These are arguments you’ve made before and I wonder what the answer is.

These are arguments you’ve made before and I wonder what the answer is.

Yeah, I keep hammering these points, don’t I? I don’t bring them up in a vacuum, though. I bring them up again in the context of yet another precision-oriented move by Google (i.e. Instant).

If Google is the McDonalds or Krafts of the search industry then you’d think there was a place for a Whole Foods version.

Yes, that’s the counter argument that always gets made. “If Whole Foods is so important, why doesn’t one exist?!” I reject that argument, though, because it presumes that there are two different markets. Daniel said as much.

But both the proteins/sugars/fiber/fat analogy, and the dictionary/book analogy don’t make the “separate market” presumption. They acknowledge that precision-oriented and exploratory needs are two sides of the same coin, i.e. that a person can’t live on all protein, any more than he or she can live on all sugar. Both sugar and protein are needed.

So what we really need is a precision-oriented system as good as Google, but one that ALSO offers exploratory search.

But I reject the notion that it’s one or the other. That you use one engine for precision-oriented needs, and another for exploratory needs. You should be able to *pivot* back and forth between the two, on the same engine.

Which is all the more reason I am aggravated every time I see Google take yet another big, time and energy-consuming, architecture-rewriting known-item step, with no accompanying exploratory step. It’s like riding a bicycle with only one wheel. Sure, it can be done. It’s called a unicycle. But wow, it’s difficult, and you can’t go very far on it, much less up hills or offroad or anywhere interesting that you’d really like to be able to go. You have to stick to the paved, well-traveled city roads. Which is really what Instant does.

So what we really need is a precision-oriented system as good as Google, but one that ALSO offers exploratory search.

Absolutely. And perhaps you feel Google’s known-item search quality and interface is good enough that it’s not worth improving. Google clearly feels differently–hence the investment in those improvements.

Google is investing in exploratory search. There’s room to debate whether the investment is sufficient. And I’m in a tricky position here because there are investments I can’t talk about. So I’ll accept if you feel the investments are insufficient.

What I maintain, though, is that making precision-oriented search better is not making exploratory search worse. At worst, this would a no-op for users and a big waste of investment for Google. Obviously I don’t think that’s the case.

What I maintain, though, is that making precision-oriented search better is not making exploratory search worse.

In absolute terms, expressed via what currently is: Of course you’re correct.

But in opportunity cost terms, I still maintain that it is harmful to exploratory search. Or, at least harmful to the searcher as a whole, because it perpetuates the imbalance. It oversaturates one nutrient and completely misses the other, which causes the searcher as a whole to be improperly nourished. Do you see what I mean?

And I understand that this is a matter of perspective, only. Not of “rightness” or “wrongness”. It’s a matter of whether you or I see two markets vs two sides of the same market. If one sees two markets, then of course there is no harm with Instant. But if one sees two sides of the same market, then naturally there is a problem.

I feel like we’re having the information retrieval version of a liberal vs. conservative debate

Google is investing in exploratory search. There’s room to debate whether the investment is sufficient. And I’m in a tricky position here because there are investments I can’t talk about. So I’ll accept if you feel the investments are insufficient.

My goal is in no way to get you to reveal something that you’re not supposed to reveal; I do hope you know that.

That said, if you are indeed allowed to talk about the number of searches that get performed within the same exploratory-categorized information need, and that number does go up as a result of Instant, I would hope that you could talk about that, just as much as others at Google talk about the 2-5 second per-query time savings. However, if the number of queries goes down, for exploratory needs, I would hope that you could talk about that, too. Either way, it would be an interesting datapoint to add to the discussion.

Coming in to this party a little late, but can I just pick up on one of the threads above: there’s lot of discussion around notions of “average” users and “typical” search tasks and so on … which I think are totally valid questions and fundamental to understanding the utility of features such as Instant. What we’ve just posted on Search Facets is an attempt to articulate these factors as “dimensions of the search experience”. Whilst we’ve tried to adopt a reasonably principled view, it is by no means the final word and I’d welcome other opinions on the utility of the framework.

BTW, we had some formatting problems with IE yesterday where the post got truncated half way thru. If you did stop by, then I’d invite you to take a second look:

Edited excert from Tony’s post “… let’s clarify exactly what we mean by Google Instant (GI). The basic idea is that instead of presenting a static page of results after each query, the search results are updated in real time after every key press as the user is typing.”

You know, I’m sticking with “Queries have been getting longer over the years. Consumers don’t like writing long queries. Seems to me that Google Instant is solving this problem.” And, will add that consumers (ie. Joe Public) know that they can rely on Google’s results and it isn’t the results per se that are the focus of Instant but the query formulation. Looking at it from Google’s perspective, it will now be collecting ever finer-grained query logs which will help them in myriad ways including voice search.

So what’s your take on the argument that Google Instant *can* support exploratory search, by letting users rapidly iterate on queries? Can’t users “learn” and “investigate” more successfully, if they get instant “navigational” results back?

I personally don’t really buy it, but I would be interested in your take.

Jeremy: I guess it depends on how we define exploratory. On reflection, I think the post may have an undertone that Google Instant ticks all the boxes on the left of each dimension, but none on the right… But I don’t believe that to be true, at least in the sense that users can and will use it to rapidly iterate on queries in ‘exploratory’ scenarios (particularly in cases where their domain knowledge is limited and the assets provide suitable (i.e. meaningful) feedback). So in that respect, perhaps GI wasn’t the ideal example to illustrate the framework – but it was topical

The thing I am just trying to get my head around now is what happens when you extend the Instant approach to include not just results but facets & values too – I’m not sure what the design principles will/should be for this but it’s certainly an interesting question.

Tony: The problem with facets in W3 search seems to be that they just don’t come with web content/indices. Ontologies offer super-classes linkable to keywords/index words, but (as some ontology management has taught me) their sub-classes are far to heterogeneous to enable an intelligent exploration/selection. – My latest insight is that one needs to combine semantic component analysis and taxonomies in order to get usable groups of facets for web search. I am sure one could combine this approach with Google Instant (or the like), which still does little for orientation in searching.

Lars: yes, I agree totally about the limitation with W3 search – I guess my comment (/open question) was more aimed at site search or enterprise search (or anything for which reliable faceted data is available).

The other question I’m pondering is how Instant should intersect with query correction strategies such as auto-correct and DYM – my first instinct was that it doesn’t make sense to invoke it until after an explicit key press.
After all, being able to rapidly explore lexical variations is one of the key benefits of Instant results – so if we override that with an opaque algorithm (however well intentioned) we will break that benefit completely. But I can see some edge cases where it might be useful. There’s also some wider issues of censorship & privacy to this, which are outlined here: http://www.malcolmcoles.co.uk/blog/google-puts-the-anal-in-analytics/

And, will add that consumers (ie. Joe Public) know that they can rely on Google’s results and it isn’t the results per se that are the focus of Instant but the query formulation.

@Dinesh:

So Google subtly prompts the user to be more specific, when the shorter query isn’t disambiguating enough? (Not disambiguation in terms of jaguar/jaguar. But disambiguation in terms of typing in “49ers”.. and then clarifying that you meant tickets, schedule, roster, etc.) Is that what you mean by query formulation?

It’s important to keep in mind that “exploratory search”, as Marchionini defines it, is not just about facets. It’s about learning and discovery.

One thing that I would *really* like Google (and Bing and Yahoo) to do is give us a way of turning off various ranking signals, such as popularity. Or “sort by recency”. I want to be able to type “49ers”, but not have a directed goal, such as tickets or roster. I want to see what people have said about the 49ers in years past, versus in this year. And typing “49ers what people are saying” isn’t going to cut it. Instead, adding and removing other, non-textual cues to the query, could very well get me to web pages that I might not have otherwise found, by adding *any* more terms to the formulated query.

That’s exploration. *That* would be valuable to me, if it were made instant.

@ jeremy
I am on your side with your points but I’m looking at Instant from Google’s motivation and its impact down the line on other services. For example, it is obvious that if the query logs become finer-grained then it becomes ever easier to train voice recognition systems (across multiple languages). The Instant query logs should also help with Translate.

I don’t quite follow.. how would Instant help Translate? Would user’s decisions about whether to continue typing or to stop, when using Instant, serve to give Google better feedback on the veracity of their n-gram conditional probabilities? Which then translate to better Translate? Hmm. I could see that.

Sure, Google is correct when they say that many image formats are outdated, and can be improved upon. But then how do they go about improving them? Do they offer a nice color management system in Chrome? Far from it:

Who cares if the images load a few milliseconds faster.. I want the images to look better.

As usual, it’s speed over quality. And while it’s true that some fancy new compression algorithm might not hurt the relative placement of the pixels in an image (introduce artifacts, etc.), it comes at a cost. All the time spend optimizing size means that the colors won’t be right, because there is no color management built into Chrome. All the engineering talent has been focused elsewhere.

So going back to my point about separate markets vs. two aspects of the same market.. this is yet another example. It’s not enough just to get images to load faster. You also have to get the colors right, once they do load. The overall quality of the image is a function both of pixels/artifacts.. AND color profiles. And to only focus on one aspect, and ignore the other, keeps the overall quality of the image as a whole from improving.

I see the same patterns of lopsided optimization in their web search arena. I know, I’ve said this a dozen times above. I felt the need to share the image blogpost/example, though, to drive the point home.

I switched to Chrome for its speed. On the occasions I use Firefox for sites that require it, it feels painfully slow. And I still notice slow page loads for some image-heavy sites–especially when I’m tethering to a non-broadband connection. I don’t perceive a problem with image quality. Maybe I’m just an impatient philistine.

Am calling you no such thing, nor am I claiming to be some great artist or intellectual, myself. It’s just that when I read that Chrome blog post, I perceived yet another example of the same sort of Google mantra, only this time in an area unrelated to web search. The idea that you want images to load slightly faster (new compression/image format), vs. wanting the images to be more beautiful/accurate/true (color management). It’s the same pattern of behavior, expressed in a different domain.

It’s frustrating, because Google claims that they focus on the user and all else will follow (quote: “Whether we’re designing a new Internet browser or a new tweak to the look of the homepage, we take great care to ensure that they will ultimately serve you, rather than our own internal goal or bottom line.“).

But that always seems to translate out the same way. It focuses on only one kind of user. And leaves the rest of us in the dust. Yes, that was a speed-related pun.

@ jeremy
I should have said “could also help with Translate” and was referring to the translation of languages through their voice recognition service.

Google has always focused on ‘fast’ as a core tenant and so no surprise there wrt Instant. This would permeate other products and services and when a call has to be made between fast image loading vs image display quality, then I’d assume fast wins.

My point is simply that you can’t have it both ways. You can’t say that your corporate culture is to make everything fast, because fast is categorically better than slow.. while simultaneously saying that you’re going to focus on the user exclusively, and what the user wants, rather than on what Google wants (again: “Whether we’re designing a new Internet browser or a new tweak to the look of the homepage, we take great care to ensure that they will ultimately serve you, rather than our own internal goal or bottom line.“)

It’s their internal goal to make things fast, while it’s my goal as a user to have beautiful (color managed) photos and web search results that let me look deep into the web collection: understand, synthesize, learn, discover.

Looks like those internal goals are winning.

(And please note that I say this not to criticize Google for the sake of criticizing Google.. but because I’d really like to see some of that Google quality engineering applied to *my* user needs.)

Again, I understand that pattern you’re trying to generalize, but I actually think Chrome isn’t a good example to make your case. The importance of speed in web search may be controversial, but browser speed (especially startup time) was a problem for a lot of users, and Chrome has attracted a very passionate following in large part because of how far it has gone to address that problem.

But, alright, then let’s not talk Chrome.. let’s again talk about Google’s attempt at yet another image format for the web: WebP. That was the main subject (rather than the Chrome browser) of that Google blogpost that I cited to kick off this non-search subthread:

“Thus we come back to the conclusion I’ve made over and over on this blog — the encoder matters more than the video format, and good psy optimizations are more important than anything else for compression. libvpx, a much more powerful encoder than ffmpeg’s jpeg encoder, loses because it tries too hard to optimize for PSNR. These results raise an obvious question — is Google nuts? I could understand the push for “WebP” if it was better than JPEG. And sure, technically as a file format it is, and an encoder could be made for it that’s better than JPEG. But note the word “could”. Why announce it now when libvpx is still such an awful encoder? You’d have to be nuts to try to replace JPEG with this blurry mess as-is. Now, I don’t expect libvpx to be able to compete with x264, the best encoder in the world — but surely it should be able to beat an image format released in 1992? Earth to Google: make the encoder good first, then promote it as better than the alternatives. The reverse doesn’t work quite as well.”

I sense a high degree of irony here. This is the same thing that folks have been saying about Yahoo and Bing web search. You can’t just have a search engine that is as good as Google’s… or even quantitatively 10-20% better, otherwise people aren’t going to switch. You have to have a search engine that is qualitatively different or better.

And with this whole new image format, the blog writer makes the point that what matters, qualitatively, is not the image format itself. It’s also not the entropy characteristics of the encoder. It’s the psychological characteristics of the encoder. It’s that “good” is better than “fast”.

And yet Google still says “fast” is better than “good”.

Whatever example we want to use, am I not correct in saying that there is a problem with categorically declaring these two things at the same time (a) we serve the user, not our own internal goals, and (b) fast is better than slow? When you’re lucky, “fast” will serve the user best. But there are many information need, image viewing need, etc. scenarios in which the fastest solution is not the one that serves the user best.

Exploration may be affected adversely by the auto-complete feature of Google Instant, as it will try to route searchers to similar previously found results. This may lead the searcher to accept these “popular” results, thereby undermining the exploratory aspects of search. Increasing speed doesn’t kill exploratory search, decreasing diversity does.

Agreed. Catering to lazy users is great when there is a clear-cut best answer, but it is likely to reduce diversity in other cases. But the alternative is to force users to do more work when we could let them do less work and satisfice. It’s an interesting trade-off.

No, this whole idea of forcing users to do more work is a false dichotomy. Because if the user has an “explorational” type of information need (the Gary Marchionini sense of exploratory search, by which is meant things like learning, synthesis, comparison, analysis, discovery, etc.) then the user already has to type in 100 different queries to get what they want, even if auto-complete is turned on. If auto-complete saves the user 0.7 seconds per query, that’s a meager improvement, and does not fundamentally change the overall time (and effort) that it takes to complete the task as a whole.

No, the alternative is for the system to do something more than just return 10 results. The alternative is to provide explicit query modes for automatically synthesizing or comparing different queries. For example, imagine if you had the ability to type in two queries side by side. Two columns, two query boxes. And then what the system could do is give you an intelligent “diff” between the two sets of results.

Just knowing how two concepts were similar or different might save you from having to type in 30 more queries, until you got enough understanding to give you the same level of information. Again, here are the choices and consequences.

(a) user types 100 queries, no auto-complete. Diversity is larger, but effort is great
(b) user types 100 queries, with auto-complete. Diversity is smaller, but effort is 10% less per query, because of auto-complete.
(c) user types 20 queries, but types them as side-by-side comparative queries. user gets as much information as from 100 queries in (a), but has to expend 80% less effort, because only 20 queries are typed.

What I dislike in these discussions is that the argument is always framed as a choice between (a) and (b), when really (b) is a red herring. The choice should be between (a) and (c). And (c) blows both (b) and (a) out of the water.

However, it costs the search engine much more in terms of processing power. And it’s not always clear what sort of ads to show next to a “comparative query”. The user saves immensely, though. Which is why it frustrates me that we never see type (c) queries offered by mainstream web search engines.

Forcing is a strong word. But I think there’s no escaping that, at least in some cases, encouraging users to explore all options vs. satisficing is a trade-off. In other cases, there is no doubt that supporting exploration can save time. That is one of the reasons I love working on systems that exploratory search!

By not providing a certain type of functionality, I think you are indeed “forcing” the users to use only what is left, what is available.

And if what is available won’t do what they’re trying to do, won’t explicitly support the exploration that they’re engaged in, then yes, I think it is fair to say that the search engine is “forcing” them to do more work.

And it has nothing to do with exploring all options vs. satisficing. Going back to my example, about how it would take 100 queries on a standard search engine (even with auto-complete turned on), vs. 20 queries on an exploratory search engine. Whichever one the user is using, they still have the choice to “explore all” vs. “satisfice”.

Do you see what I’m getting at? Doesn’t matter if you’re satisficing or going full-tilt. A search engine purpose-built to your type of information need is going to allow you to do less work, overall. And auto-complete is not that type of engine. Even when satisficing and using precision-oriented autocomplete, you have to do more work (nay, are “forced” to do more work!) than you do when satisficing and using an exploratory engine.

I think you see that, Daniel, but I think 97% of the rest of the search community doesn’t. I think that they think that if you add up those 0.7 second savings across 20 satisficing queries, you save more time than if you only have to do 4 satisficing queries.

Well, I no longer work for Google, but I do appreciate the benefits of Google’s auto-complete functionality — including instant results. Of course, I’d also like rich support for exploratory search. But I’ll take what I can get.

But folks like “cute love quotes gal” (which again, represent 97% of the community), above in comment #65 need to understand why 0.7 seconds faster on a single query does not necessarily translate to faster overall task completion time — and in fact might be not only slower overall (because you’re having to do 5x the number of queries to get the same information, even if you’re only satisficing) but also require more work from the user overall (because you’re having to do 5x the number of queries to get the same information, even if you’re only satisficing). When you have an exploratory information need.

[…] This means that having top billing on the search engine is more important than ever. Here is a fun test: type in search terms that you would like your company to rank for. See yourself? Now type in related but different keywords. How about now? Google Instant allows you to test your SEO instantaneously!

In the coming weeks, Google is likely to offer some tips for webmasters – either directly through Webmaster Tools and/or on their blog. It may pay to visit these sites more frequently than you normally would – at least for the next 4-6 weeks. […]