John Battelle On The Future Of Search: Part Two

In the first part of my interview with John Battelle, we talked about the actual search experience—the act of searching and our expectations of what the results of that act might be. But as we started talking about change, it soon became clear that change in the act of search translates into change throughout the entire search industry.

Ironically, as I write this, I’m down on Captiva Island emceeing the Search Insider Summit. To be honest, this interview with John and this entire series played a large part in determining the central question we’re addressing here—namely, where is search going from here? And many of the things John Battelle and I talked about are emerging in various presentations. Everyone who’s paying any attention to the industry at all agree on one thing: everything we know is about to change.

Search as the all knowing assistant

One of the poster child apps that’s shaking up the current status quo is Siri, an iPhone app that offers a personal digital assistant that’s really a voice activated meta-search tool. Battelle offered Siri up as an example of a different type of search experience. The challenge with creating an effective digital assistant, however, is that an assistant is only as good as their knowledge of you and your wants. To be truly useful, an assistant needs to be able to understand exactly what you’re looking for with a minimal investment in communication from you. The best assistant is the one that responds to your needs intuitively, anticipating them with no explicit expression on your part. But, as John explains, this is not a trivial challenge to overcome:

Battelle: Siri is trying to do that very thing. This is a very nontrivial problem. Netflix, which I just signed up for again after abandoning the DVD base because I just kept losing them, I finally signed up to the streaming because my kids wanted to watch movies over the weekend. So I signed up for it and now Netflix thinks I’m an 11-year-old girl. The instrumentation of a relationship is difficult. Humans are infinitely complex and so this is not an easy problem to solve, but the fact that there are so many smart companies trying to solve it is thrilling to me.

Expressing intent in a way that’s understandable to a machine is one way to help increase the odds of success, and it’s one that Microsoft’s Stefan Weitz and I talked about in an earlier interview. Human language is incredibly ambiguous and nuanced. Even with all the experience we’ve had communicating with each other, we constantly misunderstand what someone else is trying to tell us. So, I asked John, is cracking the language code in a way that can be parsed by a machine the answer?

Battelle: I think its part of it, I don’t think it’s the whole enchilada. Look at the valuable information that you can extract from how any one of us interacts with a well-designed application, then create a dataset for that. Say I use the New York Transit application to navigate my way through New York for 3 or 4 days… all of the questions and back-and-forth that I use that app for, which is essentially a structured search session—right? Now, match that against a set of data which is the transit map. I say, “I need to go over here. I want to go over there. I prefer this route over that route,”—that becomes a dataset that should inform other searches that I’m making on things that seemingly are unrelated but may not be. That should be available as metadata for future searches. And figuring how to inform that is as important as parsing the line or the spoken phrase that I’m making in the moment.

Now, if I take that spoken phrase and go and search for “Chicago rental car” four months after interacting with that New York Transit map application, how can we take the metadata from that interaction with New York and inform the appropriate response in Chicago. Perhaps the best suggestions would be, “Hey, you know what? You don’t need to rent a car. You can use the Chicago Transit. Here’s an app for it. You can get from the airport to everywhere you want to go without having to rent a car. Plus, you’ll save $150 which we know is a goal of yours because you’ve been interacting with the Mint application and it said that a goal of yours is that you want to save $200 a month and here’s a way that you do that”?

Tying all that together, that’s the Holy Grail because then it starts to understand you. If you only parse just the query, even if you get the natural language right and the intent right, you’re missing the whole person.

Who do you trust?

Of course, to allow this degree of intimacy with any assistant, even a digital one, requires you to surrender a tremendous amount of personal information, which brings up trust as a concern. Whom do you trust with sensitive information about yourself? Because this can go way beyond transit usage patterns into a degree of revelation that lays our souls bare.

As John was talking about this, I thought of a book I read a few years ago by Marc Singer and John Hagel called Net Worth. In it, Hagel and Singer introduce the concept of an infomediary, someone who you trust with private information about yourself. The infomediary then acts as both a gatekeeper and match maker to help you in future consumer choices. In the book, which was published in 1999, Hagel and Singer speculate at length about the type of organization that may emerge as the infomediary.

Perhaps, given the promise of making our lives easier, the cloud based apps that power these digital assistants will come to fill this role. It’s a tremendously powerful position to hold, which means it’s also potentially tremendously profitable. So, if this comes to pass, who might be successful? Certainly, both Google and Microsoft seemed to be well positioned to emerge as the dominant players:

Battelle: I think they have a chance of touching it, either through partnership or through self-declaration by their consumer. Look, for example, at what Google’s doing with social search. The way that they’re tooling it to work is “Hey, tell me who you are on Twitter, here’s your Google profile, tell me other URLs that are important to who you are and we will try to instrument around what you’re willingly telling us.” I think people, if they see a value, will willing give a service data that is orthogonal to that service, right? So if I know that my search experience is going to be better if I tell Google who my LinkedIn profile friends are, or if I know that a particular app is going to be smarter because I connected through an API to my Mint feed, that’s a trade I may be willing to make.

I think personal feeds and the consumers’ ability to say, “Sure, you can have my feeds because I’m going to see value from it and I know that we’re in a trusted relationship”… I think that that handshake is going to be increasingly made in our culture. I think, however, that we need to have a conversation about that handshake and understand it. We’re in the midst of that and it’s going to get more and more interesting over the next decade. I think that the handshake between large services now and what will become a flood of new streams of valuable data from apps, from interactions on other sites and services will allow a Google or a Microsoft to touch and have access to a ton of data about us. But the bond of trust and the cultural contract that we have with those services is going to have to be very well understood. I think we’re sort of slouching our way there, but we’re increasingly having a conversation about that cultural contract and social contract.

Who’s going to win?

What Battelle is touching on is a information marketplace with extraordinary implications. As I said, the monetization potential is immense. And it’s not only Google and Microsoft that may emerge in those roles. Some would argue that Facebook is already much further down the path. But what if it’s none of those? What if we’ve never heard of the company that may emerge as trusted repository of our most personal information? That’s where Battelle is placing his bets:

Battelle: It’s hard to handicap it this early, but I would go with an unknown at this point. Really, I think that, you know, just like 10 years ago, no one would have said that we’d be talking about Google the way we are. I think 10 years from now it may well not be any of the ones we know today.

As a person who has had the opportunity to get a privileged glimpse into Google, I had to ask John if their current success might actually play against their chance of breaking through into this new opportunity. Are Google’s hands tied by their current successful but heavily search-centric revenue model?

Battelle: I don’t think so. I think the culture of the company is one that’s going to allow it to continue to surprise itself. I do think, like any company that has billions of dollars in profits sunk into one main line of revenue, it’s going to make decisions to protect that revenue which may retard its ability to compete in new markets. But at the same time I think that they are hyper-aware of that. I haven’t seen a lot of companies that are seemingly as aware of the potential for their own demise. Another way to put it is the Yellow Pages is still an insanely profitable business. And they’ve been supposedly dead for 15 years.

I think that they’re going to be fine and I also think they don’t have the Yellow Pages culture. But the fact is, there’s a lot of spaghetti on those walls at Google. They’re trying a lot of new stuff.

Boiling the ocean, aardvarks and the Singularity…

John and I wrapped up our interview by talking about another dark horse, or in this case, aardvark. Aardvark (now a Google property) tries to bridge the man-machine interface issue by tapping into the power of social networks. Aardvark distributes your questions to the people in a social network best able to answer it. But anytime you introduce humans into the equation, you introduce scalability issues. What, ultimately, is the best approach to connect the requirements of humans with the efficiency of machines?

Battelle: That’s a great question. I don’t have a short answer for it. I think that the folks at places like Aardvark are obsessed with trying to figure that out. Every time there’s a pattern, you code it and move on, and the patterns are almost infinite. I think the goal is not to understand what ocean you’re boiling but just keep boiling a cup at a time and eventually get to the ocean.

But it is the greatest question of search, which is how do we get to the point where we can meld man and machine? I think our whole culture is this sort of grand narrative marching down that particular road, and hopefully by the time we get there we’ll be smart enough to know what it means to have essentially created life. And that’s a big, amazing, interesting concept—you know, created life artificially. And that’s I’m convinced, knowingly or not, why most of the people are in this industry, because it’s just got this sort of sexy, immortal vibe to it—we’re working on a big problem that if we solve it could either destroy us or let us live forever.

As I said to Battelle, that’s a hell of place to leave it, but John has never been accused of understating anything. So, with shades of Ray Kurzweil and the Singularity now enmeshed with online search, I’ll leave it here. In the next column, I turn to Google with my original question: Where is search going from here?

Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.