Google Equates "Design" With Endless Testing. They're Wrong

Two days ago, Google unveiled Google Instant — the company's newest solution to making googling even faster. The feature presents search results which appear even as you're still typing (for background, watch the super-creepy, super-enthusiastic, almost Onion-ish video below).

Google describes the challenge as being to present that mass of information whizzing by your eyes in a way that's "relevant not distracting," so the solutions thus had to do with grouping information in clever ways. For example, here's one prototype that trying to group results from similar searches only after you typed:

And here's a version with the suggested search results to the left of the toolbar:

Obviously, none of these were going to work — the first prototype above was bound to show you irrelevant searches; and the second simply increased the number of places on the screen you had to look to perform a search. A little design know-how would have made that obvious, but Google instead tested these ideas out.

We ran through a sequence of prototypes, usability studies (testing with people from the community), dogfooding (testing with Google employees) and search experiments (testing with a small percentage of Google users).

If you have any type of design background, it's probably funny to you that Google frequently mentions "design," but doesn't mention any "designers" involved — the Google design process seems to simply be creating a bunch of fairly obvious alternatives, and testing the hell out of them.

Before Google Instant, probably the most infamous example of Google's design-by-testing approach was the "41 Blues" —- Google's engineers apparently couldn't decide on two shades of blue for showing search results, so they tested 41 of them to see which attracted the most clicks. (They eventually settled on a blue that is basically the average of all the blues used in hyperlinks across the web. Duh.)

You might imagine such a robotic approach to design would be frustrating for anyone with a design background. And you'd be right — Douglas Bowman, a prominent UI designer and now the creative director at Twitter, left Google, apparently due to day-to-day inanity of trying to actually design a user experience there:

I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can't operate in an environment like that. I've grown tired of debating such minuscule design decisions. There are more exciting design problems in this world to tackle.

He expands:

When I joined Google as its first visual designer, the company was already seven years old. Seven years is a long time to run a company without a classically trained designer. Google had plenty of designers on staff then, but most of them had backgrounds in CS or HCI. And none of them were in high-up, respected leadership positions. Without a person at (or near) the helm who thoroughly understands the principles and elements of Design, a company eventually runs out of reasons for design decisions. With every new design decision, critics cry foul. Without conviction, doubt creeps in. Instincts fail. "Is this the right move?" When a company is filled with engineers, it turns to engineering to solve problems. Reduce each decision to a simple logic problem. Remove all subjectivity and just look at the data. Data in your favor? Ok, launch it. Data shows negative effects? Back to the drawing board. And that data eventually becomes a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions.

Sounds about right, given all we know about Google.

But back to Google Instant. What's baffling about the whole thing is that Google's "solution" to providing instant results still seems so primitive and ugly. In the name of shaving a second off of a user's search, is it really worth it to make them go through the pain of scanning five to seven different results pages as they type?

A second might matter tremendously to an engineer — and indeed, Google is happy to point out that those seconds, over a year, add up to hours of time saved. But that's kind of a silly way of thinking about it. After all, it's not like you get all those hours at the end of the year as a dividend. If you saved a second while googling, what are you going to actually do with that second? Take another sip of coffee?

The chief mandate of design thinking is empathy — and I'm pretty sure Google's engineers didn't have too much empathy for all those over the age of 28 who don't find it all that useful to have their eyes assaulted by information they weren't looking for in the first place.

Which brings me to my last point. Testing can only tell you so much — and it often only reveals that people only like things that are similar to what they've had before. But brilliant design solutions convert people over time, because they're both subtle and ground breaking.

For example, imagine if Google, in presenting its progressive, instant results stripped all the text away from below the hyperlinks. What if the details only filled in as your typing began to slow? Might that have decreased the visual confusion? Could they at least have worked on a way of reduce the sheer pixels of information you have to look at, while also providing you with more content? I'm pretty sure that testers would have complained about "missing something." But over time, I'll bet that the reduced strain would have emerged as a boon.

Testing can, at best, prevent massive mistakes. But it can also give you a blinkered perception of reality — and that's just as dangerous. Remember that New Coke was vastly preferred over regular Coke in blind taste tests — the New Coke disaster happened because marketers never tested the deep real-world associations people had with the brand. Coke is bitter and acrid, and that's what makes it a "Coke."

The exact wrong lesson to take from that is that you should test even more, with greater detail. The right lesson is that testing artificially limits the worldview of the people doing the testing. Market research can't tell you whether the "problem" you're trying to solve is even the right one to be addressing. It can't tell you that the entire project you're working on was a bad idea to begin with.

Google is the perfect example of this. As they've added new products, has your G-mail or Google Reader gotten any easier to use, or less stressful on your eyes? Have either of them become a pleasure to look at or play with? No. But why not? Every new Apple product seems to make progress on that — and I'm not even talking about hardware. Just look at the music listings in new, redesigned iTunes.

Then again, I'll bet the testers (and engineers) still prefer their Google as it is now, simply because it's what they've always known.

Add New Comment

14Comments

I see no problems on Google's Searching Design. As a company running by Engineers and at the end of the day, what really matters is that Google search works.However, as Google keeps increasing its products portfolio a consistent common Design language is required to make all the things working together.That's why Apple is always a step ahead of other companies, Design plays an important role on product development and drives the decision making process for new launchings. At Apple Engineers follows what Designers demands, at Google is the opposite.

I can't believe you didn't address google image search! Isn't that the ultimate proof? Seems to literally just get worse and worse over the years, or maybe just so marginally better that it seems worse with the addition of time.

Interesting post. However, while it gathered interesting information on the subject, I saw a lot of criticism and I didn't see any interesting constructive criticism or suggestions. By the end of the post, if I had to summarize: "Google, you should hire an expensive Designer and slow down on user tests"... It feels a little preachy. In fact, I thought the fact you mentionned the new iTunes interesting because I hated Ping and the whole iTunes 10. It hasn't been catching on, it really doesn't really serve your point.

There is a body of philosophical thought and cognitive science research suggesting that the best tools are the ones that become direct extensions of our bodies, or in the case of information technology, of our mind. For instance: once I become skilled at using a hammer, I forget about the hammer and focus on the nail. When a tool is slow/laggy, it kicks the user out of their flow state -- milliseconds matter in the perception-action feedback loop. I push back against this point of view in this article, and suggest that tools for search and information management that respond instantly to the actions of the user will allow them to explore the information space more thoroughly and effectively -- forms of these tools will be the future of our interactions with digital content.

Google is the most used brand on the internet. Their sites are simple and the services interconnect in effective, efficient and pleasing ways. Google.com looks better, cleaner, more intuitive than any other serach engine.

All the testing (especially the 51 shades of blue - yikes!) reminds me of the authority that movie production companies used to give their test screening subjects (and perhaps still do): if enough people said that they would have preferred a happier ending, they would change the ending of the movie.

You'll never end up with a single, brilliant concept using such methods. These require at least a little autocracy, which, I think, is what makes Google's "20% time" generally so effective: they are cohesive, possibly radical, ideas in which a single individual really believes. Quite the opposite of what is essentially a "design-by-committee" approach.

I definitely had some similar thoughts upon using Google Instant. Here's a post I wrote about how this "feature" reveals a dangerous attachment to current solutions and a use of their dominant position to force things upon customers. http://somethingventured.me/20...

I don't see why this would be subject of cricism. User testing is the one and only validator for good design decisions. Every good designer knows this, especially when the decisions closely pertain to User Experience.

Google's design process might be broken, but I think it's a misunderstanding (albeit a common one) to blame testing for their failures -- testing should allow them to test *more*, and *more creative* ideas, not fewer, more narrow ideas. That's the beauty of testing, and it's a shame Google doesn't embrace this, as they have the traffic, and had the talent to do some amazing things. Google's problem isn't that they test too much, it's that they don't test enough. Why not test the idea you mention? Why not test hundreds of other ideas? If anyone could do it, they could.

I covered this issue specifically in regards to Doug Bowman leaving Google, which you mention, in this piece: http://design2-0.com/articles/... (Bowman kindly called it a "must-read" on Twitter.) Be interested to hear what you think!

Great article. However, while 'testing' can limit design thinking, design 'research' can expand it. There's a big different is putting ideas out there and getting feedback -bad ideas AND good ideas-- versus exploring needs, behaviors and attitudes and factoring them into the design process.

Any company that puts out new products is going to have some failures you can point to, but short of making the embarrassingly obvious comparison to Apple this article doesn't offer much substance to back the argument that Google doesn't design well. Sounds a lot like someone who got a degree in "Design" but is upset because the they didn't get an interview with Google.....

Great article. The video you embed really is on the creepy side. Google is making a huge splash about Instant and after using it for a few days, is it really such a big deal? A better test for Instant would be to make it an preference that surfers can turn on or off.

It's sounds like they take a statistical approach to the matters - perhaps not too surprising, considering how much the operations of their more popular enterprises would depend on statistical values, such as search ratings, and more detailed qualities, such as word-stemming when spidering the pages. Maybe they'll bop on over to FC sometime, and start to catch the design light of day :) Some more industry-wide, open, and perhaps slightly academic work, on the practical sides of human-computer interface design might also help to support as much, that Google would get a little more "into" the finer, subtler aspects of such things, and a little less into such that could be qualified as statistical hair-splitting.