The challenge of data-driven decision making

Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that.

Overall, the approach makes sense. The people who work for the company use their knowledge, experience, and insight to come up with potential solutions to a problem. Google tests the various solutions to see what works, and based on the data from these tests, decide what the best solution is. (I wonder what happens when the tests are not conclusive?)

Google’s process is based on a belief that everything can be quantified, that the goals for the work and the qualities being sought after are something that can be measured, counted, and statistically analysed. This is how most companies work, so as Bowman himself says, it is no surprise to see these beliefs and practices in play at Google.

In most cases, however, individuals still at least get to make implementation decisions without having every small decision challenged. Organizations trust their people to do the work they were hired to do, just as they trust them to review the performance of their work and make the necessary adjustments.

Google process is set up to minimize the impact of any given person in the design process. Success will be determined by the system and by the data. The only thing the designer does is provide different options to be considered. Then the testing machine takes over, the final design determined by statistical tests. In this environment, there are no star designers, no person or group of people who’s insights are responsible for the company’s success.

Compare to a company like Apple, which despite doing I would assume to be a fair amount of number crunching, leaves design decisions in the hands of people. Jobs and Ive to be sure, but the other designers at Apple as well. Of course, it may be that designers at Apple face a different tyranny, of having their decisions challenged and overruled on subjective grounds by people like Jobs or Ive instead of by hard data.

The difference here is that in Google case, the people are replaceable. Someone like Bowman, as talented as he is, can leave Google with little impact on the quality of the design of Google’s products. The same cannot be said of Apple, who are most likely highly dependant on Jobs and Ive as the final arbiters on design decisions.

Bowman states that he isn’t interested in working for a place where design “lives or dies strictly by the sword of data.” The key word here is strictly. Any designer has to contend with the fact that at some point, to some degree, the impact of their designs will be quantified, their success measured by numbers and data representing some form of return or benefit to the organization. The same can be said of anyone working in a organization, not only designers. The question is the degree to which design decisions are based on data rather then on the expertise of the person involved, and what impact that will have on the ability of the company to retain skilled people.