Share

In 2011, Airbnb had a problem. The room-sharing site was growing fast, but so were customer complaints. People just couldn't figure out how to use the service. The issue was so severe, Airbnb was getting an average of one customer service call for every room booked.

"We thought we were building this really efficient platform," Riley Newman, Airbnb's chief data scientist, said at a San Francisco conference this week. "But really, from this perspective, we were like a phone-based booking service."

To figure out how to fix this problem, the company asked Newman to look at the data. At a time when the concept of "Big Data" has become a marketing cliché, the assumption is that folks like Newman can wave their hands over a Hadoop cluster and instantly conjure the answers they need. But as Newman's ultimate solution goes to show, sometimes the best resource for solving data problems is the one that gets the least credit: the human brain. For startups especially–even though so many of their brethren love to hawk automated thinking–the most efficient tool for the job might be made of flesh and blood.

The Air Divers

At Airbnb, Newman first did some fairly standard computer-aided number-crunching to learn that most calls came either when customers were booking rooms or heading out on their trips. Then Airbnb's data nerds created a hierarchy of tags to funnel the calls into the right buckets. But that still didn't solve the problem, Newman said at Databeat, a Big Data-themed conference hosted by tech blog Venturebeat. It just pointed out the hot spots on the site where people were having problems.

To find out more about what was going on, Newman assembled a small team that became known as the "Air Divers"–the people who would dive deep into the individual complaints and surface with answers. Each was given a couple hundred support tickets connected to a specific issue that the data had identified as a hot-button topic. They would go off and read through each one, then come back and propose a fix. And in the end, this is what turned the situation around.

Newman said that, among many data scientists, the default assumption is that sophisticated natural language processing algorithms should be used to automate all that reading. Algorithms won't take as long, people assume. But the irony, Newman tells WIRED, is that startups don't always have time to arrange for such computational heavy lifting. "We needed answers for what we were going to do by the next day," Newman says. "We didn't want to take the time to build out a perfected model."

The Human Vacuum Cleaners

During a later presentation at the conference, a leader of the IBM team that oversees Watson, its Jeopardy-winning supercomputer, described the effort to model human behavior in computers as a shift toward probabilistic calculations. Humans might not be so good at making precise calculations at the same rate as computers. But we're very good at vacuuming up vast amounts of unstructured data from the world around us and using our intuitions to make guesses that often turn out to be right.

Plus, there are other advantages to going the human route. Newman showed a photo of Airbnb in its earliest days, when the founders and early employees all sat crammed together at one table. If someone in customer service identified a problem, he could just tell the product guy across the table what was wrong.

As for the fixes his human brain identified, Newman said they weren't too complicated. A consolidated trip page to better answer the most important question: "Where am I going?" Better tools for the people hosting Airbnb users. Tutorials. FAQs. The approach might seem quick and dirty, but the data-processing might of the human brain was enough to bring customer service calls down 75 percent. "Sometimes," Newman said, "it's the simple things that get you a lot further and a lot faster."