You are here

The Big and Small: The Benefits of a Data Friendship

Nate Higgins

October 26, 2017

I’m an analyst at heart. I think in numbers, trends, and statistics. My organization, Cambridge Systematics, was founded 45 years ago by four MIT professors who wanted to improve transportation by using systematic processes to solve challenging problems. Naturally, given our heritage and practice, we find ourselves often referred to as a blend of academics and management consultants. We equally are comfortable sitting in the boardroom discussing how to help invest billions of dollars as we are sitting in front of a research panel debating the next great problem-solving framework. We can back up our discourse, of course, with numbers.

That’s what differentiates us – the need, and the ability, to prove our work with facts. We apply our curiosity to answer our clients’ questions -- we developed procedures and code to analyze 500 million rows of travel time data, built open source software to display the locations of all of the buses in New York City, and analyzed sensors on trucks in California to test for emissions. We also model behavior, answering questions like: How many people would want to use a new commuter rail line? Where would they come from? How would the price of a ticket impact the number of riders? Analyzing these big datasets gives us insights into how and why people and goods move.

On the other hand, my friend Buck Sleeper, Senior Designer at Continuum, digs in deep to understand how individuals behave. Rather than focusing on broad understanding, as we do with big data, Buck gathers information by interviewing a few of the “right” sources to develop a deep understanding of what people do. I’m convinced that mashing up his qualitative approaches with my quantitative approaches will help us view things in new ways. We want to see the city and all of the people in it, at the same time.

Closing the Credibility Gap

Big data analysis helps us build a broad understanding of how a population makes decisions, but it doesn’t tell us how you make your decision. The data is good at telling us how to improve mobility by improving people’s commutes, but it isn’t good at telling us how you feel about getting to your work. Understanding both at the same time could change the game.

Big data helps us measure the performance of a system, but it can’t show what truly matters to people. We can relate changes in transit fare policy to ridership, and changes in ridership to system performance. But what do people truly want? What do they really need? Is it more important to keep fares low or to offer a delightful experience? In one great example, Continuum learned that ferry riders in Boston are among the happiest commuters. Why? Because they have views of the ocean and refreshments on board. While we can’t deliver a waterfront view to all commuters, we can talk about providing a refreshment cart that could make transit rides more pleasant.

On the flip side, we big data folks can help the small data perspective by targeting market segments. We have tools that can tell us the demographic profile of the population passing through any selected roadway link on their way to work. With this in hand, Continuum could target interview candidates much more quickly and make the case that these folks represent the general population.

Understanding our Customers

We measure things, but I want to make sure we measure the right things. We know a ton about how people and goods move and are learning more every day. We strategize with our clients about how to solve problems. We write plans. And we implement plans. But our approach can be enhanced by taking into account the behavior of the people being impacted by these changes. Using big data and broad brush strokes alone makes it challenging to answer questions like: What is travel truly like for the poor working class? Is it true that people in the poor working class tend to borrow and share vehicles among them? Do inconsistent, changing work schedules make it difficult to get where they need to be? Are there strategies available to help these community members get to work affordably and quickly? These are the types of questions we’d like to answer.

Buck’s ethnographic approach has him working with a small number of people to understand how they experience things. He can uncover people’s true selves, looking beyond what they say and observing what they do (because these often are different). I can imagine how pairing these insights with a traditional planning exercise could help us. We could talk in the same breath about economic development for all and how to get the poor working class to work.

The Benefits of a Data Friendship

This qualitative and quantitative approach to the world will help us tell a better, more complete story. The small data perspective will help make our system-level view more human and more relatable. Our big data perspective will help Continuum tell the story about how their findings can scale. In my mind, this is a friendship worth exploring.