Harnessing the Full Power of Big Data

5 Insights from This Week's CityLab

Big Data – the use of large datasets to solve problems and deliver better “customer” value – was a big topic of discussion at this week’s CityLab, which brought leading urban practitioners and thought leaders together in New York to explore ideas in economic development, environmental sustainability and other issues that are shaping the world’s metropolitan areas. A plenary session on Big Data featured a conversation with Geoff Mulgan (CEO of NESTA), Emer Coleman (formerly of the London Datastore), Chris Corcoran (from New York City’s data analytics team), and Anthony Townsend (author of Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia). That session, combined with discussion of data in other sessions, provided an interesting lens into the current state of development in the field of cities and data. Here are five takeaways:

City governments need to expand their capacity in order to harness the full power of big data– Despite New York City’s strong focus on using data to drive policy, Chris Corcoran noted that out of approximately 300,000 city employees, fewer than ten are currently employed by the city’s data analytics team. As data comes to play a bigger and bigger role in the work of city governments, local officials and advocates are seeking to extend cities’ capacities in this area. Academic centers like the University of Chicago’s Urban Center for Computation and Data and NYU’s Center for Urban Science and Progress are helping to develop new tools and train a new generation of people who will be equipped to do this work inside of and outside of government.

As the amount of data available grows, so too will our understanding of that data’s potential–Chris Corcoran noted that New York City government receives about 50,000 data points per day from its residents, along with all of the data points generated by the city’s 300,000 employees. Across the country and around the world, the move to open city data is creating ever-increasing opportunities. Despite this increasing quantity of data, in many cases, as Emer Coleman pointed out, the return on investment for cities in opening up their data may not always be known in advance. But as more and more data gets opened, the more opportunities will exist for governments, businesses, and city residents to find new and in some cases unexpected uses for it. (For example, data analytics have helped to prevent building fires in New York, reduce rodent complaints in Chicago, and identify “super-users” of youth services in San Francisco). So we should expect an iterative and evolving process between data availability and data usage. Chris Corcoran helpfully sketched what one such arc might look like: from the reporting of data to predictive analytics to changes in resource allocation.

Applying big data to address big challenges remains a key frontier – Geoff Mulgan posed a provocative question: is big data at this point mainly a matter of a lot of hype and some nice apps, or does it have the potential to actually transform people’s lives? This is a question that we at Living Cities have been exploring -- a take on this question by Living Cities’ Ben Hecht and Code for America’s Abhi Nemani was recently featured in a blog for Meeting of the Minds. There was consensus that there are real opportunities to use big data to help provide “fundamental solutions to compelling problems,” including in the areas of health care, education, and transportation. The questions remain: how soon will we see these transformative impacts, and what will it take to get there?

Big data raises questions of representation and inclusion – With the massive growth of data, there is a concomitant need to ensure that the growth of data does not widen disparities of access and influence in local government. Indeed, as one panelist pointed out, even the term “big data” may sound intimidating and discourage engagement. So, on the one hand, it’s in the interest of inclusive governance to ensure that new data being generated reflects the city as a whole – crowdsourcing, as Anthony Townsend pointed out, may not generate scientifically representative data. At the same time, communities need more tools to generate and analyze data in a way that allows them to participate fully in policy discussions in their city. Over the longer term, cities and civic hackers are seeking to address the question of how best to update the processes by which local government interacts with its residents for today’s data-enriched world. Traditional governmental mechanisms (think about the archetypical town hall or community meeting) are being significantly rethought.

Trust, privacy, and silos remain critical barriers – Discussion focused on a number of the particular barriers on which progress is needed to enable the growth of data as a tool to help address city problems. First, city residents often do not trust city government with their data: as one panelist pointed out, some people have greater trust in providing their data to apps like Foursquare than they do to their own government. Second, and relatedly, proponents of Big Data recognize the need for progress on privacy issues, including embedding privacy dimensions in the design of data collection tools. Third, city governments are seeking new and proven approaches to reduce silos across agencies within government, and also how to reduce barriers to collaboration around data across sectors (e.g., with the private sector), so that all of the data relevant to particular problems can be assembled and analyzed.

As municipal governments work to modernize their operations to better tackle 21st century challenges, local officials, intermediaries and practitioners will need to address these issues in order to harness the full power of Big Data. We see breaking through these barriers as a critical step towards making cities places of greater economic opportunity for all. We’ll continue to share our thinking, learning, and activity around this subject as it develops.