Opening government, the Chicago way

Chicago looks to use its data, developers and citizens to become a smarter city.

Cities are experimenting with releasing more public data, engaging with citizens on social networks, adopting open source software, and finding ways to use new technologies to work with their citizens. They’ve been doing it through the depth of the Great Recession, amidst aging infrastructure, spiraling costs and flat or falling budgets. In that context, using technology and the Internet to make government work better and cities smarter is no longer a “nice to have” … it’s become a must-have.

In 2011, with the election of former White House chief of staff and congressman Rahm Emanuel, Chicago has joined the ranks of cities embracing the open government movement. Before his inauguration, Emanuel released a strategic plan that explicitly endorsed open data as a part of Chicago’s future. The new administration hired its first chief technology officer, John Tolva, and a chief data officer, Brett Goldstein. In the months since, the new Chicago government is doing something notable, as far as governments go: it’s following through on some of its open government promises.

Interviews with Chicago journalists and open government advocates, along with Tolva and Goldstein themselves, led me to a clear conclusion: there’s something new going on in the Windy City that’s worth sharing with the rest of the country and world.

“Appointing Tolva and Goldstein was one of the biggest ways in which Rahm has followed through,” said Virginia Carlson, president of the Metro Chicago Information Center (MCIC), in an interview this summer. “The two of them make for a powerhouse, with Brett helping with releasing the data, in terms of the APIs and the time he’s spent with the community.”

The city has been releasing about two datasets a week since the new administration came into office, said Brian Boyer, news application developer for the Chicago Tribune. (That data trend is a big part of what motivated Boyer to work on the Panda Project.)

From where Tolva sits, what’s happening in Chicago is not limited to open data or involving the tech community in improving the city. The culture of the mayor’s office “changed radically with Mayor Emmanuel,” said Tolva (@ChicagoCTO), speaking in a phone interview this summer. “I’m seeing the passion of the startup world here.”

There’s a long road ahead for open government in Chicago — the legacy of corruption, fraud and graft in City Hall there is legendary, after all — but it’s safe to say that a new generation of civic coders and city officials are doing what they can to upgrade both the tone and technology that underpins city life.

“There was a lot of catching up to do,” allowed Tolva. “A lot of it has been the open data publication. We’ve been getting very high-value datasets out almost every day. We launched an app competition. We got a performance dashboard up.”

All of that is only the first step, he said. “It’s part of a larger vision for stoking the entrepreneurial fires, where open data is used for much more than transparency. Data is a kind of raw material that the city encourages people to use. We’re working on a digital roadmap and thinking more broadly. What can we do that will help businesses make the city more livable in a systemic way? One way we’re going about that is rethinking what public space means. What are the kinds of data and interoperability standards that will allow that invisible architecture to be as accessible as a park is, and as malleable in purpose?”

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively.

Tolva also offered some constructive criticism for the technologists in the open government movement to consider: “The community of civic nerds has not done a great job at engaging the big civic innovators who have no knowledge of technical skills or that area,” he said. “We’re trying to bring them together. One of my roles — the reason we’re in the mayor’s office — is to try to be that translator between the architects and the urban planners of the world and technologists.”

Tolva said he is working on both economic development and applying technology to empower others to help the city work better. “I’m working with the commissioner to evangelize and convene innovators in Chicago’s technology community, including Threadless, Groupon and EveryBlock. We want to promote that sensibility from the mayor’s office, in terms of business developments. One third of my job is the analytics part of that, bringing data-driven decision making to the city departments, down into the individual commissions.”

The movement toward opening up Chicago’s government data predates the Emmanuel administration, as Carlson reminded me when I asked about new releases since the inauguration.

“The conversations started in November of 2009,” she said. “The city has been building its data catalog for over a year and a half. We’ve been waiting for someone to come in and pull the switch. Maybe one quarter of what’s now available was available before the new mayor took office. Three quarters of the data was sitting on internal servers waiting for someone to say, ‘yes, we can publish it!’ The salaries of city workers, for instance, was absolutely something that Rahm has released, along with lots of 311 data.”

311 data has been the target of much of the initial open government activity in cities around the country, given the insight it can provide into the problems that citizens are reporting and the customer service they receive from their governments. When city officials can look at what 311 data can reveal about their urban environments, for instance, new opportunities emerge for improving the way government can target its efforts in cooperation with developers and citizens. That’s the kind of “citizensourcing” smarter government that Tolva is looking to tap into in Chicago.

“This is as much about citizens talking to the infrastructure of the city as infrastructure talking to itself,” he said. “It’s where urban informatics and smarter cities cross over to Gov 2.0. There are efficiencies to be gained by having both approaches. You get the best of both worlds by getting an Internet of things to grow.”

The most important thing that Tolva said that he has been able to change in the first months of the young administration is integrating technology into more of Chicago’s governing culture. “If a policy point is being debated, and decisions are being made, people are saying ‘let’s go look at the data.’ The people in office are new enough that they can’t run on anecdotes. There’s the beginning of a culture merging political sensibility with what the city is telling us.”

“I’m proud — and a bit harried by — the number of people asking for a regression analysis,” said Tolva. “We have policy analysts who are dabbling with ArcGIS and trying Python.”

The business case for open data

Like every other metropolis, Chicago has budget constraints. In the current economic climate, spending public dollars has to provide a return for taxpayers. Accountability and transparency are important civic goods — but making a business case for open data requires more grounded arguments for a city CFO to support these initiatives.

“The mayor is firmly committed to innovation that really matters and that can be built upon,” said Tolva. When it comes to the business case for open data, Tolva identified four areas that support the investment, including an economic rationale:

Trust — “Open data can build or rebuild trust in the people we serve,” Tolva said. “That pays dividends over time.”

Accountability of the work force — “We’ve built a performance dashboard with KPIs [key performance indicators] that track where the city directly touches a resident.”

Business building — “Weather apps, transit apps, that’s the easy stuff,” he said. “Companies built on reading vital signs of the human body could be reading the vital signs of the city.”

Opening Chicago’s data

Opening up Chicago’s government data further will take time, expertise, and political support, along with a lot of hard work. Applying it is no different. For now, Tolva and Goldstein have the former three components firmly in hand. The latter is what lies ahead.

“In the realm of public safety, I had a good sense of the relevant data structures,” said Goldstein in an interview this summer. “The city is an enterprise that’s so large, with so many different functions and so many different data structures, that making sense of the landscape and developing a plan is a challenge.”

From enterprise resource planning systems to public health to transportation, there’s great diversity in how city data is structured and stored.

“One of the things Chicago has done very well is collect data,” Goldstein said. “Now, one of the things we need to do is develop a holistic vision for an enterprise data architecture and data warehouse. How to do you take the things that are meaningful from architecture and then make them meaningful to the public?”

Given the challenges involved here, it wasn’t surprising to hear Goldstein say that “we’re not where I want to be yet” — but he’s approaching the process methodically. “I want to know the entire lay of the land, have everything mapped out and understand the next steps.”

As he looks ahead, Goldstein is less worried about access or load concerns, given the city’s use of the Socrata online platform for open data. He’s more focused on sustainable design.

“I want to make sure that the path we take the city on is sustainable and has a more open architecture,” he said. “I find that when we choose proprietary solutions, it’s hard to get the data out. If I’m going to sit down and code, I’m going to do it in Python, use Linux, and I’m going to be happier about it.

Goldstein is well aware of persistent issues around data quality that have dogged the use — and reputation — of open government data releases. “I’m very traditional in how I deal with data,” he said. “It’s the same as working with analytics. You need to make sure data is clean and high quality.”

The process to get to clean data is, as Goldstein described it, quite methodical: “We have multiple phases for how we roll out data internally, starting with working with the business owner. We figure out how we’ll get it out of the transactional database. After that, we determine if it’s clean, if it’s verified, and if we can sign off on it technically. “

The last step is analyzing whether the process is sustainable. “Some people send a spreadsheet, upload it and maintain it manually,” said Goldstein. “That’s not sustainable. We have hundreds of datasets. We’re not going to do that. You need to write code that updates data on its own, and then you can focus on new datasets.”

At a high level, Chicago’s chief data officer emphasizes the value of open data in providing the city with insight into its business processes. “Opening data alone isn’t enough,” Goldstein said. “We’re giving people the data to make meaningful apps and do meaningful research — but are we putting out a tabular dataset? Is it spatially enabled? Are we offering KML files directly versus a downloadable file? If we keep the KML file updated, then [application developers] can access the data directly from the app.”

In this respect, Goldstein’s focus on making data clean, sustainable and directly available suggests that he’s attuned to what citizens want when they build applications. An open data study from late last year found that a majority of citizens prefer to explore and interact with data online, as opposed to downloading data to examine in a spreadsheet.

To fully embrace this vision, however, Chicago is going to have to build out its data capabilities to become a smarter city. “The first step is moving over to a more open platform,” said Goldstein. “You don’t have to make a multi-million-dollar investment to get a fancy GUI and something meaningful. If you bring something over to Linux, between Python and R you can produce some remarkable outcomes. These are some really low-cost solutions.”

They’re looking to use city data to make the city more productive and the processes better, said MCIC president Virginia Carlson. “For example, what if the city wants to understand zoning and the retail food landscape? Using its own food licensing and food inspection data, they can see where food is being sold. If Walmart is coming in, can the city mine its own data to understand where food deserts are and have a much richer understanding of its landscape?”

The city won’t be working on this alone either, emphasized Goldstein. “We have great academic partners and lots of people coming to the table. We don’t need to be afraid of using these tools. It’s high time.”

Refining apps competitions

The design of the Apps for Metro Chicago competition, offers some insight into how Chicago has learned from what other cities have done in their own open government and open data efforts. The competition is taking a next-generation approach, trying to provide technical assistance and connect communities with software developers.

“When I think about where we are, versus a San Francisco or Boston, it’s because of examples of what worked and what didn’t,” said Tolva. “The judging criteria for the competition takes into account the sustainability of an idea, along with its cross-platform nature.” In the video below, city officials talk about open data and building applications that are useful to the community.

Given the points that have been raised about the sustainability of apps contests, tying development to the demonstrated needs of citizens looks like an idea whose time has come. Look to the submitted ideas for NYC Big Apps in version 3.0 of its competition, for instance.

“We’ve elevated business viability in the judging rubric and are working with a great partner, MCIC,” said Tolva. With regard to NYC BigApps 3, “there are all kinds of apps that we’d love to have,” he allowed, but the applications in Apps for Metro Chicago have to solve business problems.

“The judging rubric has it that you have to demonstrate community participation and then release open source code,” said Carlson. “The app has to be free to users for a year. We’re very conscious that we don’t want this to be a big competition … and then it’s over.”

Tolva also focused on building community around apps contests and bringing more voices into the process. “We’re using the Apps for Chicago to get a new kind of civic engagement and participation, which you can get involved in whether you write code or not,” he said. “We’ve invited community leaders and groups to the table. The idea for a ‘Yelp for social services’ didn’t come from a technologist, for example. We’re curating ideas from non-technologists.”

The hypothesis in Chicago is that this hybrid strategy will result in better outcomes for taxpayers, developers and, crucially, citizens. “The apps competition needed to have a data expert, with someone outside of the city running it,” said Carlson. “Justin Massa helped write the rules. Chicago was the first place to bring in unbiased external experts. Can we understand what we need to by doing open data right? This story is just beginning. The questions will be if, in six to eight months, whether this model works. We need to promote data sharing and cleanliness between data departments, to have data tickets, an internal account and a liaison, who can share that information, getting that productivity feedback and communication with developers.”

The better part of an apps competition is the feedback on the data, said Carlson, not just how the city can use data on the public-facing side but apply data on the enterprise architecture side. “We’re trying to capitalize on the cool factor to enhance internal processes, working with staff, and trying to get data to understand the city.”

Writing the rough code of history

“We have been trying to get data out of state and local government for more than 20 years,” said Carlson. “For me, to see this tide coming along from loosely affiliated millennials willing to stay up all night is inspiring. That’s what’s creating the energy to free up the data — this distributed network that’s been living and breathing opening up the data.”

There’s more than the energy of millennials to celebrate here, however, as she emphasized. “They’re pushing the data out to citizens as a way of running the city,” she said. “It’s in a business enterprise kind of way
— that’s the way Rahm is thinking about it. Using it internally hasn’t been emphasized a lot, but it’s a big part of what they’re trying to do.”

To get anywhere close to achieving that goal, Chicago will have to close the IT gap between the public and private sector, particularly in the emerging field of data science.

From the outside, it looks like the city’s technology officials are hungry to improve how Chicago uses technology. “In the private sector and research community, we do cutting-edge work,” said Goldstein. “Why shouldn’t the government do this? Why should the bar be any lower?”

For now, as the new administration finds its way, there’s hope that Chicago will take a leading role among other cities adopting open government.

“The combination of committed political leadership, engaged civic leaders and a vibrant start-up scene has made Chicago the place to watch for people who care about technology and society,” said John S. Bracken, director of media innovation at the John S. and James L. Knight Foundation, when asked for comment. “We’re living in what is potentially one of the most important times in the city’s history.”

Tolva is bang on in focussing on the disconnect between urban technologists and planners. I would go one step further and suggest an even more important disconnect, that between teckkies and politicians. What is still lacking (although there has been progress) is a complelling street-level narrative that will move open government into the political mainstream.

http://www.zylun.com/live-chat-outsourcing/ Live Chat

Props to Chicago leadership for this move toward transparency and technology. This is definitely an example I hope other municipalities and larger divisions of government will follow. Perhaps it will also lead to more cities casting free wi-fi umbrellas over their territory. At least we can hope.

http://www.touchtonic.com Paul

You know, it will be interesting. When your parents get on networks / apps you initially feel uncomfortable. I wonder how people will take with the interaction from not just political leaders in Chicago but all cities in the future. I’m not sure it will improve the relationship, but it could do some very interesting things.

A lot of different, creative ways to think about this. That would be nice if they gave everyone free wifi – but don’t count on it so soon :)

http://www.touchtonic.com Paul

You know, it will be interesting. When your parents get on networks / apps you initially feel uncomfortable. I wonder how people will take with the interaction from not just political leaders in Chicago but all cities in the future. I’m not sure it will improve the relationship, but it could do some very interesting things.

A lot of different, creative ways to think about this. That would be nice if they gave everyone free wifi – but don’t count on it so soon :)

Featured Video

The growing role of software architects: “Architecture has become much more interesting now because it’s become more encompassing," says Neal Ford, software architect and meme wrangler at ThoughtWorks.