How Not To Do Open Data

At the Socrata Customer Summit, Waldo Jacquith challenges open data enthusiasts to take their work to the next level. He doesn’t make them comfortable, though. “You are doing a mediocre job,” he announces. “You came here today because somebody told you that you needed to come here.” He then gives the audience a bit of a break, admitting, “maybe – and by maybe, I mean probably – you’re working under difficult circumstances.”

He moves on, schooling attendees, “Most of your sites have very little data that’s of any value to anybody.” He lists examples, using publicly available datasets he found beforehand, and proving these datasets are useful to very narrow constituencies. Then, he prods his audience about their open data practices, suggesting, “you inflate your dataset count by breaking one file into eighty pieces,” and continues, “you have no idea of who your audience is.”

He then slaps this reality check on them: “You have no clear goals or planned outcomes for your open data efforts – so how will you know it’s working? Well, easy: you won’t.” Jaquith gives his audience a breather, reminding them, “remember people, this is all delivered with love.”

Then, he doubles down.

What You’re Doing and Why It Ain’t Working

Jaquith starts scrutinizing open data portals across the Internet, showing his audience what really is – or isn’t – offered in them. He lays into Vermont’s open data portal first. He contends, “The whole site is a bolt-on. It’s evidence that someone was checking an ‘open data’ box, and moving on.” He moves on, skewering Madison, WI., then the state of Michigan, and still others. In this speech, no one’s attempts at open data are safe from Jaquith’s spotlight.

“You’re like puffer fish,” he accuses, “inflating your datasets so your open data program will look fearsome.” He acknowledges this might be a goodwill gesture toward the public, but contends it’s mainly to look good internally. “You need to show impact, and big numbers are a great way to fit that.”

The Commitment Gap

Speaking of elected officials, Jaquith says, “Governments talk a good game about open data,” but don’t back it up with budget and staffing. He gets a lot of laughs quizzing his audience about the reality of their budgets, and commenting on what could be done if funding matched the level of talk by higher-ups.

Then, he drags attendees back in the ring. Jaquith declares, “you’re producing open data for no concrete reason,” reminding the data producers, “the number of datasets is not a goal or outcome – it’s just not.”

Jaquith attacks feel-good open data projects many governments take on. He begs governments to stop having hackathons, to dump apps contests. He points to the scant benefits of these efforts. Very limited demographics participate, with results that seem clever to judges, but don’t enrich the public. He challenges open data workers to set their sights higher.

Power to the People

“You have to go to people where they are,” Jaquith explains, using the example of restaurant health inspections. Not many people are going to think of looking for an app for them, so “that means the data has to be in Foursquare, or Yelp, or Google.”

But true open data isn’t only a question of how and what, it’s also hugely about who, Jaquith shows. “You decide what data to publish by consulting quasi-random twenty-somethings. Stop.” Don’t keep going to programmers, go to the actual data experts in the government agencies, he urges. “They know what data is useful, they know what people ask for,” he points out. The right role for programmers is to turn the expertise of data producers into something beautifully accessible. The data experts, “know what data is reliable and what other agencies ask for.”

Make Your Open Data Awesome

Jaquith tasks his audience to move past easy data grabs, both to make their open data sites actually useful, and for their own sake as the people who will be compiling numbers and analyzing data as time goes on. He advises them also to automate every data acquisition and update they can, to lighten the workload and ensure longevity of the system. Then, Jaquith walks his audience through five real-world best practices for their open data sites.

He wraps with a warm emphasis that his talk was a standout chance for him to get down to it with people who share his love for open data. “I’ve never spoken to a group of exclusively people who work in open data within government. When I talk about your work publicly, I’m going to describe it in glowing terms.” He then invites them to Q&A, “Let’s talk – or yell at me.”