Why is Open Data difficult for Governments?

I’ve been asked this question a lot lately. And recently, a journalist asked me this during an interview for a story on his problem getting data on noise pollution from the Hong Kong Government. Christopher DeWolfe quoted me in the article, Hong Kong’s Silence on Noise Pollution (which was also printed in the SCMP and is online behind their paywall):

Without open-source data, there is no way to see if a similar situation exists in Hong Kong. “There’s potential for the government to be embarrassed – it’s like having an auditor, and that’s why they drag their feet on releasing information,” said Darcy Wade Christ, a researcher for the University of Hong Kong’s Journalism and Media Studies Centre’s Open Government Data initiative, which is pushing for more transparency in the government.

This question comes to me a lot because of my work at the JMSC on the OpenGov project, but also because I am very involved with the Open Data Hong Kong group. I am more than happy to discuss this issue because I think it is at the heart of how we begin to form a new relationship with our governments all over the world.

As I said in my quotation, opening data has the potential to cause a lot of trouble for the government. This explains, in part, why they are reluctant to open up certain datasets. There are obviously other reasons, including costs required to convert or manage ongoing public data. But by far, the most important is that Governments do not know what opening certain data will do. All governments will be conservative when it comes to things they do not understand. Technology (especially the Internet) has always presented this problem, because it is new, always changing and its effects are unknown. And since there is no government without politics, all decisions pay some heed to the political ramifications of doing anything.

When I was talking to Christopher DeWolf, I wanted to underscore the idea that Open Data does pose the potential for embarrassment because it enables citizens (and politically motivated groups) to audit and present displeasing results. But it also has the potential to improve how the government functions for the very same reason, in that it provides additional auditing. It can also improve the relationship between citizens and their government by engendering more trust. For these two reasons alone, it is in all of our interests to continue to find more ways to open up public data. Some people like to stress that any data the government has generated should be public because it was funded by taxpayers. I don’t usually use this argument, because it can end up dead-ending around the issue that not everything should be public – that the government is given the responsibility to protect all citizens, and that some data may negatively affect a certain group of people. I believe the Hong Kong Government takes this issue quite seriously. They are always conservative when it comes to embarrassment. This may have been why I chose to use that word (wisely or not). The HKSAR has a history of defamation lawsuits, but it is also part of a culture that believes strongly in not publicly embarrassing others. There is a time and place for showing your power, I have come to understand.

So when we talk about the reasons why any government should open more datasets, let’s keep in mind that this is a complex issue and that all arguments have many sides to them. The recent global appeal to Government Transparency and Open Data is good, but it will take time. We have to be certain of discussing both the good and the bad. We have to appreciate that governments will drag their feet and given their very jobs (not as public servants, but as politicians), we have to at least acknowledge why they may not embrace the ideals we believe in. But we can still make a little noise here and there. We can still improve how we speak and use powerful words like audit (are you afraid of an audit?). And we can also continue to explain why we want certain data. The Open Data Hong Kong group is often discussing Open Data as a general mandate. In my work at the OpenGov project, we are usually going after lesser understood and known datasets (land use and corporation information) without fully understanding the goal of what we want. We can extol the importance of having more information. We (especially journalists) can begin to articulate a premise we might have – that having certain data will help to prove or disprove our idea. But no matter what any of us do, this process will take a long time, because politically, there is very little incentive to open oneself up to critique.

We could all afford a little auditing in our lives, but who invites an auditor into their house?