Open Data might have failed, but Open Government is still going strong.

It would seem that the shine is starting to wear off the Open Government movement, with a recent report to the US congress challenging some of the assumptions which drove the dictate out of the U.S. Open Government Office{{1}}, forcing U.S. departments to publish their data sets. The report found that simply pushing out data has negative outcomes as well as positive ones (which should be no surprise), and that often the cost of pushing out (and maintaining) a data set didn’t outweigh the benefits. Most importantly, it raised the question of whether or not publishing these data sets was a good use of the public’s money.

So, has the business case behind Open Government been found lacking in the harsh light of day? Or is this one of those cases where some faith is – similar as with the investment in the U.S. highway network – because the benefits of stepping into the unknown are not calculable with the crude mechanism of ROI. The truth seems to lie somewhere between the two.

I wouldn’t confuse the investment in the US road network post WWII (or AU’s current investment in a NBN) with Open Government. The former was an investment in an asset which the U.S. government of the time made largely on faith, an investment which is currently seen to be returning $14 billion to the U.S. economy annually. (Australia’s NBN might be heading on a similar journey{{2}}.) The latter is actually a philosophical point of view about an approach to government.

The problem is that we confuse “Open Data” with “Open Government”. They’re related, but not the same. Open Government is a move to streamline service acquisition and delivery by exposing the bureaucracy of government and integrating it more tightly with other service providers, and has been progressing nicely for a decade or more now. Open Data is a desire to change the relationship between government and the population, reducing the government to a simple data conduit between the public (or corporations) providing services and the public consuming them.

Open Government has made government easier to deal with by making it easier to find and consume the services you need, and by fostering community. Everything from applying for the dole, getting a grant through to organising a council supported street party is orders of magnitude easier than it was a few decades ago, mainly due to increased transparency. This has been delivered via a range of means, from publishing information on line, through providing better explanations for the services offered and promoting multi-channel access and self service delivery. The latest wave of Open Government is seeing departments integrating external services with their own, putting even more data out in public in the process, as they move from a service-provider to a service-enabler. Ultimately though, if government (as separate from politics) is focused on keeping folk feed and feeling safe then it’s doing it’s job. It’s basic Maslow{{3}}.

Open Data, though, is based on the view that government should do as little as possible, hand over the data, and let individuals in the public get on with doing what they want. It’s claimed that this will provide transparency (the public has all the data, after all) as well as fostering entrepreneurs to provide innovative solutions to the many problems that confront us today.

It’s quite possible to have transparency and Open Government without the need to publish all your data, and maintain these published versions, as claimed by the Open Data proponents. People need to understand how the wheels of government turn if they want to trust it, and the best way of doing this is usually through key figures and analysis which builds a story and names the important players. Drowning people in data has the opposite effect, hiding government operation behind a wall of impenetrable details. Wikileaks was a great study in this effect, as it was only when the traditional journalists became involved, with their traditional analysis and publication weaving together a narrative the broader public could consume, that the leaks started to have a real impact. (It’s also interesting that the combination of the anonymous drop boxes being created by conventional media, and Open Leaks{{4}}’ anonymous mass distribution to conventional media, looks to be a more potent tool than the ideologically pure Wikileaks.)

Nor is treating government as an integration medium the only way to solve the world’s problems. While entrepreneurs and VCs might be the darlings of the moment, there’s many other organisations and governments which are also successfully chipping away at these problems. For every VC backed Bloom Box{{5}} who has mastered marketing hype, there’s a more boring organisation that might have already overtaken them{{6}}. The entrepreneur model will be part of the solution, but it’s not the silver bullet many claim it to be.

The problem is that Open Data is the result of a libertarian political mindset rather, rather than being a solution to a pressing need. Forcing government to publish all its data sets does not provide or guarantee transparency, nor does it have a direct impact on the services offered by the government. It can also consume significant government resources that might be better spent providing services that the community needs. Publish a data set of no obvious value, or build a homeless shelter? Invest in Semantic Web enabling another data set few use, or pay for disaster relief? These are the tradeoffs that people responsible for the day-to-day operation of government are forced to make. Claims by folk like Tim Berners-Lee that magic will happen once data is out there and ontology enabled have proven to be largely wrong.

However, Open Data does align with a particular political point view. Open Data assumes that we, as a population, want such a small government model, an assumption which is completely unjustified. Some people trust, and want, the government to take responsibility for a lot of these services. Some want to meet the government somewhere in the middle. Open Data tries to force a world that works in shades of grey into a black-or-white choice that driven by a particular world view.

Deciding what and how much the government should be responsible for is a political decision, and it’s one that we revisit every time we visit the ballot box. Each time we vote we evolve, by a small amount, the role government plays in our lives{{7}}. (Occasionally we avoid the ballot box and revolt instead.) Should government own the roads? The answer appears to still be yes. Should government own power stations? Generally, no. Should they own the dams? We’re still deciding that one.

It’s in the context of the incremental and ongoing evolution of government’s role in our lives that we can best understand Open Data. Forcing Open Data onto government through mandate (as Obama did) was a political act driven by a desire to force one group’s preferred mode of operation on everyone else. You might want Open Data, but other people have differing priorities. Just because they disagree doesn’t make them wrong. The U.S. congressional report is the mechanism of government responding by documenting the benefits Open Data brought, the problems it caused, and the cost. The benefits (or not) will now be debated, and its future decided at the ballot box.

Open Government is alive and well, and is driving the evolution of government as we know it. Services are being improved, governments are increasingly their integrating services with those of the private sector, and more data will be released to support this. The assumption that all government data should remain secret unless proven otherwise has been flipped, and many public servants now assume that data should be made public unless there’s a good reason not to publish. Government is investing in moving specific information assets online, were it makes sense, and departments are opening up to social media and much closer involvement (and scrutiny) with the public sector. The mechanism of government is evolving, and this is a good thing.

Open Data, though, as an expression of a political point of view, looks like it’s in trouble.