Published

Good and bad transparency

I’m trying to work out why I’m uneasy about the recent announcement of the transparency revolution sweeping Whitehall. It’s not as if I’m a civil servant any more with a vested interest to conceal. As a developer, and an SME hoping to win public sector work, and a taxpayer, you’d think I’d be cock-a-hoop that I’ll be able to hold officials and politicians to account more easily from now on.

But I’m uneasy, and I think that’s because I’m troubled by the nagging doubt that the unintended consequences of transparency are often unpleasant and I’m not hearing much about their mitigations in public policy and discourse on transparency at the moment.

Let’s separate out four examples of transparency:

Opening up state-managed, non-personal datasets for wider use at low or no cost is Good Transparency. Too often, officials publish or simply file away useful datasets – lists of postboxes, recipients of New Years’ Honours, public sector vacancies and so on – in formats such as PDF that make it harder to turn that data into useful services for public good or commercial enterprises. We should make this data more readily-accessible, in flexible formats, and happily, that’s exactly what a Cabinet Office team has been doing very successfully for a year. That’s not really transparency per se, it’s just good publishing practice. (Yes, the fight to open up Ordnance Survey maps was a struggle, and is arguably a cross-subsidy by the taxpayer for work previously funded by the users of OS products. But I’d argue the wider social benefits of free mapping data outweigh this).

Opening up government contracts to a wider range of bidders, and so that smaller bidders in particular can win work, and understand how the labyrinthine government procurement machine works, is clearly a positive example of transparency too. It’s more controversial, in that what might on one hand drive down costs, might conceivably make it easier for cartels to develop, or inhibit the willingness of bidding companies to be frank and detailed in their proposals. And without reform to the very process, risk appetite and above all length of government procurement, simply being able to read more of the documents won’t entice more SMEs to get involved. But those problems are fixable, and greater transparency in this field is likely to secure real economic benefits.

Opening up performance data – timesheets, civil service salaries, government spending, MP’s expenses, whatever it is that enables rankings and comparisons in individual or institutional performance to be determined – isn’t such a clear-cut positive. We know that setting targets or criteria, and above all publishing data on performance against them, inevitably focuses effort on those, and risks distorting job performance in other areas (there are better sources on that, but I can’t find them right now).

Let’s take Joe Harley, now infamous for being one of the Whitehall high earners, with an annual salary of £245,000-249,999 for his work as CIO at the Department for Work and Pensions. Nice work if you can get it for buying a few computers, the media will say. Now, I don’t know if Joe does a good job or not (I don’t hear much about DWP IT failures – so he’s doing OK, right?). In fact, I don’t really know what Joe’s job is, or what the challenges are in delivering IT to support 120,000 people and managing a roughly £856m budget. I don’t even know what CIOs in other sectors earn, or how well they do their jobs relative to Joe, or whether someone better would be found to replace him if he were to go elsewhere, given the civil service recruitment freeze. All I know is that Joe works for DWP and earns just under £250k, and that that is quite a bit more than the Prime Minister.

It’s not that opening up performance data is a bad thing per se. But in opening up what will only ever be a selected few datasets which describe characteristics of an individual or institution’s performance, it’s human nature to draw inferences about quality and appropriateness which the data in front of us don’t justify. It’s a lesson MPs and MySociety learned a while ago, which is why they add a spurious statistic – how many times an MP has used alliterative phrases – to MPs’ profiles on TheyWorkForYou, as a gentle reminder that rankings don’t tell the whole story. Next time the civil service top earners list appears, I’d love to see an inside leg measurement data column (though I suspect this might show some remarkable, if somewhat spurious, correlations).

I have increasingly come to worry that there is an error at the core of this unquestioned goodness. We are not thinking critically enough about where and when transparency works, and where and when it may lead to confusion, or to worse. And I fear that the inevitable success of this movement–if pursued alone, without any sensitivity to the full complexity of the idea of perfect openness–will inspire not reform, but disgust. The “naked transparency movement,” as I will call it here, is not going to inspire change. It will simply push any faith in our political system over the cliff.

On one hand, I don’t like the idea of policy being bought, if it happens. But I’m not sure that knowing which industry groups a politician or civil servant has met is necessarily enlightening in itself. In my experience, a Secretary of State’s chance meetings with ‘real people’ at constituency surgeries and visits influence their thinking pretty strongly too, but the data would be unlikely to show that.

Government is a fascinating study in unintended consequences. Its scale and diversity, and the interplay between ministers and the civil service, the centre and the frontline mean that things often don’t go to plan. Broadly-speaking, government is a risk-averse organisation in many ways, sometimes to the extent that on occasion it has preferred ineffectiveness to perceived impropriety, waste to uncertainty.

The coming wave of transparency could transform this in a hugely positive way, using open data on costs, opportunities and performance to become a much more creative, cost-effective and agile institution, mindful of the money it spends and the results it achieves, and ensuring individuals are accountable for their work.

But it might make things worse, frightening senior managers into becoming more guarded, taking fewer ‘risks’ with even small amounts of money, and focusing on the process to the detriment of the outcome. It may also make public service less attractive not only for those with something to hide, but for effective people who don’t want to spend their time fending off misinterpretations of their decisions and personal value for money in the media. And to mirror Lessig’s point, it may push confidence in public administration over a cliff, in revealing evidence of wrongdoing which in fact is nothing of the sort.

Two cheers then for the revolution sweeping down the corridors of Whitehall. But let’s be mindful of its consequences as yet unseen.

Glad this point is being raised, as there’s nowhere near enough discussion of the effects of ‘transparency’. A lot of thinking (too much?) seems to assume that maximum accountability is better than corruption, but in reality all systems require some level of ‘privacy’ to remain functional – namely a) where the stated goals (the “story”) are out of line with the true picture (“reality”), and b) where experimentation and failure are needed in order for the system to remain flexible.

The same “problem” (or feature?) has afflicted surveillance systems for years, such as CCTV. An old blog post on the polarising nature of surveillance/transparency might be of interest: http://bit.ly/bcLzFo

I think it’s possible to categorise transparency into data about “things” vs data about “people” – or, perhaps, data about the world, and data about people. The difference being that when data is about people, the people it’s about will reflect on how that data will look rather than reflect on the job in front of them.

On the other hand, I think increased transparency needs to be set in a much wider context. Transparency is about monitoring things, but without a balance with other motivations it becomes an input as per above. What inputs, other than the threat of accountability, do we need in order to make a system work well? What makes *you* want to do a job better?

As an inveterate data fiddler, I’m quite keen on people providing data in easy to use forms. I was recently playing with the electoral data, and even for this pretty uncontentious area finding data is a rocky road. Local council election results, anyone? I’d also like to play with the OS 1:25,000 dataset…

There are downsides, in the US *some* of the transparency movement appears to have been driven by climate change “people-who-it’s-most-straightforward-to-describe-as-denialists” and it’s essentially an attempt to disrupt the work of climate scientists. And you can see an echo of that in the UK with the FOI requests to the Climate Research Unit in East Anglia.

As you highlight with the senior civil servants salaries, it can lead to populist witch-hunting. However now, if you’re so motivated, you can at least seek out the wider set of figures and see for yourself. In the past we would likely have been leaked a few of those salaries, rather than the whole lot. Can’t help thinking this particular data release may have unintended consequences in providing an upward pressure on salaries.

While it has potentially negative effects on the civil service, the other issue I have with it is that it puts the focus on process over outcome.

Inevitably, somebody will always fail to get the policy outcome they want, and blame an unfair process rather than the fact that they simply failed to make a sufficiently convincing argument for their proposals.

If the minister had slightly more meetings with people from one side of an argument than another, is this going to be taken as evidence of unfair process?

And the bigger, more complex and more important the decision, the more often there seems to be a focus on processes.

This means the focus of the debate is increasingly on a set of grievances about process rather than on getting the outcome right and justifying (or not) the decision to the public.

So beyond a certain point, transparency serves to undermine democracy rather than promote good governance.

And when the disputes over these process issues are taken to the courts (eg airport expansion) then attempts to improve transparency in fact hand the issue to judges, thereby removing it from voters and elections.

In the short term, a sudden rush to transparency of government may well have negative outcomes (cynicism, unreasonable expectations) – we’ve seen the damage that the UK MPs expenses scandal has done to the public’s faith in that country.

However, over the long-term transparency is transformative of systems and processes, and ultimately of performance. I understand that this may be discomfiting to some people in the UK public sector, but it will no longer possible be possible for many organisations to carry on working a certain way once the public can see the detail of what they are doing. Specifically how much energy they are devoting to processes that serve the internal purpose of ‘being busy’ (I speak from experience of working in several public sector organisations), rather than ones that genuinely serve the end user.

Another important point is that in a world of increased transparency the general public will increasingly become more sophisticated in its analysis of what it sees, we can imagine a proliferation of newspaper articles and blogs giving guidance on interpretation of public data. In my view, the danger of rabid and misguided responses diminishes with greater release of performance data (and, as noted above, greater systemic changes driven by data release).

[…] Good and bad transparency Government is a fascinating study in unintended consequences. Its scale and diversity, and the interplay between ministers and the civil service, the centre and the frontline mean that things often don’t go to plan. Broadly-speaking, government is a risk-averse organisation in many ways, sometimes to the extent that on occasion it has preferred ineffectiveness to perceived impropriety, waste to uncertainty. The coming wave of transparency could transform this in a hugely positive way, using open data on costs, opportunities and performance to become a much more creative, cost-effective and agile institution, mindful of the money it spends and the results it achieves, and ensuring individuals are accountable for their work. […]

[…] government is serious in its aim to open up public data (though as @lesteph points out, this move towards radical transparency is not without its own attendant risks), so data releases are going to happen. The question is, are we going to help that data flow so […]