Charities are becoming increasingly sophisticated about how they gather evidence of impact - and there are many great guides and tools out there to help them do so. But there are other ways that the third...

Articles

A growing wariness between what is fact and what is fiction in recent years has been highlighted by increased political polarisation, a fractured media and the accelerated pace of technological change. The Evidence Initiative, the recently launched...

This week marks International Open Access Week 2018. At the Alliance for Useful Evidence, we’re big supporters of the principles behind the open access (OA) movement: scholarly research should be available to everyone, not just to...

Resources

How to learn the right lessons from Carillion

When government implements far-reaching policies whilst ignoring the evidence there can be catastrophic consequences, writes Geoff Mulgan (Nesta).

Private finance initiatives (PFIs) and public-private partnerships (PPPs) are in the spotlight again thanks to the collapse of Carillion. There are many dimensions to this. But I hope that at least one lesson is that this whole unhappy story is a classic example of what goes wrong when policy is made in an evidence-free way.

The underlying ideas behind public-private partnerships were promoted vigorously in the 1980s. For some, the motive was purely ideological: to dismantle the state. There were also other more pragmatic arguments, in particular the claim that, although capital would cost more to the private sector than the state, these schemes would bring new methods and skills into the public sector which didn’t have a good record of large capital projects.

That was at least a plausible hypothesis. But the policies were introduced without any trials, tests, experiments or evidence assessments. They were pushed through by fairly tight-knit cliques that straddled the civil service, ministers and a few big financial organisations (their revolving doors involving figures like John Redwood have been well-documented). Together they successfully resisted any attempts to evaluate the ideas.

PFIs were clearly bad value to the taxpayer; they didn’t transfer risk; and they involved high transaction costs and high costs of capital

It was already clear 20 years ago that many of the deals were very bad value for money, even as the New Labour government proposed to greatly expand the programmes. I was favourable to a more mixed economy of provision but always sceptical of the PFIs and PPPs I had seen close-up. When I was running the Strategy Unit (in 2001/2) I therefore commissioned a very senior figure from a big city firm, with extensive experience of the public sector, to do an honest review.

His review was scathing. He showed that a majority of the then PFIs were clearly bad value to the taxpayer; they didn’t transfer risk; and they involved high transaction costs and high costs of capital.

The review was circulated internally. The response was explosive. I was shouted at by both Gordon Brown and John Prescott, and various advisers of theirs, for having had the temerity to question such a crucial policy. Various threats then emanated from government about penalising the company where the author worked. PFIs and PPPs had – bizarrely – become an article of faith, not a potentially useful idea that needed to be tested.

Some of the recommendations of the report were quietly adopted, despite the denunciations, and helped to stop the worst deals. But plenty of bad practice continued, fuelled by a big machine of beneficiary organisations in consultancies and investment who raked off huge fees for setting up the deals.

This is a particularly bad, and costly, example of how policy shouldn’t be done. It was driven by ideology not evidence; bulldozed through without experiment and testing; and then kept out of the gaze of proper scrutiny, partly because it had cross-party support at the time and partly because of powerful supporters.

Returning everything to state management would be exactly the wrong lesson to learn from Carillion and other PPPs: a purely ideological mirror to the neoliberal arguments that produced the problems in the first place. The right lesson is that government should be pragmatic in finding the right mix of roles of government, business and third sector, and should be open, transparent and experimental as it navigates how to get the balance right. It’s not exactly rocket science.

Related resources

Geoff Mulgan, Chief Executive of Nesta, and David Halpern, What Works National Advisor make the case for an international platform for commissioning and funding research. Many governments – and people – around the world are wrestling...

Geoff Mulgan (Nesta) questions the popularity of the term ‘theory of change’ and suggests that it can encourage too simplistic an understanding of why change occurs. The phrase ‘theory of change’ is bandied around throughout the world...

Through the below parody, Ray Pawson, Professor of Social Research Methodology at Leeds University, explores the reality of the complexity of the science of evaluation, focusing on RCTs and their growing reputation as the ‘gold...

Join the Alliance for free

If you’re interested in the use of evidence in decision-making, join our network of 3,700+
individuals from across government, civil society and academia in the UK and
internationally. By joining you’ll: