In defence of the Research Excellence Framework

An as-yet-unannounced review of Business, Innovation and Skills-funded bodies raises a potential threat to the future of research funding. We need an intelligent, evidence-informed debate about the costs and benefits of assessment

Sajid Javid, secretary of state for business, innovation and skills, has called in the suits from McKinsey to carry out an “efficiency and effectiveness review” of all BIS-funded bodies, including the research councils and HEFCE.
Photograph: Wiktor Dabkowski/DPA

In academic circles, it’s become fashionable to denounce the Research Excellence Framework (REF) as the sector’s very own spawn of satan. The REF, a six-yearly assessment of the qualities and impacts of UK research, is used to allocate around £1.6 billion of public funding on an annual basis. In these austere times, you might think that would be enough to win it a few friends. Instead, the critics are lining up.

I’m prompted to write this by the full-throated attack on the REF launched last week by the free market think tank, the Institute of Economic Affairs (IEA). In a five-page briefing, the IEA calls for the REF to be abolished because it is too costly, distorts research priorities and can be gamed.

Instead the IEA wants quality-related (QR) funding, which is currently allocated through the REF, to be moved to the research councils. But for an economics outfit, the paper is surprisingly sloppy – it doesn’t even annualize these costs to reflect their spread across an entire assessment cycle. The £246 million full cost of the REF sounds like a lot if you set it against one year of QR funding. But across a six-year period, this is equivalent to 2.4% of the £10.2 billion of QR funding that is set to be allocated (or around £41 million per year).

If the REF were scrapped tomorrow, many of the evaluation and audit functions that have grown up around it would continue, as part of wider university management of research. Pressures for quality, impact, accountability and institutional benchmarking wouldn’t disappear. Indeed, one could argue that the REF acts as a bulwark against more harmful currents washing through higher education and research, such as university rankings and league tables.

So to make an evidence-informed judgement about the costs and benefits of the REF, we first need to compare it to the other side of the dual support system. We also need to reflect on the potential consequences of severe reductions or total abolition of QR funding, which would starve entire research fields (particularly in the social sciences, arts and humanities) of the long-term investment they need, and hit early career researchers the hardest.

In the 2015 spending review, the research community isn’t being offered a return ticket to some (mostly mythical) golden age in which public funding was doled out, without any demands for accountability. Instead, our already stressed funding system is facing another five years of flat cash – or worse – as the Department for Business, Innovation and Skills (BIS) grapples with George Osborne’s goal of up to 40 per cent more cuts. As Paul Jump, one of the most astute commentators on UK research policy, wrote recently in the Times Higher:

I sometimes ask opponents of supposedly neoliberal excrescences such as the impact agenda and the research excellence framework how large a cut in the research budget they would be prepared to take in order to be rid of them....[It] does strike me as a very safe bet that research spending would be considerably lower – perhaps much more than 25 per cent lower - than it currently is if the academy were unable to demonstrate its quality and impact in terms that politicians and officials, for better or worse, can understand.

George Osborne remains broadly supportive of science, but BIS’s role as a champion of the research system can no longer be assumed. In-year cuts of £150 million to HEFCE’s budget, announced last week, are a worrying signal of what may lie ahead in the autumn spending review.

Beyond the spending review itself, which got underway on 21 July, the Political Science blog can exclusively reveal that Sajid Javid has recently commissioned the consultancy McKinsey to undertake an “efficiency and effectiveness review” of all BIS-funded bodies, including the research councils, HEFCE and Innovate UK.

This review, details of which have not been publicly announced, is clearly intended to shape BIS’s spending review submission to HM Treasury. Sajid Javid was apparently dismayed by the number of bodies that BIS is funding, so turned to McKinsey, which prides itself on a radical approach to cost cutting.

The terms of reference for the McKinsey review, the evidence it will be based on, and its relationship to more public processes like the Nurse Review are unclear. A call to the BIS press office elicited no comment. My concern is that developments like this pose a further threat to QR funding, given the inherent bias inside HM Treasury against block funding allocations, particularly when expert peer review, rather than Whitehall edicts, control how the money is being spent.

I hope my fears are misplaced. The IEA says that research assessment promotes game playing. But next time you feel inclined to launch a broadside against the REF, do us all a favour, and bear in mind whose game you may in fact be playing.