Written by James Wilsdon, Kieron Flanagan and Stian Westlake, The Guardian

The research community hasn’t had much to smile about in recent months. Faced with the grinding uncertainties of Brexit, concerns over university finances, and a root-and-branch reorganisation of the funding system, 2016 has been morphing into the sector’s own annus horribilis.

So when news broke over the weekend about a big boost for R&D funding, the first reaction of many was to raise a sceptical eyebrow. Experience says it’s wise to treat big, headline numbers with caution until there’s an opportunity to scour HM Treasury’s fine print.

Last time there was a significant boost in UK research spending was 2004, when Gordon Brown as chancellor unveiled his 10-year framework for science and innovation – and even that was not all it seemed. Not long after, the financial crisis hit, and a decade of austerity began, in which budgets have basically flatlined

When news broke of an extra £2 billion a year, it seemed likely that at least some of this new funding would be reheated from past announcements, or funnelled into business R&D tax credits and loans, which have a debatable impact on the “real” research and innovation economy. The PM’s speech to the CBI offered little in the way of further detail, and the journal Nature reflected the mood of many in its headline “Cautious welcome for UK government’s vague £2-billion research pledge.”

But with the full autumn statement in front of us, it’s clear that this really is new money. By 2020, government spending on R&D will grow to an additional £2bn over and above existing spending, an increase of about a fifth. The investment ramps up year-on-year to reach this level, and means a total boost of £4.7 billion by 2020-21. As HM Treasury points out, this is the largest increase in R&D investment in any Parliament since 1979.

The autumn statement documentexplains the two streams through which the new funding will flow: a new Industrial Strategy Challenge Fund (ISCF) for collaborative research between industry and academia, targeted at priority technologies; and a broader boost to UK capacity in research and innovation. Intriguingly, the document stresses grant funding – even by Innovate UK – when until recently government had been pushing Innovate UK to replace grants with soft loans.

The challenge-led approach of the ISCF is something that bodies like Nesta and many others have called for. It represents the culmination of a revival of interest in activist technology policy in the UK, which has been building since the financial crisis. Earlier steps in this direction were tentative and criticised for being sub-scale.

Historically, public spending on applied technological R&D began to fall in relative terms in the 1960s. In the 1980s, Margaret Thatcher believed strongly that it was important for the taxpayer to continue to support so-called “basic” research but that it was the role of industry, not government, to fund “applied” R&D. The problem was, outside a few key sectors, the bulk of UK industry took its cue from government, running down its own investment in R&D relative to international comparators. The UK’s technology base withered and no amount of effort to wring out innovations from basic research through commercialization and technology transfer could hope to compensate.

So it’s noteworthy that a Conservative government has looked afresh at international competitors, such as the US and Germany, and decisively rejected the Thatcherite distaste for large-scale technological R&D spending. Though of course, the driving force behind this shift is a defensive need to anticipate and prepare for the impacts of Brexit.

This week’s windfall is also a real credit to Greg Clark, the business secretary and Jo Johnson, minister for universities and science, and to the less visible but crucial influence of Sir John Kingman, who recently moved from the top job at HM Treasury to become the inaugural chair of UK Research and Innovation, the new mega-funding agency that is being created through the Higher Education and Research Bill. As one policymaker told us off the record yesterday: “This is classic John Kingman. Be as bold as you can and then double it.” It was Kingman who was the principal architect of Gordon Brown’s 10-year framework in 2004, and having steered HM Treasury through the bank bailout and quantitative easing in the decade since, he is perhaps the only Whitehall figure who could have delivered such an ambitious plan.

Beyond the headlines, there are plenty of details still to be worked out. How much of the new money will go to the research councils, and how much to the ISCF (through Innovate UK)? How will new challenges and technological priorities be defined, and with what mix of government, academic, disciplinary and user input?

The autumn statement promises an “evidence-based” approach – but by definition, there is no reliable evidence about technologies that don’t yet exist, so priority-setting for R&D is always a political process. However, it is one that the UK could do better, perhaps by adopting tools like Arloesiadur, the dashboard that the Welsh government has developed to understand its innovation strengths.

More importantly, will this extra investment be enough to offset the damaging uncertainties being created for the research community by Brexit? Some, such as Richard Jones, argued yesterday that the scale of new investment suggests the government has resigned itself to ending UK involvement in European R&D programmes after 2020. But officials at BEIS are adamant that this is not the case, and UK participation – in whatever form – remains a key item in the Article 50 negotiations to come.

Another uncertainty is whether any of this new money will come through the quality-related (QR) side of the UK’s funding system (allocated via the Research Excellence Framework). The higher education and research bill emphasises the importance of balance across the two pillars of the dual support system. As Graeme Reid, professor of research policy at UCL, told us yesterday: “With so much new investment, the government’s commitment to a balanced funding system is being tested. This is a chance for ministers to demonstrate that they are serious about preserving that balance.”

How the new ISCF is implemented will of course be key to its impact. We are told that the new fund will be modelled on DARPA, but it needs to draw the right lessons from the US defence research agency. DARPA funds groundbreaking, challenge-based research, and one reason it works is because it can fund the best research whether it takes place in universities or businesses. Formulaic approaches to funding, associated with research grants, sit uncomfortably with this risk-taking approach. So it is hard to see how UKRI will play a DARPA-like role given the cultures and practices it will inherit from its constituent funding bodies.

In any case perhaps DARPA is not what the UK really needs right now. We may need to walk before we can run, and a steady focus on rebuilding our technology base may be more appropriate than leaping into riskier waters.

On other fronts, the review of the Small Business Research Initiative (SBRI), announced on Monday, could help deliver the holy grail of innovation policy: using the government’s massive procurement budget to drive new technologies. To get this right, Whitehall will need fix the internal wiring of departments, joining up procurement and strategy functions. The review should kick-start an open debate about how best to achieve this, drawing on evaluations of the SBRI and comparable international initiatives. The Advanced Systems Agencies modelis one model worth exploring.

The review of “patient capital” and the government’s commitment of £400 million to co-invest in growth capital funds could also benefit high tech businesses. This co-investment model has a good track record in encouraging venture capital investment, both in the UK and elsewhere (for examples, through Israel’s Yozma programme).

Overall, this week’s announcements represent an unexpected and transformative statement of intent. There will always be voices calling for more, but the prospects for UKRI to deliver a step-change in the UK’s research and innovation performance now look genuinely exciting. And lingering opposition to UKRI, which was expected to intensify when the higher education and research bill heads to the House of Lords on 6 December, is now dead in the water. To prove the value of the new body, Jo Johnson simply needs to smile and utter the words “£4.7 billion”.