The UK has had to increase coal-fired electricity production to pay for the cost of generating highly subsidised renewables.
Photograph: Murdo MacLeod

Climate change and public sector debt are two of the biggest issues for modern governments and they come together in the vexed question of how far the state should intervene in energy markets.

According to the International Monetary Fund, energy subsidies accounted for 8% of total global government spending in 2011 ($1.9tn, or £1.2tn), due mainly to the political appeal of keeping energy prices down.

The Labour party has certainly taken an interventionist view. At last September’s Labour party conference, then-party leader Ed Miliband said a Labour government would freeze prices, break up the big six energy suppliers and scrap Ofgem. But some commentators see high prices as the consequence of a failed energy policy.

Successive governments have micro-managed the electricity market since 2008 with a host of mechanisms to boost renewable energy and decarbonise the power market by 2030. These culminated in the electricity market reforms in the 2013 Energy Act, steered by former Lib Dem energy secretary Ed Davey, that were designed to guarantee a long-term price for low-carbon generation, including renewables, nuclear and carbon capture and storage, and protect them from the vagaries of the wholesale electricity market.

With the Conservatives’ general election victory, the picture has changed. With a majority government and a Tory energy secretary, the Conservatives could take a less interventionist approach. The government has already said it will end subsidies for onshore wind power and rely on the market to bring down prices.

But is less intervention better?

The case against intervention

Dieter Helm, a professor of energy policy at the University of Oxford and author of several books on energy policy and climate change, says the government was right to set a decarbonisation target. But he says the slew of mechanisms that were introduced to promote uptake of renewable technologies has increased consumer bills by 32%, failed to lower CO2 emissions and led to the tightest supply margins of the past quarter century, which has raised costs further.

Helm says the policy was based on a faulty assumption that fossil fuel prices would carry on rising, which would drive down, over time, the relative cost of the renewables technologies. “[Former energy secretary] Ed Davey thought he was protecting us from high and volatile gas prices,” comments Helm. “That’s why he thought building things like offshore wind was a good idea.”

Big six energy suppliers could be broken up after Ofgem triggers full investigation

Read more

Instead, as global energy prices fell, prices for coal fell faster than gas, which has half the CO2 emissions. This paradoxically led the UK to increase coal-fired electricity production to pay for the cost of generating highly subsidised renewables – instead of investing in cleaner gas-powered capacity. A similar situation happened in Germany, which has also intervened heavily with subsidies for renewables technologies.

“The result is our emissions are higher than they would otherwise have been, our prices are much higher than they would otherwise have been and we are no more secure,” says Helm. “For someone like me, who really cares about climate change, this is not a good way to go.”

Helm believes that when the state acts as a central buyer of electricity it is taking a view on future prices and picking technologies to back with subsidies. He highlights the proposed tidal lagoon in Swansea Bay as an example. “Very, very expensive technology has been chosen, with very limited benefits in terms of carbon, or competitiveness or security or whatever else.”

In his book The Carbon Crunch, Helm argues that government should have a technology-neutral approach to new renewables generation capacity; set a carbon floor price that meets the carbon budget constraints and makes coal pay for the pollution it causes; and have a far bigger budget for research and development in potentially disruptive technologies such as graphene and next-generation solar.

The case for intervention

Investigation is victory for consumer power over big six energy companies

Read more

Simon Skillings, author of a briefing paper (pdf) on electricity market reform for environmental thinktank E3G, says the government cannot just leave energy to market forces, as some critics suggest, or it will risk failing to meet the legally binding commitment to cut 80% of CO2 emissions by 2050. And nor are prices for CO2 emissions on the EU Emissions Trading Scheme anywhere near high enough to stimulate necessary investment in low-carbon technologies.

“You can’t suddenly turn a tap on in the early 2020s and expect a huge amount of renewables at scale,” argues Skillings. “You need to build up a supply chain to deliver that, and you have to start doing it now.”

According to Skillings, subsidising “out-of-market” technologies such as nuclear and renewables will be more expensive than gas or coal, but only in the short run, and the new government should focus on cutting consumer bills by integrating the UK’s energy market with other EU countries and deploying smart technologies to reap cost benefits from energy efficiency.

A similar argument in favour of intervention comes, not surprisingly, from former energy minister Davey, who in March told a debate on energy policy held by thinktank Policy Exchange that, if anything, governments should be more involved, to deal with decarbonisation.

“Given that we’ve built our economy and our society over the last 200 years on fossil fuels and we’ve got to get out of at least fossil fuel pollution in the next 30 or 40 years, we’ve got to act pretty quickly,” he said. Davey pointed to evidence that the coalition’s energy policies were working, citing a 165% increase in renewable energy since 2010 and falling prices for solar and onshore wind.

The new energy secretary, Amber Rudd, is known to be a strong believer in business and markets as drivers of change, rather than regulation and government intervention. In May, Guardian environment editor Damian Carrington noted that the balance between those two approaches is at the heart of arguments about how to fight climate change and also how to ensure the UK’s energy system is simultaneously low-carbon, affordable and secure. In the year of the Paris summit, the moment when the world must make tough decisions in order to start the process of keeping the rise in global temperatures below 2C, Britain needs to decide where it stands on energy intervention.

Talk to us on Twitter via @Guardianpublic and sign up for your free weekly Guardian Public Leaders newsletter with news and analysis sent direct to you every Thursday.