In 2008 prizewinning environmentalist author Mark Lynas experienced a "eureka moment". Reading the hostile comments underneath an article outlining his objections to GM foods on the Guardian website, he decided his critics were probably right.

A couple of years later, Lynas had another eureka moment when he read Stewart Brand's book, Whole Earth Discipline, in which the American writer tore up the green rulebook and came out in favour of urbanisation, nuclear power and genetic engineering. A few months ago, Lynas appeared in a TV documentary, What the Green Movement Got Wrong, alongside Brand – and inside the ruins of Chernobyl which, he argued, had not been nearly as devastating a disaster as most people think.

Next week Lynas publishes a new book, The God Species: How the Planet Can Survive the Age of Humans, in which he takes his argument with the green movement a step further. The book accuses the greens of having helped cause climate change through their opposition to nuclear power, and calls this a "gargantuan error, and one that will echo down the ages".

"Anyone who still marches against nuclear today," he writes, "as many thousands of people did in Germany following the Fukushima accident, is in my view just as bad for the climate as textbook eco-villains like the big oil companies."

The idea for Lynas's new book came to him in another "moment of revelation" two years ago. Lynas, who is a part-time climate adviser to the Maldives government (he is also a visiting researcher at Oxford university), was invited to sit in on the meetings of a group of scientists in Sweden. The group were aiming to flesh out the concept of "planetary boundaries", coined by sustainability expert Johan Rockström.

The best-known of these so-called boundaries is the climate-change one – the amount of carbon dioxide in the atmosphere. But there are boundaries too for biodiversity, nitrogen, and ocean acidification. The idea is that, beyond these limits, Earth's systems will begin to break down.

Lynas's revelation was that these new rules about how to live on Earth should immediately replace many older green ideas, and over drinks he and Rockström agreed that Lynas would write a book with the aim of popularising them. But the most attention-grabbing passages in the book come in Lynas's denunciations of the green movement, and when we talk he makes no attempt to play them down. Instead he draws my attention to his blog, where over the past fortnight he has enthusiastically joined in attacks on a recent IPCC (Intergovernmental Panel on Climate Change) report on renewable energy. And he argues that "the green movement in itself is dying – I'm an environmentalist but not a green".

Lynas, who describes himself as a "recovering activist", was involved in direct action in his student days. He joined protests against the Newbury bypass and Manchester airport, and was heavily involved in the anti-GM movement of the 1990s, ripping up sweetcorn and sugarbeet crops from fields in East Anglia, and on occasion being chased by police and police dogs.

But is he a maverick iconoclast, stirring up controversy for the media by turning on his old allies? Or are the views expressed in his book symptomatic of broader divisions?

Eighteen months after the failure of the Copenhagen climate summit, there are signs of wider frustration. With no sign of progress in setting global emissions limits, a steady stream of reports gives cause for alarm to those who are already worried. Last week it was the turn of the oceans, with a warning about pollution and overfishing, last month a sudden upsurge in Amazon deforestation. This week climate sceptic Michele Bachmann launched her bid to become the next US president, while the EU was forced to put off a vote toughening emissions targets following reports that Tory MEPs were planning to reject it.

"People think that getting some publicity, having some tea with a minister and civil servants, lobbying parliamentarians, is making a difference, but it's not," says Charles Secrett, the former Friends of the Earth director who two weeks ago wrote an article accusing the organisation of being bureaucratic and out of touch. "Protest ain't going to win the day. Nor is a sort of incremental engagement with government and industry. The movement as a whole has got to collaborate more, pool resources – money, staff, ideas – and generate real cross-party pressure."

Novelist Ian McEwan spent years researching renewable energy for his 2010 novel, Solar, and says when he began "there was a positive mood for action, a public awakening. Now I think everyone has fallen back to sleep. Copenhagen was something of a fiasco, and the UEA emails didn't help. And the ideological deniers are well organised. At this point I don't see change coming from a bottom-up process, from a kind of peasants' revolt. I think the consumer moment has passed and people have got bored."

This feeling of a missed opportunity, and of 2009 as a high-water mark in public engagement with the issues, finds many echoes. Though activists trumpet their recent successes in having seen off the third runway at Heathrow and a new fleet of coal-fired power stations, as well as helping persuade David Cameron to commit the UK to a strict timetable for cutting emissions, they admit that disappointment after Copenhagen, and uncertainty about the future, have been difficult to manage.

Tamsin Omond of direct action group Climate Rush remembers this is a heady time. "2009 was the year we said we would do one action a month, and we did. Everyone saw this as the one chance and the feeling of momentum – that we only had to work really hard until December, and then we could have a rest – was really present. Everything we did would get in the papers and journalists were phoning up all the time. I was completely caught up in it."

I was caught up in it myself: in 2009 I joined the Green party and stood as a candidate in a council byelection a few months before Caroline Lucas was elected Britain's first Green MP. It was the year Age of Stupid director Franny Armstrong had the idea for 10:10, on her way to a debate with Ed Miliband, and launched the campaign at Tate Modern and in a special issue of G2. And it was the year newspapers around the world, led by the Guardian in an unprecedented gesture of editorial solidarity, printed the same leading article demanding action on global warming on their front pages.

Post-Copenhagen, consensus is harder to find. The recent ructions boil down to three issues. The first is nuclear power, with Guardian columnist George Monbiot, former Greenpeace director Stephen Tindale and McEwan among those to agree with Lynas that atomic energy is vital if we are to wean the world off fossil fuels.

Another disagreement is summed up by Charles Secrett's complaints about Friends of the Earth. Some activists believe that the big, long-established NGOs need to get better at mobilising their supporters and achieving a greater degree of focus and coordination, as well as building up links with nimbler and more dynamic direct-action campaigns.

But the biggest issue of all is the nature of environmental politics. Is the green movement a leftwing, anti-capitalist movement? Mark Lynas believes it is, and that those who style themselves as greens should be marginalised and allowed to die off so that they can be replaced by a new breed of market-friendly environmentalists like him. "If it becomes a culture war like the debate over abortion or something, you can't win," he says. "I want an environmental movement that is happy with capitalism, which goes out there and says yes rather than no, and is rigorous about the way it treats science. The green movement needs a clause-four moment – the Labour party had to go through that."

Those within the green mainstream reject this analysis outright. Jonathon Porritt argues that social justice is intrinsic to the sustainability agenda, while Greenpeace director John Sauven points out that the charity has worked closely with all the main political parties in Britain and with multinational corporations abroad. "It's a very broad camp, isn't it? On the one hand you've got the anti-capitalists, and then you've got quite a strong body within the Conservative party that takes the environmental agenda very seriously – John Gummer's quality of life report was an excellent piece of work."

He believes Lynas over-eggs the nuclear point, and that the power of the economic and political interests aligned against change – above all the US fossil fuel lobby – must be understood. Others point out that there is already a strong emphasis on green growth and development, and the economic opportunity represented by the new industrial revolution that we need to carry us into a post-carbon world.

But Lynas is not alone in believing that the intense focus on aviation has been offputting, and there is general agreement that Britain must learn from the US, where many Tea Party supporters believe climate science is a socialist conspiracy. This week energy minister Greg Barker suggested that a debate started by Margaret Thatcher had been hijacked by the centre left.

Campaigners cite the Heathrow and forestry protests as examples of what a broader coalition of interests can achieve if they go about it in the right way. Climate Rush's Tamsin Omond coordinated a "Saving the Forests" letter to the Daily Telegraph with The Lady editor Rachel Johnson, and says: "If we haven't been good enough at appealing to people across the board then we are missing a trick. We are all on the same planet, we have a ballooning population, diminishing resources and a changing climate, and we really need to grow up and see the situation for what it is. You can say these things to people who have never voted anything other than Tory. I have said them and I don't think it's impossible at all. We need to be talking to everyone."

McEwan says the green movement is not to blame if climate change has slipped down the agenda. "I think it's got a lot to do with human nature. Most issues have a narrative, with the sense of an ending or resolution – the referendum is passed, the government falls – but this really is a lifetime story, and not just our lifetime, but our children's and their children's. We are decades away from the point where we say, 'We've finally deflected the rising curve of Co2 emissions, so let's have one last push to fix it for good.' We've made no impact on this rising curve as yet, and it's hard to keep interest and optimism alive."

And he adds: "I've never voted for the Tories, but I'd make my judgments at the next general election based entirely on the respective parties' attitudes and intentions in matters of climate change. This is the overwhelming issue that encloses all others. If Cameron and friends came up with a more feasible and effective plan than Miliband, then I would have to vote for it. I think that's all we, as citizens, can do."