The day of "The Great European Speech" finally arrives – and then disappears again. Due to the Algerian crisis, the prime minister has been forced to delay it. But that makes little difference: much of the content is already known, and it is no surprise. He would like to stay in Europe, but only if British demands are met. Earlier, Cameron had bottled out of his crude plan to speak next Tuesday, on the 50th anniversary of the Elysée treaty that founded the EU and celebrated an end to a century of three Franco-German wars. The ineptitude of his diplomacy has not helped his cause. Country after country has warned against demanding special British opt-outs instead of negotiating change together for all. The Finns put it best: pick currants out of the bun and he'll come away only with crumbs. Blackmail won't work, said others. How could it when Cameron stands at the exit door with a gun pointed at his own head? They'll shout "Shoot!".

His posturing provokes a rattling artillery of heavyweight opposition. Obama sent an emissary with a stern warning that the US needs Britain in Europe, or else the special relationship is a bridge to nowhere. That shot Liam Fox's fox, with his coterie advocating a virtual 51st state alternative to the EU. Nick Clegg and Vince Cable were bound to put the boot in, as were Ken Clarke and Michael Heseltine. But the CBI's kicking was less expected: "It's essential that we stay at the table to bang the drum for businesses and defend our national interest." Another knee to the groin came in a letter from 20 big beasts of business, including British Telecom, Sir Martin Sorrell and Sir Richard Branson, reprised this week in the FT. So who has Cameron pleased? Not even his own fruitcakes who want an in/out vote now: the Bruges Group damns his plan as "deliberately disingenuous". Which it is.

Cameron's rash promise of an undeliverable "new settlement" may give him a brief mini-bounce, but Ben Page of Ipsos Mori says Europe – at around 3% – is nowhere a prime public concern. If asked, people say they want referendums, though in five years it's unknowable what they would be voting on or why. Peter Kellner of YouGov delivers a devastating analysis showing the fear factor already swinging voters away from the exit. Cameron may be making a monumental blunder, looking weak and opportunistic at the helm of a riven party, instead of decisive.

To outsiders and to history his position will look intellectually incomprehensible. If he really wants to stay in, he risks voters using a mid-term 2018 referendum to take revenge, heading for the exit willy-nilly. Whatever the question, "no" will mean out. Anti-EU rhetoric sits oddly with Cameron's role as president of the G8 where he hopes to pull off an important EU-US trade deal that shows the value of operating as a block.

How odd, too, that he should pick on the justice and home affairs brief, just because it's easy to quit as it's up for review – for most people the European arrest warrant to catch criminals and terrorists is one of the best reasons for EU co-ordination. A Labour MP reminded Cameron that 39 paedophiles have been sent back to the UK for trial. What about "punching above our weight", a Conservative creed? We waste billions on Trident pointlessly patrolling the seas to cling to an anachronistic UN security council seat, yet once outside the EU that seat would soon be gone.

But most eccentric of all, why leave now when Europe is in the grip of German-led neoliberal austerity, a perfect political match for the policies of Cameron and George Osborne? That shows how the Tory dance on Europe takes them to another universe of fantasy politics. That's what the electorate in the real world, where jobs and growth determine their votes, will see.

Take just one industry: construction. This week the National Audit Office delivered a stern warning about the lack of public funds for construction in energy, waste and rail: relying on private money will see consumers paying needlessly high prices for decades ahead – those very debts to our children Cameron and Osborne warn against in justifying their cuts. Construction lost another 60,000 jobs last year, with 20% public housing cuts, road building cuts, and government promises of investment from foreign sovereign wealth or pension funds nowhere to be seen.

Simon Storer, of the Construction Products Association, speaks for the £50bn industry manufacturing bricks, tiles, aggregates and everything for building. Even the threat of leaving the EU fills him with alarm, at a time when government dithering on a south-east airport and nuclear power is already causing serious damage. His industries are heavy energy users, most foreign owned with no commitment to stay: Pilkington glass is Japanese, cement makers are French and Mexican, Worcester Bosch is German and so on. Would they stay if we left the EU? "Absolute madness to risk it," he says. Neither EU working rights nor planning restrictions hold back his companies: "It's money, pure and simple." State investment, not an EU exit, is needed to kick-start work – which could begin in a month if local authorities were handed cash to start road repairs now, leading to a virtuous circle of employment, taxes paid and benefits reduced.

I first met Storer at a fringe meeting on construction at the Conservative conference, holding his head in his hands as a rightwing ideologue on the panel said the answer to roads was to rip out cycle and bus lanes and the answer to overcrowded rail was to make everyone stand. This frequent clash between the real world and the Conservatives' imaginary world is increasingly dawning on voters. Labour's grip on reality will look all the stronger as long as Tory Europhobia fires burn. The shadow cabinet discussion on Europe this week should, said a member, have been YouTubed as an exemplary sight of a party expecting to be in power seriously analysing Britain's future role.

Labour has difficulties: refusing a referendum risks looking elitist and "Westminster knows best" in an anti-politician era. Immigration is feared. Of course the EU needs reform – along with a release from failed austerity squeezing the breath even out of German growth.

Ed Miliband held pretty firm this week, refusing a meaningless referendum. He left the door open a crack: in these crisis days, who knows the state of the EU five years hence. But his refusal to flirt with a British exit should see statesmanship better rewarded than blowing with the breeze of public opinion that doesn't care that much anyway.