Looking for legacy, that's the game now. How do you bottle the butterfly moment, siphon off the spirit? Can we at least prove the Olympics made money, changed lives in east London or turned couch potatoes into runners? According to research on previous Olympics, probably not. But that misses the point. Glorious, epic, expensive, feelgood fun is how people sometimes blow their money on parties, holidays or celebrations, so why shouldn't a nation do likewise?

But holidays end. Frenetic moods come and go, from the response to last summer's riots to Diana's death. Reversion to the norm is a mighty powerful force. Politicians trying too hard to capitalise or make political meaning out of that fleeting surge of happiness should tread warily. The Tories counted on a double bounce from the Jubilee-Olympic summer, but the polls won't budge. Feeling good isn't the same as feeling good about the government.

Boris Johnson as buffoon was a one-man Mexican wave of dancing dressage, but I doubt it brought him any closer to real power. Nick Clegg was booed in the hockey. David Cameron was booed in the boxing – not helped by his MP's "leftie multicultural crap" tweet on the opening ceremony. Cameron's attempt at turning his own party's dreadful record on cutting school sports into an attack on Labour backfired badly. No sooner had he sneered at "Indian dancing" as a PE activity than hey presto, a flotilla of exquisite Indian dancers floated across the closing ceremony.

Cameron did badly because this was no moment for crude political point-scoring – let alone plain untruth. He derided Labour's record, but from 1997 children doing two hours a week of sport minimum rose from 25% to 90%, over half doing much more. The national mood was for praising success, so he hit a bad note in blaming teachers for failing to supervise more sport. His insistence on traditional team games reeked of his Flashman background, badly out of touch.

What of most children who never make a school team, humiliated by never being picked, begging their mums for "off games" notes? Modern PE has something for everyone: Indian dancing could become a lifelong fitness habit. Announcing a small sum extra for school sports, Cameron disguised his 69% cut that has caused a 60% reduction in school sport hours. Children's free swimming might have delivered future Rebecca Adlingtons, but Cameron axed that: 350,000 fewer swim at least once a week. "There's a danger of thinking all you need is money and a target," said the PM. That's no danger, it's often a recipe for success – in the NHS, in schools and in sport. He would have done better just to applaud: the political lessons of the games all tell against him.

Labour wisely avoided crowing too directly over Olympic golds, for fear of undermining GB athletes' sheer grit. But Britain was 36th in the medal tables in Atlanta in 1996 – and after Labour's £600m investment came fourth in Beijing, and now third. With rather more subtlety, Ed Miliband drew his own parables. "Britain's success this week did not happen by chance", he wrote in the Sunday Telegraph, leaving the implication but no boast. "We have had a glimpse of the country we can be. Our duty is to do everything we can to keep that spirit alive. A country united, confident, proud and welcoming to the world. Some will say it can't be done … I know it can. That is up to all of us."

The shaping of his future theme is there – look what we can do together, a collective endeavour, a public effort. He didn't say it, but for all their shameless demands, those sponsors only paid 6%, the public paid the rest. You can see the outlines of Labour's campaign: imagine what we could do to revive the desperate state of the economy. Jobs, growth, building, investing, firing up all that scientific, technical and artistic genius that brought all this together. Of course we can do better. Optimism that we have the power to shape our own economic and social destiny is Labour's to seize.

For there is no optimism anywhere else. Cameron and Clegg face forces pulling them apart. This conference season will find angrier opponents of the coalition in both parties. Today's Guardian poll showed only 16% of people expect the coalition to hold for much longer. Clegg's restive party wants revenge for the loss of both Lords reform and voting reform. Yet what should matter to them far more is the worsening state of the economy. Mervyn King has just delivered a more dire judgment than any before, of zero growth this year – far lower than expected over the next two years. Bank lending has seized up, exports are down, the balance of payments is the worst for 15 years. Meanwhile the Trussell Trust is opening four new food banks a week.

With David Cameron and George Osborne lashed to a failed Plan A and no sign of shifting, lashed to a failed Plan A, the one credible reason for the Lib Dems to break the coalition is to save the country from yet worse damage. Given what Clegg has led his party to vote for – benefit cuts for the poor, tax cuts for the rich – it is almost too late. But for each recession month that they stay on, tolerating all this, the Lib Dems lose credible reasons for ever making the break.

Reshuffles make scant difference to the public: only Osborne's head on a platter would signify, but Cameron will not reverse his deficit policy. Instead it's rumoured he will bend rightwards and bring in his angry brigade to swap for Ken Clarke and George Young, his sensible tendency. He might as well try to head off the Ukip threat now he has abandoned all disguise as a detoxifying moderniser. But he will struggle to find any useful Olympic metaphors that shine a kindly light on harsh austerity.

• This article was amended on 14 August 2012. The original referred to the Trussle Trust, rather than the Trussell Trust, opening four new food banks a week. This has been corrected.