When it slowly dawned on David Cameron, as he watched TV footage of London burning in his Tuscan holiday villa, that he would have to put on a suit and come home, he must have spared a thought for the late Jim Callaghan.

Few PMs want to lose their longest annual break from the hamster wheel. Cameron has liked to demonstrate that he has steady nerves. Even the workaholic Gordon Brown won plaudits for resisting hyperactivity when his first summer at No 10 was punctuated in quick succession by terrorist attack, foot and mouth disease and floods in 2007.

But Cameron's attempts to play the hands-off chairman of the board – a deliberate contrast to New Labour's centralising instincts – have already backfired on NHS reform and much else. His inattention to detail is proving a weakness.

In any case, every prime minister for the past 30 years, caught far from home when a domestic crisis broke, also spares a thought for Old Labour's last prime minister who was attending a major summit in the West Indies in January 1979 when the "winter of discontent" – trade union strikes against government pay policy – came to a head.

Callaghan's fate haunted them all, as it must have done Cameron this week. There was no overwhelming reason for the PM, battered by the phone hacking affair, to come home because the financial markets had panicked over the twin debt crises in the eurozone and the United States.

For once Britain's credit-worthiness was not in the firing line and not even his chancellor, George Osborne, was abandoning his holiday in Los Angeles. A misjudged dash back to No 10 might have simply served to further spook the already jittery markets. Only in mid-afternoon Downing St officials were saying the PM was being kept well enough informed by colleagues in Whitehall. EU colleagues stayed on their beaches and mountains too.

But three successive nights of riot, looting and arson on the streets of the capital, spreading beyond the M25 too? That had the makings of a perfect storm. At stake is more than law and order, a policy where coalition ministers are suspected by many supporters of being tougher on police budgets than they are on criminals.

If herd-like global financiers, watching 24/7 TV screens from their own holiday villas, decide that London is not as safe a city in which to invest, work or own expensive property – that props up the capital's overheated housing market – then sterling, the FTSE index and sale of Treasury gilts might feel the kind of heat no fire brigade can extinguish. And then there are next summer's Olympics: aren't they being staged within this week's battle zone.

Shortly before Cameron's decision was announced (Ed Miliband was not far behind) the Met's acting commissioner, Tim Godwin, had finally made a direct appeal for calm on TV after retired senior officers had started complaining on air about a lack of "strategic leadership" at Scotland Yard. Ministers must share some of the blame for that too.

Theresa May, the home secretary, had already abandoned her holiday. London's hedonistic mayor, Boris Johnson, who is also Cameron's unofficial stalker, had also decided to come home, possibly rattled by Ken (where exactly is he?) Livingstone's day of brazen attacks on government spending cuts and Johnson's campaign for tax cuts for the rich. Who in this "cabinet of millionaires" speaks for ordinary Londoners, asked Livingstone whose condemnation of the rioters may not have persuaded voters that he was using poverty to justify nihilistic acts of mayhem.

In Britain's quasi-presidential system none speaks with the authority of the prime minister. So Cameron will be visible before and after chairing the emergency meeting of Cobra – it is a room, not a snake – where all varieties of crises are managed by multi-disciplinary teams. He will not risk Callaghan's fate.

Back in 1979 the then-PM came back from his summit, weighed down by huge global problems, and told reporters at Heathrow "I don't think other people in the world would share the view that there is mounting chaos."

The Sun, then in the early stages of its predatory career as political assassin, famously translated that as " Crisis, what crisis?"