In the space of a few short hours on the night of Thursday 4 June David Miliband held the fate of Gordon Brown in his hands. When James Purnell, an old friend from the early years of Tony Blair's Downing Street, resigned from the cabinet, Miliband could have provoked a bloodbath in the Labour party if he had followed his example. But in a brief phone conversation with Lord Mandelson, who was in No 10 discussing the imminent cabinet reshuffle, Miliband confirmed he would be staying.

A week after the late night call, Miliband laughs at the irony that a man who became Brown's great enemy, Peter Mandelson, was asking him to save the PM.

"The government is much stronger for Peter at the heart of it," the foreign secretary says as he recalls his conversation with Mandelson, who was awarded the title of first secretary of state after saving the Brown premiership. "I'm not going to go into [our conversation], but we didn't sort of talk about the weather."

But he confirms for the first time that he did seriously consider resigning from the cabinet. "I'd made my decision on Thursday," he tells the Guardian, saying he tried to persuade Purnell not to resign on the night of the local and European parliamentary elections.

"Sometimes you can make your decisions with great planning and calculation and sometimes you have to make them rather more quickly."

While Miliband and Purnell made different choices – he admits those on either side of the divide feel "passionately" – the foreign secretary still has the greatest respect for his old friend, who was strongly criticised by No 10 for notifying Brown minutes before his resignation was announced on TV. "Look, James is a good friend, he has good talent. He was a very very good minister. It is important to say that. He had the respect of his staff, he had good ideas for the future, he was good on detail and was a very good colleague. He contributed to cabinet. He is a loss. But he's not dead. He can contribute. Whether you are inside the government or outside, we've got to make a go of this party of ours."

Liberated

Such views will not be shared by Brown, who believes Purnell behaved in a discourteous manner. The prime minister is also wary of Miliband, though No 10 breathed a collective sigh of relief when he announced he would be staying.

The foreign secretary makes clear he too has changed: he feels liberated, as he knows he is virtually unsackable. The ex-head of Blair's policy unit, who has assumed a lower domestic profile since his call in the Guardian last summer for a "radical new phase" of New Labour and his unfortunate appearance with a banana at the Labour conference, shows all the enthusiasm of a former thinktank warrior as he makes clear he will spend the next months outlining a vision to save his party.

This will involve uncomfortable truths about the challenges and a bleak assessment of the party's failings. "We have all had a very, very cold, electoral shower. You have to take very, very seriously the scale of the rebuff...My generation in the Labour party know what a privilege it is to be in government, because we saw enough of opposition and we saw how many good people it destroyed to know the utter futility. We also know from the seats we represent that my generation cannot afford to let that happen again."

Sitting at a large wooden table in the foreign secretary's grand room in Whitehall, Miliband says he has thought very carefully about a "route map" for Labour. Looking fresh after a three-hour "political" cabinet meeting in No 10, discussing public service reform, Miliband says Labour must pass what he calls three tests.

"I was thinking really hard about this overnight – what is the route map?" he says as he outlines the tests Labour must pass: providing a period of competent and stable government; resetting its political compass; ensuring it is at the "cutting edge" of policy making.

The challenge is formidable because Labour has woefully failed voters. "We did very badly and we have to take collective responsibility for that." The party that once embodied the values of Middle Britain has lost touch with its former supporters to such an extent that nobody knows what it stands for. "I know what our instincts are, but the voters don't."

But Labour has not sunk to the low of the Tories in the mid-1990s. "John Major was down to the cones hotline at his conference in 1996. We are not down to the cones hotline. Our curse is not to have a lack of policy. If we have any curse it is that we have more policy than we know what to do with. What we have got to do is decide what are the high-impact policies for the new political landscape."

Miliband's remarks show the essential dilemma facing young supporters of Blair who believe the intellectual argument is on the side of the centre-left. Today's challenges – climate change, terrorism and the worst recession since the war – cry out for an active state, they say. But Blairites believe it has been all but impossible to articulate this with a PM who struggles to express himself in a warm and human way. Having made his decision to stay, Miliband now praises Brown for being the right man to lead during the economic crisis.

Choosing his language with care, to avoid talk of the green shoots of recovery, he indicates the worst of the recession may soon be over. "Let us just remember that if this does turn out to be one of the industrialised [world's] shortest recessions, shallowest recessions, fastest recoveries out of recession, that is a remarkable achievement. That is not about minimising the depths of the recession. It is about building a different sort of economy out of it."

Amid this background, Labour must prepare for an election that will challenge to all the parties. "The next general election will be the first of the global age. There is a new political landscape, people feel more free, but less secure; people feel there is a greater need for social justice, for economic and social stability. But they know it cannot be achieved in one nation."

Labour will have to deliver fresh ideas on crime, antisocial behaviour, housing and public services. There will also be a renewed push on creating a mainly elected upper house in parliament, and on electoral reform, as Labour responds to the expenses crisis by embracing ideas that have been gathering momentum.

"Why should not every MP in their constituency be able to say 50% in my constituency voted for me? That is what AV [alternative vote] does. Now that is not going to be for the next election. But it is part of the political class being less comfortable about saying, 'things have always been like this and let us leave it'."

Vacuum

Miliband believes the Tories are vulnerable as they opposed the economic rescue plan and appear to be preparing the ground for cuts in public services. "Every single action we have taken they have opposed. It is not just that they have got a vacuum elsewhere. On the defining crisis of this time, they have been profoundly wrong."

Miliband outlined his vision for the future of Britain during a rare week spent entirely in London. While he feels optimistic that Labour can again dominate the political landscape, he was shocked to the core by the election success of the British National party. This has resonance for his family, which lost 80 members in the Holocaust. Miliband will have the BNP's success in mind later this month he will pay a private visit to Auschwitz, the Nazi death camp in German-occupied Poland where many of his family perished.

"I am fortunate in that it is my parents' generation that encountered fascism, not me. But when I say [the BNP] are the descendants of the people we fought in the 1940s, I am thinking of my dad and my mum and my relatives."