He said he would “never stand against David Cameron” but managed to fan the flames by admitting he has regular talks with colleagues about the “long-term future of the party”.

He was spoken of widely last weekend as an MP favoured by backbenchers as a candidate to replace the Prime Minister.

This sparked one politician to claim there are actually 16 potential contenders, including several cabinet ministers, who are privately gathering supporters in preparation for future leadership bids.

These were named as Theresa May, the Home Secretary, Chris Grayling, the Justice Secretary, Owen Paterson, the Environment Secretary and George Osborne, the Chancellor.

Two former leadership contenders - Mr Fox and David Davis - were also the subject of speculation. On the backbenchers, Graham Brady, Jesse Norman and Steve Baker were tipped for higher office in future.

Loyalist Tory MPs last night dismissed the talk of replacing Mr Cameron, who is more popular than his party according to many polls.

Douglas Carswell, Conservative MP for Clacton and a frequent rebel over Europe, said it would be “daft to even contemplate a leadership contest”.

“Voting against the government because of differences of policy is honourable,” he wrote on his blog. “But briefing journalists about some dim-witted ‘plot’ because you didn't get the ministerial job you wanted is not."

The speculation about Mr Cameron’s job sparked off rumours that another contender could be looking to depose George Osborne as the Chancellor.

William Hague, the Foreign Secretary, had been named as a possible replacement for Mr Osborne if the Chancellor fails to turn the economy around.

However, Mr Hague told the BBC Sunday Politics that there would be no “job swap”.

“These things are up to the Prime Minister but if he was here, he would tell you very clearly there will be no such job swap. The Chancellor is doing a great job.”

"I’ve said I don’t want to be a politician in my 60s. I’m 51 now, that gives me a bit of mileage yet. But I did come back into politics to do this job, to support David Cameron and to do this job. So it’s always been my intention – and his intention thankfully – that I would be Foreign Secretary. We haven’t set a time limit on that.”