Nick Clegg's hopes of reforming the House of Lords, completing a journey begun a century ago by the deputy prime minister's Liberal predecessors, ran into severe trouble on Tuesday when 91 Conservative MPs defied a three line whip to vote against the measure in the largest rebellion of the parliament.

A furious David Cameron confronted the leader of the rebellion just outside the House of Commons division lobbies late on Tuesday night as it became clear that normally loyal Tory MPs were determined to register their opposition to House of Lords reform.

One Tory described the behaviour of the prime minister, who raised his voice to the rebel leader Jesse Norman in front of colleagues, as "disgraceful" and a return to his "Flashman character". In a sign of the PM's anger with Norman, four whips sought Norman out in the Strangers' Bar minutes later and asked him to leave the parliamentary estate. One rebel said: "Jesse is a good man and so he did leave." The rebellion by the 91 Tory MPs failed to block the second reading of the bill which was passed after Ed Miliband instructed his MPs to give their backing. The bill was given a second reading by 462 votes to 124, a majority of 338, in one of the largest votes of the parliament.

But the bill – which would ensure that 80% of the members of a slimmed-down chamber are elected by 2025 – will face a bumpy ride after the Tory rebellion and after the government was earlier forced to withdraw a "programme motion" for the bill. An alliance of Labour MPs and Tory rebels prompted David Cameron to conclude he had no option other than to abandon a vote on a "programme motion" that would have set a timetable for the bill.

The move effectively stopped the bill in its tracks because the programme motion contained a provision to send it to its next stage – consideration on the floor of the Commons at committee stage.

A senior government source said: "This was a tactical withdrawal to avoid an operational defeat."

The strength of feeling was highlighted when the Tory MP Conor Burns resigned as a ministerial aide to allow him to vote against the bill at second reading. Angie Bray was later sacked as a ministerial aide to the cabinet office minister Francis Maude.

The coalition is now entering one of its most difficult phases as Tory MPs question the prime minister's authority. A central tactic by Downing Street – to delay a ministerial reshuffle to persuade aspiring MPs to support the government – backfired as loyalists joined the rebels who numbered close to 100. "There was strength in numbers," one senior MP said. "But they were brave."

Downing Street insisted that Cameron, who persuaded Clegg to allow him to withdraw the programme motion to give him more time to try to bring Tory MPs onside, believes the bill can be saved. But Clegg and Cameron, who held a series of tense meetings through the day, face a formidable challenge when they try to revive the bill in the autumn.

Government sources say ministers will seek to table an allocation of time motion, following the practice of the last Labour government to impose a time limit on debates.

But Miliband and the Tory rebels insisted they would stand by their respective positions. The Tory rebels are opposed to the bill and Miliband rejects out of hand any timetabling for it.

Norman, the rebel Tory ringleader, told Radio 4's PM programme: "The bill is a dead duck. The question is how long will the government go on before it recognises that and how much further will it have to go in putting the country through a lot of additional pain when the real energies of parliament and the government should be focused on fixing the howling economic gale that we are now in."

Lib Dems hope the new motion will represent a challenge to Miliband. An allocation of time motion can be amended, unlike the programme motion dropped on Tuesday. "Ed Miliband will have to answer the question he is refusing to be drawn on: for how many days should the bill be debated," a Lib Dem source said.

Clegg had told Cameron the Lib Dems would have to review their support for his plans to reduce the size of the House of Commons if the programme motion was defeated. David Laws, former Lib Dem chief secretary to the Treasury, said defeat for the programme motion would have "set off a chain reaction". Laws said: "In coalition you sign up to a deal and it's not a kind of pick and mix combination ... A deal is a deal and we do have to honour these agreements that we made to each other."

Downing Street said that Cameron's decision to withdraw the programme motion had avoided a major crisis in the coalition. "In the end David Cameron has boxed clever on this. He has averted the car smashing into the wall. We will do our level best to take the heat out of this."

The coalition plans to intensify pressure on Miliband, who is a supporter of Lords reform but says it is wrong to impose a timetable on debating such a major constitutional issue. Lord Oakeshott, the former Lib Dem Treasury spokesman, said: "It is a real test of leadership for all the party leaders. Nick Clegg has to hold his nerve. Dave must not cringe to the Tory dinosaurs who have tasted blood. But most crucially of all, Ed Miliband has to show whether he is really genuine about reform."

Miliband was criticised for double standards by Clegg aides. But Labour sources insisted that Miliband has adopted a consistent and credible position.

Sadiq Khan, the shadow justice secretary, told the BBC: "This is not a wrecking tactic – far from it. We've already given our assurances we'll do all we can to ensure the bill progresses. Instead, it's about making good an inadequate bill. And that means allowing parliament the time to revise, amend and improve the bill free from the threat of debate being stifled. The future of a reformed House of Lords should be all the better as a result."In his resignation statement Burns, aide to Northern Ireland secretary Owen Paterson, voiced unease at being forced to vote against Tory thinking. Burns told MPs: "I couldn't look myself in the eye if I voted for this bill at second reading and clearly that is incompatible with membership of Her Majesty's government ... what an Alice in Wonderland world we now live in that by voting for something that's been a mainstream view in this party for decades, indeed generations, now leads to incompatibility with serving in the government."