Nick Clegg has signalled his determination to press ahead with reform of the House of Lords in a move likely to anger Conservative MPs who have identified the issue as an electoral millstone that contributed to the dismal performance by coalition parties in last week's local elections.

Sources at No 10 rejected claims that the Queen's speech has been rewritten in the light of last week's election results to give it a more populist flavour – proposals such as gay marriage, which weekend reports said had been dropped, were either never due to be in it, or shelved some time ago. But the issue of Lords reform remains a faultline in the coalition. Lib Dem ministers are determined to press ahead with legislation to make the upper house largely elected, while Tory MPs and peers from all wings of the party spent the weekend identifying it as a key example of a policy showing the government out of touch with the concerns of voters.

Among the Tories joining that protest on Sunday were Lord Fowler, who said it was "bad politics" and not worth any votes, Tim Yeo, who said Lords reform should be "relegated right to the bottom of the queue" and Julian Brazier, who said it was a "ridiculous fringe" policy.

Although the Tory inquest on the local election defeats is wide-ranging, and will continue on Monday with the publication of a rightwing alternative Queen's speech on the ConservativeHome website co-ordinated by David Davis and John Redwood, there was no support on Sunday for Nadine Dorries, who said the Conservative party would split unless David Cameron and Osborne resigned. Her claim that 46 Tory MPs would demand a leadership election by Christmas, enough to trigger a ballot, was also widely dismissed.

But Lord Baker, a former chairman himself, and Lord Ryder, chief whip under John Major, both said that Cameron needed to appoint an MP as a party chairman because the two people doing the job at the moment, Lady Warsi and Lord Feldman, could not do enough from the House of Lords to defend Cameron politically. Ryder also said Cameron needed to concentrate more on substance.

"[Cameron] has to keep his eyes on the horizon and not be going down the cul-de-sac of day-to-day party management over the management of 24-hour news," Ryder said. "To this day, he seems to lack coherence, so nobody knows what he stands for, what his beliefs are, what his convictions are. They want to know more about him, they want to know where he really wants to take the country."

In his Guardian article Clegg writes: "The first two years of the coalition were a rescue mission for the economy. The second half has to be about reform. That means in politics, too. For more than a century, we have been debating the commonsense idea that the people who obey the laws of the land should elect the people who make them. Instead of getting ourselves tied up in knots in Westminster about this, we just need to get on with it."

Clegg is not claiming that Lords reform should be the government's top priority. Vince Cable, the Lib Dem business secretary, told Sky that the government should implement Lords reform "quietly and quickly" and in private Lib Dems say that it is the Tories who are pre-occupied with the issue.

"We are not obsessed with Lords reform," a senior Lib Dem source told the Guardian. "It's some people on the Tory right who are obsessed with it. It's a gigantic form of political displacement activity for them. It's completely absurd to claim that Lords reform is the reason that the Conservatives did not do as well as they did in the local elections."

On Sunday Osborne repeatedly stressed that Lords reform was not a priority for him or for the government . But, unlike his Lib Dem colleagues, he hinted that the government would be willing to compromise on Lords reform if the bill started to clog up the parliamentary timetable. In his Telegraph article Cameron did not mention Lords reform. The lesson the Conservatives should draw from the local elections was "not about tacking right or moving left", he said. "The message people are sending is this: focus on what matters, deliver what you promise - and prove yourself in the process. I get it," he said.

"Parliament can discuss these issues. Parliament's very good at discussing constitutional issues," Osborne told the BBC's Andrew Marr Show. "But it is not going to be occupying the bulk of the time of people like myself, David Cameron, and indeed Nick Clegg."Osborne's comments will reinforce suspicions that the government will eventually concede a referendum on Lords reform. Cameron and Clegg have both said that they see no need for one, but ministers may compromise to get the legislation through the Commons.

Cameron and Clegg have both said that they see no need for a referendum on Lords reform, because voters backed the idea at the 2010 general election, but Labour are demanding one and, with Tory rebels threatening to support them, ministers may decide that granting a referendum is the only way to get the legislation through the Commons.

Lord Ashdown, the former Lib Dem leader, has backed calls for a referendum, but many reformers are opposed to the idea because they believe that it would lead to the idea being rejected – a prospect that seems even stronger in the light of the way elected mayors were almost totally rejected in the 10 referendums last week.

Ahead of the Queen's speech today, the deputy prime minister yesterday promised to "wipe the slate clean to rid people of the deadweight of debt". Polly Curtis, with your help, explains why that's not correct.