Culture secretary Maria Miller came under increasing pressure over her expenses from within the Conservatives after a government minister suggested she could have done a better job of saying sorry and a parliamentary aide signalled she should be concerned for her position.

Esther McVey, a Tory work and pensions minister tipped for promotion, said Miller's 32-second Commons apology for over-claiming expenses was not how she would have done it herself, while Nicola Blackwood, a Tory aide in the business department, indicated she would be "really quite worried indeed" if in the same position.

Throughout the furore over Miller's over-claimed expenses, David Cameron has given her his full support. On Monday, he said the culture secretary had "done the right thing" after she was asked by the Commons standards committee to repay £5,800 and apologise for failing to co-operate. However, the first slight crack in Cameron's position emerged when he acknowledged that there might need to be changes to the Commons standards committee, including giving a formal power of veto on its decisions to lay members.

Questions have been raised about why the committee watered down an original recommendation from the independent commissioner, Kathryn Hudson, for Miller to repay £45,000, giving the impression that MPs still appear to be policing their own ethical standards despite the expenses scandal of 2009.

The pressure on Miller from within the Conservatives may come to a head at Wednesday's meeting of the 1922 committee of influential Tory backbenchers. Ahead of this meeting, McVey broke cover to tell ITV's The Agenda: "I can honestly say it wouldn't be how I would have made an apology."

While acknowledging different people apologise in different ways, she also said: "Fundamentally what we've got to do is make sure the public believe in their representatives and it is only right for the public and for politics that we get this matter right and we did actually because we changed the system in 2010."

When asked about whether Miller should go, the minister said: "David Cameron has the final say on this. He's standing by her. There are two key points here that really need to be raised. The chap on the video said 'the public are angry, look what's happening' - and I think the whole system has changed since 2010 and remember this is pre-2010 - this couldn't happen now. We do have the independent parliamentary standards committee there so we've got that and will look at things."

Nicola Blackwood, a senior Tory MP, also told the BBC: "I have to say if I was faced with the kind of questions that she is faced with, I would be really quite worried indeed. I only know what has been reported in the papers, but clearly it's very unhelpful for this to drag on in the way that it is."

Meanwhile, Dominic Grieve, the Conservative attorney general, also stopped short of wholly endorsing her position. "She's a valued colleague, as far as I'm concerned – but she has got to answer to her constituents and also answer for her responsibilities to the prime minister."

Lord Tebbit, a former Tory chairman, and a group calling itself Conservative Grassroots, have previously called for her to be sacked.

To add to the pressure, Labour MP Sheila Gilmore made a formal complaint to the standards committee over Miller's apology in the Commons, calling it "inadequate to the point of being contemptuous of your committee's report and the members' code of conduct".

John Mann, a Labour MP who has led calls for her resignation, will attend a meeting in the culture secretary's constituency of Basingstoke to hear the concerns of local voters. He also raised a point of order in the House of Commons to call for a debate on whether MPs should still be allowed to mark their own homework. John Bercow, the speaker, said it was possible there could be such a discussion before recess on Thursday. "I'm very open to these matters being aired if MPs wish to air them," he said.

There has been speculation that Miller could be moved to another role or demoted at the next reshuffle, expected after the European elections, despite a shortage of women at the top table.

But the prime minister's official spokesman said Cameron wanted her to keep her role and had full confidence in her abilities. He pointed to her work on first world war commemorations as evidence of her doing a good job.

The Downing Street aide also brushed off questions about whether the prime minister would expect capital gains tax to be paid on a property that has been designated a second home for expenses purposes.

This came after the Daily Telegraph raised fresh questions about whether Miller would pay the levy on the recent sale of her home in Wimbledon, which was sold at a profit of more than £1m earlier this year and was for four years designated as her second home so that she could claim expenses towards the cost of the mortgage.

Aides to Miller have now clarified that she is expecting to pay some capital gains tax. A spokesman said it was "common knowledge" that the London house had been her second home between 2005 and 2009.

"HMRC will present her with a bill," the spokeswoman said. "There is no hiding anything here."

Although Miller's expenses have caused Cameron a headache this week, the prime minister is facing one less problem after backbencher Andrew Bridgen withdrew his letter of no confidence in him that was sent to the 1922 committee. Bridgen, the MP for North West Leicestershire, wrote to Cameron: "It would be wrong for me to do anything that made the election of Mr Miliband as Prime Minister more likely, and the inevitably disastrous consequences this would entail for the nation I love.

"Now as we face the General Election in 2015, I believe it is right for me to offer you my full and enthusiastic support in anticipation of the crucial campaign ahead of us and I do so without reservation."

• This article was amended on 8 April 2014. We initially had the figure Miller was told to repay by the Commons standards committee at £4,500, rather than £5,800.