Government efforts to set up a parliamentary inquiry into the banking crisis are in disarray, after Treasury select committee members insisted that they should conduct the inquiry, and Labour accused the chancellor of attempting to rig the process to heap the blame on the previous government.

The Treasury select committee, which is on Wednesday due to cross-examine the former chief executive of Barclays, Bob Diamond, met in private on Tuesday to hear cross-party demands that it conduct the inquiry. Members pointed out the committee had the requisite expertise and experience of cross-party co-operation to conduct an effective inquiry.

Andrew Tyrie, the chairman of the committee, who was chosen by the government to chair the joint parliamentary inquiry, was requested by his fellow committee members to go back to the chancellor to hold further discussions. One committee source said: "Tyrie was not given a mandate to go ahead and chair the joint parliamentary inquiry. If there is to be parliamentary inquiry, we should do it."

An attempt by the board to save Diamond's job by accepting the resignation, 24 hours earlier, of chairman Marcus Agius had backfired. Agius will now become full-time chairman and lead the search for a new chief executive.

Diamond, who has earned £100m since he joined the bank's board six years ago, decided on Monday night he could no longer tough it out after attending a rehearsal for his appearance on Wednesday in front of the Treasury select committee.

It is understood that, in telephone calls on Monday evening, Bank of England governor Sir Mervyn King and chairman of the Financial Services Authority (FSA) Lord Turner also exerted pressure on the Barclays board.

Diamond's decision to go was relayed to the Barclays directors in a hastily convened late-night telephone board meeting, and announced at 7.30am.

Speaking at an FSA public meeting on Tuesday morning, Turner said: "There were major challenges for Barclays in making sure they could convince people there had been fundamental cultural change and we simply communicated to the board that these were the issues they needed to think about; it was for them to agree whether they could achieve that degree of change under the current leadership."

Diamond's departure was followed this afternoon by that of Jerry del Missier, the chief executive's long-time lieutenant, who was promoted to chief operating officer last month. Barclays said del Missier had been cleared by the FSA of any wrongdoing.

Labour is calling for a full judge-led inquiry into the banking crisis, instead of the joint parliamentary inquiry being instigated by George Osborne. Tyrie has said he will only chair it with cross-party support, in effect giving Labour a veto over the process.

Peers are due to vote on whether to set up a judicial inquiry on Tuesday. Miliband said: "If he doesn't order that judge-led inquiry, I think he will be failing to understand both the gravity and scale of this crisis. We've had missed opportunities before; we've got to seize this moment."

It is understood that Labour and Tory MPs are concerned by the way Osborne has sought to limit the scope and the membership of the proposed inquiry. Labour MPs also believe Osborne sees the inquiry solely as an attempt to put Ed Balls, the shadow chancellor, in the dock for failing to regulate the banks properly in the middle of the last decade.

They point out that no party's hands are clean, and highlight the fact that Lord Green, the trade minister and former chairman of HSBC, was chairman of the British Banking Association at the time of the Libor crisis.

At a series of private meetings, MPs on the committee have questioned the chancellor's proposal to limit the terms of the inquiry to the Libor issue. There are also concerns that the inquiry will have only nine members – five MPs and four peers.

There are suspicions about Osborne's intentions because the main peer on the committee would be Lord MacGregor of Pulham Market, a loyalist who is vice-chairman of the association of Conservative peers.

The battle over the inquiry is turning into a political trial of strength with David Cameron determined to be seen in charge of events.

In an interview on Monday night, Tyrie said he would not go ahead with the joint parliamentary committee inquiry until there is a clear Labour support in a motion passed by both Houses – effectively giving Miliband a veto over the plans.

The cabinet discussed the issue briefly this morning, and said it would be tabling a motion in the Commons and Lords shortly.

In a sign of fresh flexibility over the terms of reference of the joint inquiry, the prime minister's spokesman said: "A parliamentary inquiry can make judgments about precisely what issues they look at." However, he insisted it was important for the inquiry to be completed quickly so any recommendation could be absorbed into imminent legislation on financial regulation and banking.

Labour has claimed the terms of reference proposed by the government on Monday were too narrow. The party said the inquiry was being asked "to look at the Libor rate-setting process and what lessons could be learnt from them in relation to transparency, conflicts of interest culture and the professional standards of the banking industry".

Labour said these terms of reference restrict an inquiry into Libor and not wider issues of banking over the past decade. The government disagrees, saying the inquiry is broader.

The Liberal Democrats have supported a parliamentary inquiry, as opposed to one led by judges. Stephen Williams, the Liberal Democrat Treasury spokesman, said it would reflect badly on parliament if it felt it could not conduct an inquiry into an issue like this.