"Tom made a mistake, which he has openly acknowledged. He has not excused it, nor do I," Obama said.

Daschle, a former Senate majority leader from South Dakota, who had been one of Obama's closest advisers throughout his presidential campaign, said his tax problems meant he had lost the faith of the American people and was therefore unable to serve.

"This work will require a leader who can operate with the full faith of Congress and the American people, and without distraction," Daschle said in a statement released by the White House.

"Right now, I am not that leader, and [I] will not be a distraction," he said.

In addition to being nominated to be HHS secretary, Daschle was also slated to lead Obama's healthcare initative as health czar, a post from which he also withdrew.

Daschle's retreat raises questions about whether Obama can keep his promise to make more affordable healthcare one of the cornerstone of his agenda in his first 100 days in office.

Obama's backing didn't stop criticism of Daschle's fitness to join the White House cabinet. Sen. Jim DeMint, R-S.C., said Obama was "losing credibility" by continuing to support Daschle. "Part of leadership is recognizing when there has been a mistake made and responding quickly," DeMint said.

Daschle's withdrawal came just hours after Nancy Killefer, Obama's nominee to be chief performance officer, withdrew her nomination following the revelation that she had a $946.69 lien on her property in 2005 for failure to pay taxes.

White House Insists Daschle Wasn't "Signaled" to Quit

A third Obama Cabinet pick, Tim Geithner, admitted to Congress that he had owed and paid back more than $40,000 before he was confirmed as Treasury secretary last week, and New Mexico Gov. Bill Richardson withdrew his nomination as Commerce secretary over questions about state contracts.

The setbacks are likely to embarrass Obama, who announced a "new era of responsibility" at his inauguration and are likely to embolden Republican opposition to the president and his agenda at a time when Obama struggles to get his economic stimulus plan through Congress.

White House spokesman Robert Gibbs today dismissed suggestions that the administration has had faulty vetting process. "The bar that we set is the highest that any administration in the country has ever set," Gibbs said shortly after releasing Daschle's letter.

He insisted that Daschle had removed himself on his own accord without being "signaled" from the White House to do so.

The Obama administration has gone through a rough few weeks as it has tried to gets its team in place and live up to the president's pledge to change the way Washington operates.

He has drawn criticism, however, for relying too heavily on former members of Bill Clinton's administration. He has asked for waivers from his own directive to not allow former lobbyists work in agencies they once lobbied, and liberals have worried that his eagerness to win cooperation from Republicans has been at the expense of a liberal agenda.

After a closed-door session before the Senate Finance Committee Monday, Daschle apologized and said his failure to pay taxes was unintentional.

"I deeply apologize to President Obama, to my colleagues and the American people," Daschle said. "I would hope that my mistake could be viewed in the context of 30 years of public service."

The failure by the former Senate majority leader to pay taxes on the free use of a car and driver for several years was first reported by ABC News.

Obama did succeed in having Eric Holder confirmed today as the country's new attorney general, but that occurred only after a bruising two-week confirmation process.