The House Armed Services Committee is investigating the deal that released Sgt. Bowe Bergdahl in exchange for five terrorist leaders.

Testifying before the House panel, Hagel defended the administration's decision to ignore a law requiring 30 days' congressional notice for any Guantanamo Bay prisoner release.

He conceded the deal wasn't a perfect one, but said time wasn't on their side.

"Under these exceptional circumstances, a fleeting opportunity to protect the life of an American service member held captive and endangered for almost five years, the national security team and the president of the United States agreed that we needed to act swiftly," he told the panel.

According to White House officials, Hagel was the one who approved the deal.

"I hope they're not just pushing him out to be a fall guy for this," Rep. Buck McKeon, R-Calif., told Fox News Tuesday.

"It was the president of the United States that came out (in the Rose Garden) with the Bergdahls and took all the credit, and now that there's been a little push back he's moving away from it and it's Secretary Hagel?" McKeon said after a news briefing Monday.

Lawmakers question whether Bergdahl deserted and whether the United States gave up too much to secure his freedom.

Both Republicans and Democrats have criticized the Obama administration for not telling Congress ahead of time about the swap.