The ousted head of the US Revenue Service has apologised for "foolish mistakes" in targeting conservative groups but insisted the agency's actions were not politically motivated.

Steven Miller was forced out of his job as the acting head of the IRS this week after it was revealed the federal tax agency had targeted conservative non-profit groups for additional scrutiny on their applications for tax-exempt status.

In a congressional hearing, lawmakers from both political sides hammered Mr Miller over abuse at the agency, which President Barack Obama has criticised as outrageous and unacceptable.

Mr Miller apologised for the IRS' actions, but said it was a case of the agency's employees not being able to handle an influx of non-profit applications made as the Tea Party movement gained momentum.

"I want to apologise on behalf of the Internal Revenue Service for the mistakes that we've made and the poor service we provided," Mr Miller said during the tense hearing at the House Ways and Means Committee.

"Foolish mistakes were made by people trying to be more efficient in their workload selection."