“I don't understand that,” Buffett told CNBC. “If it's some kind of a guilt tax or something of that sort because banks were among the whole U.S. that was saved back in 2008, everybody was taken care of then.”

The government forced some banks into the bailout program that didn’t want taxpayer money. And now the government is being paid back with interest, Buffett points out.

“The government's made a lot of money off Wells Fargo, Goldman Sachs, JPMorgan. Where it’s possible they'll lose money, is in the auto companies. . . . I'm the last guy to suggest that you should go and put a special tax on autoworkers.”

But if the government is really looking for companies that benefited from government losses, it would have to turn to autos, Fannie Mae and Freddie Mac, Buffett says.

“To say that they (banks) should be paying for the fact that the government lost a lot, or may lose a lot of money in Freddie and Fannie and perhaps the auto companies, it just doesn't make any sense to me.”

Buffett isn’t the only expert who opposes the bank tax.

“Any tax right now hurts the economy,” Forbes magazine publisher Steve Forbes told CNBC. “Any tax on the financial system is going to hurt the economy. Taxes aren’t going to dig us out of this hole.”