How IRS drives taxpayers crazy

National Taxpayer Advocate Nina Olson has identifed 25 of the most serious problems facing taxpayers today. The IRS, however, isn't the only one to blame. Congress plays a role, too.

Any taxpayer who's gotten into a tussle with the IRS over an audit, a delayed refund or a case of identity theft may be heartened by a new report to Congress on Thursday from the National Taxpayer Advocate.

And all the while, Congress has greatly expanded the agency's responsibilities -- for instance, it plays a key role in the administration of Obamacare.

As a result, Olson notes, the IRS last year was only able to respond to 61% of taxpayer calls received, and 47% of correspondence. It has also reduced or eliminated taxpayer services at walk-in sites around the country.

Going forward, the agency will only respond to "basic" tax law questions from January through April. So much for those who file tax returns at other times of the year.

Bill of rights: Another big problem Olson identifies is the lack of a formal, consolidated taxpayer bill of rights that both IRS employees and taxpayers are educated about.

There are "dozens" of such rights scattered throughout the U.S. tax code, she noted. For example, taxpayers have a right to confidentiality and to challenge an IRS position.

"If taxpayers believe they are treated ... in an arbitrary and capricious manner, they will mistrust the tax system and be less likely to comply with the laws voluntarily," Olson's report said.

Identity theft: The way the agency helps victims of identity theft also leaves much to be desired, according to Olson's report.

The IRS takes too long -- nearly a year, on average -- to resolve cases of taxpayer identity theft, Olson said.

That leads to very long delays in victims getting their refunds. What's more, she said, the agency should centralize the process of resolving identity cases, which currently involves 20 different units.

New IRS Commissioner John Koskinen on Monday said he was "extremely concerned about the deep budget cuts that our agency has had to absorb over the last few years."

In response to the advocate's report, the agency said Thursday that it is making progress on a number of the issues that Olson raised. But "[i]It is important to note that the IRS must carefully balance limited resources to meet its dual mission of providing taxpayer service and enforcing the tax laws."