The agency has reduced its workforce by 8,000 since fiscal 2009 and had an $11.3 billion appropriation for fiscal 2014, down from 2009ís $11.5 billion, according to the report.

Meanwhile average wait times at the agency have nearly doubled, from 8.8 minutes in in fiscal 2009 to 16.8 minutes in 2014. The IRS also plans to lower its audit coverage from 1 percent of filings in 2014 to 0.8 in 2015.

The IRS is also struggling with an increasing workload related to the Patient Protection and Affordable Care Act and increased identity theft actions.

The agency has delayed its Information Reporting and Document Matching System and its Return Review Program and cut staff training in order to mitigate some of the budget cuts, according to the GAO. However, any additional funding for the agency in 2015 should be paired with the consolidation of duplicative programs and cost-saving efforts, the GAO recommended.