Additional Materials:

Contact:

In September 2006, the Internal Revenue Service (IRS) started the private debt collection (PDC) program for using private collection agencies (PCA) to help collect some unpaid tax debts. Aware of concerns that PCAs might cost more than using IRS staff, IRS began studying the collection costs and performance of PCAs and IRS. In March 2009, IRS announced that it would not renew its PCA contracts based on the study and announced plans for increasing collection staffing. As requested, GAO is reporting on whether (1) the study was sound as primary support for IRS's PDC decision and (2) IRS has planned or made changes to its collection approach based on its PCA experience and PDC study. GAO compared IRS's study to federal and other guidance on what should be included in analyses to support program decisions and analyzed IRS's changes given expectations that IRS would consider PCAs' best practices.

IRS's comparative study of the PDC program was not soundly designed to support its decision on whether to continue contracting out debt collection. Although the study was not originally intended or designed as primary support for the decision, IRS officials nonetheless used it as such. IRS did not have guidance for program managers on the type of analysis that should be done to support decisions to create, renew, or expand programs. IRS had not retained sufficient documentation on the sample used in the study or documented some analyses that would have been helpful if performed. The study results may be overstated or understated because the study sample was not generalizable to the program as a whole. The study had a narrow objective of comparing results for IRS working the same cases as PCAs had, and as a result, the study design did not consider other factors recommended by Office of Management and Budget and other guidance on conducting program analysis. For example, the study did not analyze alternatives to program scale, such as expanding it or scaling it back. Program analysis guidance states that to the extent possible all costs and benefits should be counted and alternative means of achieving a program's goals should be considered. But the study did not identify important costs and benefits, such as whether taxpayers' compliance costs would be different if IRS or PCAs work debt cases. Nevertheless, neither GAO nor IRS officials know whether the study results and decision on the program would have differed significantly if it had been designed to be primary support for IRS's PDC program. In commenting on a draft of GAO's report, IRS disagreed that the PDC study was not soundly designed. GAO stands by its analysis detailing the study's errors, narrow scope, and lack of adherence to guidance. These design and methodology deficiencies limited the study's usefulness in supporting IRS's decision. IRS has not made or planned changes to its collection approach based on its PCA experience and study. In authorizing the use of PCAs, Congress required IRS to report to Congress its measurement plan to identify any of the PCAs' best practices that IRS could adopt to improve its own collection operations. IRS did not continue to report to Congress as required. In an unpublished draft report, IRS asserted that it had reviewed a number of PCA practices and found no immediate opportunities to change its collection approach. IRS did not provide GAO documentation on the study to support that conclusion. In part because PCA-type cases had previously been considered low priority, IRS officials were surprised by the PDC study results, which indicated that IRS staff might have better results working PCA-type cases than some of the cases IRS normally works. IRS officials said that they initiated a pilot study in 2009 to help them decide whether to use IRS staff to work selected types of PCA cases. As GAO concluded its review, IRS provided conflicting information on the role of the pilot study. On one hand, IRS said a collection selection system to be implemented in January 2011 overtook the need for the study. On the other hand, an IRS official said that the results from PCA-type cases were not used in the development of the new case selection system. GAO recommends that IRS (1) establish guidance on analyses to support program decisions, (2) establish a policy requiring documentation of program studies, and (3) ensure that PCA-type case results are considered for IRS's new case selection model. IRS agreed with the first two recommendations and agreed in principle with the third, which GAO revised to reflect updated information that IRS provided.

Recommendations for Executive Action

Status: Closed - Implemented

Comments: IRS implemented Consolidated Decision Analytics (CDA), which includes case modeling to better identify and prioritize tax debt cases with the best potential for collection, including PCA-type cases in the inactive queue. With implementation of CDA in December 2011, inventory in the Automated Collection System (ACS) function and the queue are scored and prioritized. IRS also established an annual assessment to monitor CDA model performance based on case outcome data from the ACS and Field Collection functions, to include PCA-type cases.

Recommendation: The Commissioner of Internal Revenue should ensure that PCA-type case results are considered and incorporated as appropriate into the Consolidated Decision Analytics (CDA) model.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Closed - Implemented

Comments: IRS's "Guidelines for Conducting Program Evaluations," (published in May 2012 to guide studies to support program changes) includes a section titled, "Documenting the Evaluation." The section notes the importance of documenting studies and states the minimum requirement of archiving the study design, the data files and databases created, and the final report containing the findings and conclusions. The guidance also states that the study documentation is to specifically how long documentation is to be retained after study completion and how and where key evidence is to be archived. In addition, the guideline includes the appendixes "Documentation Template" (that lays out the documentation to developed at each phase of a study), and "Documenting Sampling Plans" (that includes guidance on documenting key information such as the type of sample used, sample plans, and sampling error calculations).

Recommendation: The Commissioner of Internal Revenue should establish a policy requiring documentation for the design, analyses, and conclusions of studies supporting program changes.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Closed - Implemented

Comments: In May 2012, IRS's Office of Research, Analysis, and Statistics (RAS) published "Guidelines for Conducting Program Evaluations," which is guidance for conducting studies to support decisions to initiate, renew, or expand a program, among other purposes. The guidelines include a section on the potential types of evaluations and evaluation methods that can be employed and an appendix that offers guidance for considering different design approaches and strategies. Furthermore, the guidelines refer to Office of Management Budget Circular A-94 (specifically in reference to cost-effectiveness analysis to identify the least costly alternative to meet a program goal) and several other sources on how to conduct program evaluations.

Recommendation: The Commissioner of Internal Revenue should establish guidance on the types of analyses that should be done to support decisions to initiate, renew, or expand programs. The guidance might refer to OMB Circular A-94 and, if needed, provide any supplementation specific to IRS.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Closed - Not Implemented

Comments: This was a conditional recommendation. In commenting on the report, IRS said events (specifically, the planned implementation of Consolidated Decision Analytics) had overtaken the need to complete the study.

Recommendation: If IRS determines completing the pilot study is the best means to ensure that PCA-type case results are considered for the CDA models, the Commissioner should ensure that the pilot study has a documented methodology and criteria to guide IRS's analysis and decision.