Additional Materials:

Contact:

In 2002, CMS contracted with Quality Improvement Organizations (QIO) to help nursing homes address quality problems such as pressure ulcers, a deficiency frequently identified during routine inspections conducted by state survey agencies. CMS awarded $117 million over a 3-year period to the QIOs to assist all homes and to work intensively with a subset of homes in each state. Homes' participation was voluntary. To evaluate QIO performance, CMS relied largely on changes in homes' quality measures (QM), data based on resident assessments routinely conducted by homes. GAO assessed QIO activities during the 3-year contract starting in 2002, focusing on (1) characteristics of homes assisted intensively, (2) types of assistance provided, and (3) effect of assistance on the quality of nursing home care. GAO conducted a Web-based survey of all 51 QIOs, visited QIOs and homes in five states, and interviewed experts on using QMs to evaluate QIOs.

Although more homes volunteered to work with the QIOs than CMS expected them to assist intensively, QIOs typically did not target their assistance to the low-performing homes that volunteered. Most QIOs' primary consideration in selecting homes was their commitment to working with the QIO. CMS did not specify selection criteria for intensive participants but contracted with a QIO that developed guidelines encouraging QIOs to select committed homes and exclude those with many survey deficiencies or QM scores that were too good to improve significantly. Consistent with the guidelines, few QIOs targeted homes with a high level of survey deficiencies, and eight QIOs explicitly excluded these homes. GAO's analysis of state survey data confirmed that selected homes were less likely than other homes to be low-performing in terms of identified deficiencies. Most state survey and nursing home trade association officials interviewed by GAO believed QIO resources should be targeted to low-performing homes. QIOs were provided flexibility both in the QMs on which they focused their work with nursing homes and in the interventions they used. Most QIOs chose to work on chronic pain and pressure ulcers, and most used the same interventions⎯conferences and distribution of educational materials⎯to assist homes statewide. The interventions used to assist individual homes intensively varied and included on-site visits, conferences, and small group meetings. Just over half the QIOs reported that they relied most on on-site visits to assist intensive participants. Sixty-three percent said such visits were their most effective intervention. Of the 15 QIOs that would have changed the interventions used, most would make on-site visits their primary intervention. Homes indicated that they were less satisfied with the program when their QIO experienced high staff turnover or when their QIO contact possessed insufficient expertise. Shortcomings in the QMs as measures of nursing home quality and other factors make it difficult to measure the overall impact of the QIOs on nursing home quality, although staff at most of the nursing homes GAO contacted attributed some improvements in the quality of resident care to their work with the QIOs. The extent to which changes in homes' QM scores reflect improvements in the quality of care is questionable, given the concerns raised by GAO and others about the validity of the QMs and the reliability of the resident assessment data used to calculate them. In addition, quality improvements cannot be attributed solely to the QIOs, in part because the homes that volunteered and were selected for intensive assistance may have differed from other homes in ways that would affect their scores; these homes may also have participated in other quality improvement initiatives. Ongoing CMS evaluation of QIO activities for the contract that began in August 2005 is being hampered by a 2005 Department of Health and Human Services decision that QIO program regulations prohibit QIOs from providing to CMS the identities of homes being assisted intensively.

Recommendations for Executive Action

Status: Open

Comments: In the 3-year contract that began in 2008, CMS discontinued use of PARTner, the electronic system it previously used to collect monthly data from the Quality Improvement Organizations (QIOs) about their interventions with nursing homes, because CMS officials had concluded that the information on interventions was not reliable or useful enough to warrant the time and expense involved in collecting it. In August 2011, a CMS official reported that the the QIO contract that began that month would require QIOs to establish learning and action networks among nursing homes and that the agency planned to implement an information system to facilitate information sharing among nursing homes in the networks. However, this system will capture information about interventions the nursing homes have implemented, not the interventions the QIOs are using to assist the homes.

Recommendation: To improve monitoring of QIO assistance to nursing homes and to overcome limitations of the QMs as an evaluation tool, the Administrator of CMS should collect more complete and detailed data on the interventions QIOs are using to assist homes.

Comments: In 2011, CMS updated its QIO privacy and confidentiality regulation to provide the agency greater access to QIO information. In particular, CMS revised the regulations governing QIOs' disclosure of quality review study information to require QIOs to disclose this information, including the identities of the providers involved in the quality review studies, to CMS when the agency deems it necessary for the purposes of overseeing and planning QIO program activities.

Recommendation: To improve monitoring of QIO assistance to nursing homes and to overcome limitations of the QMs as an evaluation tool, the Administrator of CMS should revise the QIO program regulations to require QIOs to provide to CMS the identities of the nursing homes they are assisting in order to facilitate evaluation.

Comments: CMS addressed this recommendation by requiring QIOs to select most of the homes they assist intensively from among those who scored poorly in the quality of care areas on which the QIOs were mandated to focus. Under the QIO contract that began in 2008, CMS required QIOs to select at least 85 percent of the homes they assist intensively from among those whose scores on two quality measures (pressure ulcers and physical restraints) were a specified percentage away from the goals for these measures. In this way, CMS focused QIO resources on the quality of care areas in which the homes assisted most need improvement. Under the prior contract we reviewed for our report, CMS set no such requirements and instead contracted with a QIO that developed guidelines encouraging QIOs to select committed homes and exclude those with many survey deficiencies.

Recommendation: To ensure that available resources are better targeted to the nursing homes and quality-of-care areas most in need of improvement, the Administrator of CMS should direct QIOs to focus intensive assistance on those quality-of-care areas on which homes most need improvement.

Comments: CMS agreed with our recommendation and increased the number of low-performing nursing homes QIOs must assist intensively. At the time we did our work, during the 3-year QIO contract that began in 2005, CMS directed a small share of QIO resources to low-performing homes, requiring each QIO to provide intensive assistance to one to three "persistently poor-performing homes" identified in consultation with the state survey agency. The number each QIO was expected to assist was based on the number of nursing homes in the state. (QIOs in 11 states were required to assist three poor performing homes, while the others were required to serve just one or two.) Partly in response to our recommendation, for the subsequent contract that began in 2008, the agency required each QIO to provide special technical assistance to at least three nursing homes in the state that were identified by CMS as in need of assistance with quality improvement efforts. Some of these facilities may be special focus facilities; a CMS official told us that our report helped prompt consideration of ways QIOs might work with such facilities. In addition, under the contract that began in 2008, CMS required QIOs to select at least 85 percent of the other homes they assist intensively from among those whose scores on two quality measures (pressure ulcers and physical restraints) were a specified percentage away from the goals for these measures. Under the prior contract we reviewed for our report, CMS set no such requirements and instead contracted with a QIO that developed guidelines encouraging QIOs to select committed homes and exclude those with many survey deficiencies.

Recommendation: To ensure that available resources are better targeted to the nursing homes and quality-of-care areas most in need of improvement, the Administrator of CMS should further increase the number of low-performing homes that QIOs assist intensively.

Comments: Consistent with our recommendation, CMS will use additional measures to evaluate the QIOs' work with nursing homes during the contract period from 2011 through 2014. The new measures include the number of nursing homes using evidence-based practice for the prevention and treatment of pressure ulcers, the rate of consistent assignment of certified nurse aides in the nursing home, and the rate of improvement in staff turnover from the prior year.

Recommendation: To improve monitoring of QIO assistance to nursing homes and to overcome limitations of the QMs as an evaluation tool, the Administrator of CMS should identify a broader spectrum of measures than QMs to evaluate changes in nursing home quality.