Now that the Affordable Care Act (ACA) repeal-and-replace process is coming to an end, perhaps it’s a good time to turn to an area of health policy where there is considerably more bipartisan consensus: payment reform. Even here, however, challenges remain. A recent spate of evaluations, reviews, and published perspectives have cast doubt on the promise and spending-reduction potential of care coordination initiatives, shared savings accountable care organizations (ACOs), patient-centered medical homes, and bundled payments in particular. As the Trump administration, members of Congress, states, and other health care stakeholders formulate their own approaches to payment and delivery reform 3.0 (remember pay-for-performance?), it is important to avoid being overly discouraged in the face of the mixed results we have seen so far.

Let’s start with the ACA’s flagship payment initiative, which was baked into the statute and thus not a Center for Medicare and Medicaid Innovation (the Innovation Center) pilot but a program, the shared savings ACOs or Medicare Shared Savings Program. It is an understatement to say that there is considerable disappointment that on balance the Shared Savings Program appears to be actually costing Medicare money, $216 million on net by the end of 2015. This means that the losers are losing more than the winners are saving for Medicare, net of their payouts. No downside risk for the Shared Savings Program ACOs of course translates into the fact that the losses are uncapped for the program as a whole. Still, the recent summary makes clear that ACOs that entered early are now saving Medicare absolute dollars on net; that is, they are performing much better than those who entered more recently. And a recent Health Affairs Blog post pointed out that Medicare’s method for setting baseline spending targets is not likely to be the same kind of counterfactuals that the better evaluations employ, based on similar groups’ actual performance in the same time period. Nevertheless, it seems fair to conclude that even with and maybe especially with no downside risk, learning and implementing enough care transformation and key patient identification techniques to save substantial resources while maintaining or improving quality takes some time.

Bundled payments are logically superior to fee-for-service in theory, and the most interesting bundled payment experiments of the Innovation Center, Bundled Payments for Care Improvement Model 2 (prospective acute and postacute), showed that 3 percent savings are possible in orthopedics and cardiology from inpatient through 90 days after discharge, but only through postacute use savings. The bundle for spinal surgery actually led to an increase in total costs of more than 10 percent. Model 4 (prospective for hospital and physician for inpatient only) results were insignificant all around. Bundled payment models are promising incentive and care changing tools, but clearly parameters need adjusting since broad savings may be initially possible only in postacute use choices. That’s something but quite a bit less than hoped for by advocates to be sure.

Now for the most common payment reform model across public and private payers, patient-centered medical homes. The Comprehensive Primary Care Initiative (CPCI) evaluation of a medical home model for Medicare beneficiaries is sobering, since this was the Innovation Center’s high-profile effort to improve primary care performance, with more than 2,000 clinicians in more than 450 practices participating from 2012 through 2015 across seven states or regions from New York to Oregon. Total costs did not decline in any year, on average, net of care management fees, and only two regions reported net savings in any one year. Quality improvement was even rarer, with only two metrics showing improvement, and these results were driven by a minority of regions. These results are similar to the meta-analysis of 11 private-sector patient-centered medical home pilot programs reviewed by Sinaiko et al., who found no statistically significant association of patient-centered medical home practice transformations with total health spending, use, or quality in the full patient sample.

The patient-centered medical home initiatives that failed to reduce cost or raise quality on average, both from the Innovation Center and those included in the Sinaiko study, have at least one set of characteristics in common: They aimed for wholesale practice transformation, as codified in patient-centered medical home certification programs such as the National Committee for Quality Assurance’s (NCQA’s) in Sinaiko’s pilot programs, and in the milestone requirements set for CPCI practices, 39 percent of whom had NCQA certification at the outset. The milestones included 24/7 access and same-day appointment slots for all and care management for high-risk patients, features shared by more streamlined patient-centered medical homes such as the ones created by CareFirst BlueCross BlueShield (in Maryland, the District of Columbia, and Northern Virginia). But the Innovation Center milestones also included patient surveys or family advisory councils, continuous quality improvement on at least three clinical quality measures, contacting 75 percent of patients within 72 hours of discharge, enacting care compacts with two groups of high-volume specialists, implementing at least three shared decision-making tools, participation in regional and national learning collaboratives, and ensuring that each clinician could attest to meaningful use stage 2.

The point is not that any of these activities is wasteful, but evidence suggests that they were not well targeted to a smaller group of individuals who would benefit most from the additional resources. Each of the milestones is a reasonable expansion of capacity to deliver higher-quality care, and NCQA certification has helped practices improve quality and lower costs in many cases. Yet, in these recently reviewed patient-centered medical home pilot programs, it is possible that too much emphasis was put on building capacity to provide all services to all patients and not enough on the fact that cost and quality improvements might be more efficiently pursued by first identifying a subset of patients on which to focus more targeted yet patient-centered resources. The CPCI paid practices almost $50,000 per physician per member per month, and while they saved enough money so that Medicare “broke even,” they apparently spent it all on building quality capacity and did not improve clinical quality in very many measureable ways.

Identifying high-cost, complex patients and targeting resources and care delivery plans for them while adding a powerful incentive to reduce total cost of care is what CareFirst implemented in Maryland, the District of Columbia, and Northern Virginia. Three published evaluation papers (Cuellar, Afendulis, and Gimm) have reported on different aspects of CareFirst’s patient-centered medical home Total Cost and Care Improvement Program. And like the patient-centered medical home literature as a whole, a quick scan of all three papers would lead one to conclude that program impacts were “mixed.”

Cuellar et al., concluded that the CareFirst patient-centered medical home program—which focused on care plans and nurse care coordinators for complex patients as well as plan-provided data for clinicians to manage against cost and quality targets and earn substantial shared savings bonuses—lowered cost growth relative to a control group by 2–4 percent in the second and third years of implementation, using a two-part model and propensity score weighting to minimize the effects of measured heterogeneity. They defined a control group to be CareFirst patients of primary care providers who never entered the voluntary program (employers and physician practices could choose yes or no) and defined the treatment group to be those who entered, even after the program had been launched in 2011, and remained in. Thus, the target group included members who joined as the program expanded and offered new features such as enhanced information tools to natural learning by the payer, providers, and patients alike. This definition of the treatment group approach has fidelity with the actual intervention as it evolved and is similar in spirit to that now used by the Centers for Medicare and Medicaid Services (CMS) in ACO and patient-centered medical home evaluations alike, performance year (versus prospective) attribution, which adjusts the group for which clinicians are held accountable as they roll in and out of the practices’ active patient panels.

Solid Pointed Pumps Pull 42 On High Women's Odomolor Heels Gray Toe Frosted Shoes qEUgw8cC4., anticipated concerns about reversion to the mean, in that members with above-average cost growth may have been more likely to join the program and thus more likely to experience subsequent cost savings. For this reason, they excluded all those who entered the program after the first year from the treatment group but included them in the control group. This decision and other exclusions (for example, they included only those with an evaluation and management visit in the baseline year) had the net effect of making their treatment group roughly one-half the size of Cuellar’s, since so many employers and physicians chose to enter the program after the first year. Their approach is vulnerable to attenuation bias by having later actual patient-centered medical home patients included in the control group, which would skew any cost savings estimates toward zero. In effect, they traded a fear of endogeneity bias for the certainty of attenuation bias. This approach is similar to the prospective attribution eventually rejected by CMS. Using ordinary least squares regression and fixed effects on patients to control for unmeasured heterogeneity, Afendulis et al., concluded there were no program effects on cost or quality.

These studies made different choices and used different methods, leading to different results. It is up to the reader to interpret the results based on the plausibility of implications of methods choices. The Afendulis et al., approach is a fair way to evaluate the impact of the patient-centered medical home program on a defined subpopulation perhaps but does not seem true to the evolving nature of the actual program being evaluated as a “moving target,” nor does it capture the enrollment process: that employers and physicians made choices, on behalf of all their covered enrollees or patients, to enter the program over time instead of being “assigned” by CareFirst at a single point in time. For example, we know that roughly twice as many physicians participated in year three as in year one.

Cuellar et al., interpreted CareFirst’s patient-centered medical home program based on the approach that Rosenthal et al., used for patient-centered medical homes in general: The program itself evolved considerably over time and was quite different in 2013 (the third year of intervention) compared to 2011 (the first year). “PCMH implementation is an iterative and lengthy process that can often take longer than a year.”

In the program’s first year, CareFirst hired and replaced many nurse care coordinators as they learned what type of personality to hire. As they saw the surprisingly rapid rate of take up among physicians in the first year (more than 200 panels of ten clinicians each), the need for nurse care coordinators far exceeded the availability of trained staff. The case management and information tools available to practices were considerably enhanced in later years; clinicians and practices learned how to better coordinate care and for which types of patients it really mattered, in some cases new specialist and hospital referral choices came to be made; and patients may have learned how best to interact with the dyad of physician and nurse care coordinator.

We believe there is a plausible explanation for the delay of the program’s effects. While economic incentives and useful information can be supplied quickly, changing the essential relationships to achieve cost-effective care takes more time. These relationships all move at the speed of trust between two or more of these parties: the patients being coordinated, the local nurse care coordinator, the primary care physicians, the sponsoring health plan, the specialists who receive referrals, and the hospitals that supply inpatient care to these populations. Everyone in this chain has multiple incentives, not simply economic ones. To effect real change, it takes time for each party to test the intentions of the others and to agree that those intentions align with their own.

The notion that the many participants in the health care delivery continuum need time to adopt cost-saving and quality-improving practices is consistent with finding effects, if at all, in the later years. Combining the CareFirst results with a careful read of all the nuances of the CPCI evaluation and the Sinaiko meta-analysis, along with the just-released Patient Centered Primary Care Collaborative review of the published evidence, it is fair to conclude that patient-centered medical homes can save money and improve quality in some but clearly not in all cases. It is much easier if you focus on a targeted and small subset of patients (usually those with multiple chronic conditions), and it is likely to take some considerable amount of time for a patient-centered medical home program to yield results. It is imperative that more research and analysis be done to better understand why some patient-centered medical homes perform better than others and which patient-centered medical home program elements matter. But it is also true that the highest estimated effects of the more targeted CareFirst patient-centered medical home program are relatively small and less than hoped for by advocates of care coordination as the defining strategy for US health care reform.

To conclude, our read of the evidence to guide us in the land of payment reform 3.0 suggests these takeaways: improvement in care and cost performance takes time; identifying target patients may be more important than building capacity to provide patient-centered care to all, for some do not need it now but will be glad the capacity exists some day; savings may result in unexpected places (for example, postacute); care transformation while bending the cost curve is hard work and requires an up-front resource investment and a shared commitment to data sharing and incentive realignment to really take hold; the savings, when present, are still small enough that more robust cost reduction strategies than this general class of “win-win” models, which depend ultimately on utilization efficiencies and shared savings, may be necessary for policy makers to achieve the goals of freezing or even reducing the share of gross domestic product devoted to health care in the United States.

Finally, the relative success of the CareFirst versus the CPCI’s patient-centered medical home model suggests that the idea of a per-member-per-month payment—which does give resources to the practices but also puts pressure on them to spend it wisely—may not be superior to payer-provided in-kind information and care management support, for the risk to small practices is then much less and in a world with alarming levels of physician burnout, reducing provider stress is a good idea. It is true that evaluation results of payment reform models so far are not cause for celebration, but there are some positive cost and quality results and lessons are accumulating that can inform the next set of program parameters, incentive structures, and tools to help clinicians succeed. Stopping the experimentation now because of disappointment or allowing unwarranted pessimism to take hold could quickly negate the real progress that has been made on many fronts, and that is essential for our health care systems’ future affordability.

Authors' Note

CareFirst of Maryland, the District of Columbia, and Northern Virginia, funded three evaluation teams (George Mason University (GMU), Harvard, and Westat) for their patient-centered medical home program, and Len Nichols was the PI for GMU’s work, (Mike Chernew served in that role for Harvard and Joanne Sorra for Westat). Nichols is an unpaid board member of the nonprofit National Committee for Quality Assurance, which certifies patient-centered medical homes, and he is an unpaid member of the Physician-Focused Payment Model Technical Advisory Commission (PTAC), which The Medicare Access and CHIP Reauthorization Act created to advise the secretary of the Department of Health and Human Services about payment reform models.