Misidentifying Nancy Metcalf’s error

On July 14, 2013, we published a fact check of a claim from Consumer Reports’ website about the supposed savings for medical insurance consumers as a result of the health care reform bill, the Affordable Care Act. The fact check omitted a significant aspect of the story and reached some false conclusions about writer Nancy Metcalf’s story, including “Metcalf misrepresents the Kaiser study’s content.”

While reviewing Glenn Kessler’s fact check of a similar claim, we realized we had failed to keep in focus the source Metcalf used for her story. Metcalf relied on information from Centers for Medicare & Medicaid Services, under the Department of Health and Human Services and DHS Secretary Kathleen Sebelius, and the fact check should have included a reference to that material in the “References” section. Metcalf did not refer to the findings of the Kaiser study and therefore did not misrepresent its findings. We apologize to Metcalf and our readers for the error, and have revised and republished the fact check to fix the errors and clarify our criticism of the data underlying Metcalf’s claim.

Below, we have reproduced the text of the original fact check as it appeared on July 24, 2013, when we announced our intent to revise the fact check.

Editor’s note July 24, 2013: We’ve detected some problems with this fact check, stemming from a failure to sustain focus on the HHS report that served as the source for the Consumer Reports story. Look for a new version of this story to take the place of this one over the next few days, with a full explanation (and an archived version of the present state of the post) in our Corrections section.

“[The Affordable Care Act] has already saved consumers billions.”

—Nancy Metcalf, Consumer Reports, June 21, 2013

Overview

The Consumer Reports claim ignores all the ways the ACA costs consumers money as well as the effects of other cost-saving features in the bill.

The Facts

On June 21, 2013, Consumer Reports published an online article by Nancy Metcalf touting the benefits of Obamacare, specifically savings on premiums.

Metcalf:

From time to time I hear from readers who are convinced the Affordable Care Act will ruin us all financially. Not happening so far. Instead, it has already saved consumers billions.

Exhibit A: In 2012, the law directly saved consumers $3.4 billion on their insurance premiums, and will require insurance companies to rebate another $500 million in premium overcharges this summer. This is all thanks to a part of the law that requires insurance companies to spend at least 80 cents of every dollar of premium they collect on health care for their customers. If they spend less than that, they have to refund the difference.

Obamacare, also known as the Affordable Care Act, does include mandated minimum levels for insurance companies’ medical loss ratios. The MLR is the percentage of premium income used on health care benefits or “activities that improve the quality of care.”

The ACA sets the minimum MLR at 80 percent for the individual market and at 85 percent for the group market. The law sets the figure higher for the group market because the larger number of enrollees allows for lower overhead costs as a percentage of premiums. If companies do not meet the ACA standard, they must refund to customers any excess premiums charged that push the MLR below the minimum allowed.

Earlier in June, the Kaiser Family Foundation published a study estimating premium savings stemming from the ACA’s limits on MLRs. The study said consumers could save money via the MLR policy via rebates or by lower premiums that result from companies trying to avoid giving rebates.

“[I]t is hard to know with certainty what premiums would have been if the MLR rules were not in place: we cannot know for sure how insurers would have priced their products or what rates regulators would have allowed (to the extent that they reviewed rates prior to the ACA). It is also difficult to separate out the direct effects of the MLR provision from other aspects of the health reform law, particularly rate review, which works to moderate unreasonable premium increases and thus increase loss ratios. There are also data limitations. For example, prior to new reporting requirements put in place to enforce the MLR provision, there were not good data sources that break out premiums and claims on a consistent basis for major medical coverage by all types of carriers. In the initial years this data became available (2010 and 2011), there were some issues with the quality of the data, particularly regarding expenses for quality improvement and other new categories of administrative expenses that are reported on the exhibit.

So from a research standpoint, this study faces the challenges of finding a baseline for comparison and isolating the effect of the ACA’s limits on MLRs.

Analyzing the Rhetoric

We’ll examine the Kaiser Family Foundation study before analyzing the rhetoric about consumer savings from Consumer Reports.

The Kaiser study identifies a number of limitations with its research approach. The list of limitations could have been longer. Commentators such as Sally Pipes and Avik Roy have pointed out ways in which the ACA’s minimum limits on MLRs could increase premiums. The Kaiser study ignores such possibilities. Also, for the period examined in the study, health care costs have increased more slowly than normal. The Kaiser Family Foundation published a study earlier this year identifying the slow economy as the primary cause of the slowdown. Unexpectedly low health care spending implies a tendency to set premium prices abnormally high in relation to medical costs. After all, insurance companies set rates before paying claims on policies in a given year.

Though Kaiser’s study of savings from mandating MLR levels acknowledges that insurance companies started to adjust to the requirements from 2011 to 2012, it ignores perhaps the most significant adjustment: taking into account the national slowdown in health care spending. That oversight figures in particularly when Kaiser estimates customer savings, basing its estimates simply on what premiums might have cost if MLRs failed to improve from 2011 to 2012:

Premium savings for each year represent estimates of what consumers would have spent if the ratio of claims to premiums had stayed at 2010 levels. For example, individual market premium savings in 2011 were calculated by subtracting the inverse of the weighted average individual market MLR in 2011 from the inverse of the weighted average individual market MLR in 2010, then multiplying the result by the total individual market claims in 2011.

The authors of the Kaiser study do not hide its weaknesses. But politicians and journalistsreportingabout the study have tended to present the findings without the caveats. Given what it overlooks, questioning the value of the study makes good sense.

Consumer Reports does not question the value of the study at all. On the contrary, Metcalf’s article ignores the Kaiser study’s own caveats and attributes the whole of the estimated savings to the ACA’s medical loss ratio targets (bold emphasis added):

This is all thanks to a part of the law that requires insurance companies to spend at least 80 cents of every dollar of premium they collect on health care for their customers.

It’s bad enough that Metcalf failed to inform readers about the full contents of the report as well as factors that went totally unmentioned in the report. But Metcalf also failed to disclose to her readers a potential personal bias. Metcalf donated no less than $1,750 to the Obama campaign for the 2012 election cycle, according to FEC records.

More on the ACA’s medical loss ratio provisions

Insurance brokers have fought for an amendment to the ACA that would exempt insurance companies from counting their sales commissions as overhead when calculating MLRs and premium rebates.

The Congressional Budget Office estimated the effects of H.R. 1206 and agreed that much of the current improvement of MLRs comes at the expense of insurance agents and brokers (bold emphasis added):

For 2011 and 2012, CBO estimated the magnitude of the reduction in premiums resulting from the MLR policy. That estimate draws on insurance industry data and is based on two factors: actual rebates in 2012, and evidence that insurance carriers reduced administrative costs (in large part by reducing agent and broker compensation).

The CBO went on to assess that savings from the policy would tend to tail off (bold emphasis added):

Beyond 2012, in CBO’s judgment, the MLR policy under current law will continue to have the effect of reducing premiums relative to those in the absence of that policy. Over time, however, CBO expects that the reduction in premiums will be attenuated. Starting in 2014, a three-year moving average, rather than annual data, will be used to calculate the MLR, making the MLR targets easier to achieve. In addition, CBO expects that there is an increasing probability that insurers will make changes – such as increasing spending on medical benefits or quality improvement activities – that will push premiums upward.

As the critics have noted, the MLR provision creates a disincentive for insurers to control spending on medical treatment and “quality improvement activities.” For example, it will make little sense for an insurance company to fight fraud if its MLR falls close to the limit set by the health care law. The potentially misspent money helps raise the MLR, and spending to recoup misspent money lowers the MLR. The cost of administrating compliance with the MLR rules also lowers the MLR.

The Kaiser study makes its biggest mistake in calculating consumer saving from the health care law’s MLR provisions without regard for effects from other parts of the law. The study ends up giving an optimistic figure from gross savings from the MLR policy in isolation instead of giving a picture of the net costs to consumers stemming from the health care reform law. Consumer Reports magnifies the Kaiser Family Foundation’s error by erasing the caveats and reporting the optimistic conclusion as a simple fact.

It seems safe to say that the ACA has cost insurance agents millions with its medical loss ratio limits. If insurance agents do not offer a valuable service then consumers have derived a benefit from saving that expense.

Summary

“[The Affordable Care Act] has already saved consumers billions.”

Given the Kaiser study’s multiple methodological problems, one can only regard the claim as true by ignoring increased costs and assuming the study’s estimate approaches the truth by accident. The claim leaves out caveats in the original study and leaves out all mention of factors that increase insurance premiums, resulting in a fallacy of one-sidedness. Based as it is on the Kaiser study, the claim rests on a form of the Texas Sharpshooter fallacy, picking out stats that supposedly show a reduction in premiums attributable to the MLR guidelines. But the data may have a different meaning.

“This is all thanks to a part of the law that requires insurance companies to spend at least 80 cents of every dollar of premium they collect on health care for their customers.”

Metcalf misrepresents the Kaiser study’s content. The study does not claim that all the estimated savings occurred because of MLR guidelines. On the contrary, the study warns of the difficulty in ruling out other factors. Metcalf’s description of the MLR guidelines also contains ambiguity and inaccuracy, for group plans need to spend no less than 85 cents on the dollar and spending on efficiencies, not just health care, can contribute toward meeting the minimum spending goal.

It’s also appropriate to provide some direct documentation of Nancy P. Metcalf’s political preference, for it relates to her failure to disclose to readers of Consumer Reports a potential bias. The FEC makes available images of the receipts from political giving above certain thresholds. Find many more such records of Metcalf’s political giving through FEC.gov.