The author is a Forbes contributor. The opinions expressed are those of the writer.

Loading ...

Loading ...

This story appears in the {{article.article.magazine.pretty_date}} issue of {{article.article.magazine.pubName}}. Subscribe

Last night, The New York Timesposted an interesting debate on the question: Do the mixed results of an Oregon health care study show that government medical insurance should provide only catastrophic coverage? The Oregon Health Study (OHS) showed that compared to being uninsured, “Medicaid coverage generated no significant improvements in measured physical health outcomes.” Specifically, no clinical differences were found in blood pressure, blood sugar and cholesterol levels.

English: Medicare and Medicaid as % GDP Explanation: Eventually, Medicare and Medicaid spending absorbs all federal tax revenue, which has averaged around 19% of GDP for the past 30 years. Category:Health economics (Photo credit: Wikipedia)

Astonishingly, one debater, Austin Frakt, argued that there’s a simple explanation for this dismal result: “the sample was far too small for it to be able to do so.” Seriously? The RAND Health Insurance Experiment—the gold standard of evidence regarding the impact of cost sharing on utilization, spending and health status—had a sample size of 5,809 spread across 5 major categories of plans (e.g., the free care plan enrolled 1,893, the 25% cost-sharing plan enrolled 1, 137, the 95% cost-sharing plan (essentially a high-deductible plan) enrolled 1,120 etc.).[1] There were actually 14 different plans studied, so the sample sizes for each were even smaller than these figures might imply. Moreover, the RAND HIE included a random sample of people across a wide range of incomes and all ages below age 62,[2] whereas the Oregon Health Study was restricted to 6,387 low income, non-elderly, non-disabled adults who were randomly selected to be able to apply for Medicaid coverage and 5,842 of their counterparts who were not selected.

Admittedly, the RAND scientists used person-years as their unit of analysis and person-years ranged from a high of 6,822 in the free care plan to 1,401 in the 50 percent plan. But using that standard, the two-year follow-up used in the OHS study implies a sample size that is double that reported etc. Now admittedly, only 1,903 (26%) of those eligible for Medicaid in the OHS sample actually ended up enrolling in the program.[3] But remember, every single one was a poor, non-elderly adult age 19-64, whereas in the RAND HIE, 36% of participants were children and most were not poor.[4] RAND defined low income as being in the lower one fifth of the income distribution, so on an apples-to-apples basis, the HIE had approximately 750 non-elderly poor adults spread across a variety of plans, roughly one-third of which would have ended up in the free care plan comparable to Medicaid. Even if you inflate the roughly 250 such individuals into person-years, the HIE sample would be about 850[5] versus 1,903 in the OHS.

You might think this is much ado about nothing, but consider what the RAND HIE was able to demonstrate despite its substantially smaller sample size of low income adults:[6]

For those who began the experiment with high blood pressure (the 20% having the highest diastolic blood pressure), free care plan participants had a clinically significant decline in blood pressure compared to their counterparts in cost-sharing plans.

Epidemiologic data imply that a reduction of this magnitude would lower mortality about 10% a year in the free care group (the sample size was too small to actually measure this mortality reduction among HIE participants).

Yet Austin Frakt solemnly assures us that the OHS was “far too small” for it to be able to show any statistically significant effects of Medicaid on physical health. As proof of this claim, he points us to a blog post by Kevin Drum at Mother Jones who boldly asserts “that the study couldn't have found statistically significant improvements. It was impossible from the beginning.”

Not content with that sweeping (empirically unsupportable) generalization, Mr. Drum goes on the berate the Harvard researchers for how they conducted their study, reaching this breath-taking conclusion:

they probably shouldn't even have reported results. They should have simply reported that their test design was too underpowered to demonstrate statistically significant results under any plausible conditions. But they didn't do that. Instead, they reported their point estimates with some really big confidence intervals and left it at that, opening up a Pandora's Box of bad interpretations in the press.

Health services researchers will quickly recognize that Mr. Drum does not know what he’s talking about. But the lay public may not. For Mr. Drum to lecture the highly distinguished health economists who conducted the OHS on how to do their work is comical (“The first thing the researchers should have done, before the study was even conducted, was estimate what a clinically significant result would be.”[7]). The OHS investigators included not only Dr. Joseph Newhouse (the study director for the RAND HIE and one of the nation’s leading health economists), but Amy Finkelstein, the 2012 recipient of the John Bates Clark medal, one of the two most prestigious honors in the field of economics.[8] Anyone who has read the OHS study or the 62-page supplemental appendix that accompanied it should recognize that it is cutting edge research using the best available measures and methods. If Medicaid had a beneficial impact on physical health, these researchers assuredly would have found it.

It's worth noting the OHS results were not all bad.

Medicaid coverage significantly increased the probability of a diagnosis of diabetes and the use of diabetes medication. However, researchers found no significant effect on average glycated hemoglobin levels or on the percentage of participants with levels of 6.5% or higher (which is the standard diagnostic criterion for diabetes).

Medicaid coverage almost completely eliminated catastrophic out-of-pocket medical expenditures: 5.5% of the control group had spending that exceeded 30% of family income vs. only 1.1% for those on Medicaid. But the average difference in out-of-pocket spending—albeit statistically significant—was astonishingly modest. Those in the uninsured control group spent $553 annually out-of-pocket, while the Medicaid group spent only $215 less.

Medicaid coverage decreased the probability of a positive screening for depression from 30% in the uninsured group down to 21%; at least one third of this apparently was due to being less likely to be first diagnosed for depression after the lottery (4.8% in the uninsured group compared to 1.0% in the Medicaid group).

Medicaid coverage led to an increase in the proportion of people who reported that their health was the same or better as compared with their health 1 year previously (80.4% for uninsured vs. 88.2% for Medicaid).

Medicaid coverage led to a very slight increase—one-fifth of a standard deviation, for statistically-minded readers—in self-assessed mental health.

Medicaid coverage led to improvements in perceived access to and quality of care (having a usual place of care, received all needed care within past year, care received was of high quality).

Medicaid coverage led to greater use of preventive services including cholesterol-level screening, Pap smears in women, mammography in women 50 years or older, and PSA (prostate exam) tests in men 50 years or older.

These modest gains in health status and financial peace of mind came at a cost. Annual spending was $1,172 (35%) higher for those on Medicaid. Worth noting as well is that these modest improvements in access, use, quality, mental health and financial protection produced no significant difference in self-reported happiness. Despite being uninsured for at least six months, 74.9% of the control group reported being very happy or pretty happy compared to 76.1% of those on Medicaid.

So it all comes down to value for money. Are these gains worth spending roughly $1,200 apiece to give uninsured adults below poverty Medicaid coverage? Every reader will have their own opinion on this matter. But most would agree it’s not worth spending an average of $1,172 if all Medicaid achieves is a reduction in average out-of-pocket spending by only $215. In that case, it would be far cheaper to simply reimburse individuals for expenditures above some agreed-upon catastrophic threshold. The OHS also explodes the myth that giving the uninsured Medicaid coverage somehow would magically eradicate over-use of the emergency room. Despite a 50% increase in the average number of physician visits, those on Medicaid had slightly more (albeit statistically insignificant) ER visits than those who were uninsured. So one cannot use the OHS results to claim that added Medicaid spending pays for itself in the form of savings from more efficient use of the health system.

So leaving financial protection aside, the question is whether the remaining $950 or so in spending is worth Medicaid’s incremental benefits, especially given that at a big picture level, these apparently did not appreciably affect levels of happiness. If the goal is to improve the lot of the poor, can we imagine alternative ways of spending such large sums that would produce far greater gains in their welfare? I have to concur with Ross Douthat that there probably are.

The OHS demonstrates that--while by no means perfect--the status quo provides $3,350 in care to uninsured adults without spending a single extra dime on Medicaid. It further proves that low income uninsured adults apparently are smart enough and well-informed enough to navigate that system to find and secure the most important treatments, resulting in approximately the same physical-health outcomes as their counterparts on Medicaid. This suggests that states ought to be encouraged to experiment with innovative approaches to providing health care to low income families. The Healthy Indiana Plan, for example,

requires each participant to make a modest financial contribution, and it provides incentives for participants to stay healthy, be value and cost-conscious, and to utilize services in a cost-efficient manner. This plan covers essential health services and is similar to commercial plans. In contrast, Medicaid has unlimited benefits and services, and recipients have no incentive to be responsible for their health status, to be mindful of costs, or to utilize health care services efficiently.

Obamacare has two important design flaws related to Medicaid. First, it seeks to vastly expand a poorly functioning program—a program that the OHS shows barely works much better than the patchwork safety net for the uninsured the many people have insisted for decades is unacceptable. Second, it further centralizes control of the program in Washington, D.C. Indeed, had the Supreme Court not stepped in, the law would have forced states to swallow a massive expansion whether they wanted to or not. What we've discovered now that states have the freedom to choose is that about half the states are opposed to expansion (and that was before the OHS results were reported!). Americans would be far better served if federal bureaucrats stopped trying to micromanage Medicaid and instead gave states block grants that both encouraged far more fiscal discipline, but also created the incentive to unleash dozens of different experiments similar to OHS from which states could learn from each other what works best. If current Medicaid is the best we can do for the poor, we (and they) are in deep trouble.

[2] Technically, the RAND study excluded families in the top 3% of income, Medicare disability beneficiaries, those in jails or institutionalized for indefinite periods, those in the military and their dependents, and veterans with service-connected disabilities. But the point is that the OHS focused on a much narrower segment of the population than did the HIE, yet had a far larger sample.

[3] For logistical reasons, the OHS sample was limited to the Portland area. Statewide, only 60% of those eligible to enroll in Medicaid mailed back the enrollment paperwork within the 45-day window given to them. Of these, half turned out not to qualify--mostly because their income was not below poverty or because they had assets exceeding $2,000. Overall, 30% of lottery winners successfully enrolled in Medicaid.

[5] These are purely arithmetic calculations that assume that the sample was exactly balanced. Thus, 5,809 x .64% = 3,717 (the approximate number of adults) x 1/5 = 744 (the approximate number of low income adults) x (1,893/5,809) = 242 (the approximate number of low income adults in the free care plan) x (20,190/5,809) = 842 (the approximate number of low income adult person-years in the free care plan.

[6] There were other improvements in physical health among low income adults in the free care plan: a) For those who began the experiment with vision problems correctable with eyeglasses, there was a modest improvement in corrected vision; b) For those between ages 12 and 35 there was some improvement in oral health: decayed teeth were more likely to be filled and there was modest improvement in the health of the gums. However, the Medicaid plan studied in the OHS (Oregon Health Plan Standard) does not cover either vision care or nonemergency dental services, so it would not have been feasible for Medicaid to have influenced these outcomes in Oregon.

[7] As the researchers themselves report:

“Virtually all the analyses reported here were prespecified and publicly archived (see the protocol). Prespecification was designed to minimize issues of data and specification mining and to provide a record of the full set of planned analyses.”

“Hypertension, high cholesterol levels, diabetes, and depression are only a subgroup of the set of health outcomes potentially affected by Medicaid coverage. We chose these conditions because they are important contributors to morbidity and mortality, feasible to measure, prevalent in the low-income population in our study, and plausibly modifiable by effective treatment within a 2-year time frame.”

“Anticipating limitations in statistical power, we prespecified analyses of subgroups in which effects might be stronger, including the near-elderly and persons who reported having received a diagnosis of diabetes, hypertension, a high cholesterol level, a heart attack, or congestive heart failure before the lottery. We did not find significant changes in any of these subgroups.”

In short, while Mr. Drum’s account implies otherwise, the researchers actually did a lot of advance thinking about what to measure and how to measure it in light of their anticipated sample size, bending over backwards to focus on physiological measures that made sense and to carefully examine the subgroups for which Medicaid should have made the most difference.

[8] The John Bates Clark Medal is awarded by the American Economic Association to "that American economist under the age of forty who is adjudged to have made a significant contribution to economic