Fun With Statistics – Apples to Kumquats Edition

A common statistic from those that believe health care in the U.S. could be "better" is infant mortality, where in many comparisons the U.S. does rank lower than other industrialized countries. I've read several articles showing how that and other statistics are significantly skewed by U.S. immigration policy – lots of relatively poor immigrants in the U.S. compared with far stricter (and actually enforced) immigration policies in other countries that limit immigration by wealth or education criteria. That doesn't make it right as health care should be accessible regardless of wealth, but it does skew numbers.

But here's an interesting analysis of the statistic itself – and a good lesson on why it's very important to not take a statistic at face value.

Just as the purpose of medical school is to teach students medicine and not statistics, the purpose of health care (at least in the U.S.) is to make patients healthy, not to make mortality rates "look good." On the other hand, nations with socialized medicine have significant political incentives to game their statistics, and many of them do.

The infant mortality statistic isn't reliable. It's a self-reported figure, and each nation defines a live birth differently.

Germany requires live births to weigh at least 500 grams (1.1 pounds). Switzerland requires them to be at least 30 centimeters (11.8 inches) long. France and Belgium only count babies born at 26 weeks or later.

Apples to oranges to kumquats. We're not exactly measuring the same thing, but we're generalizing the statistic across multiple methods, leading to incorrect conclusions. The author goes on to dispute how the supposedly great Cuban health care system suddenly doesn't look so great if you look at equal measurements, in one case by just matching data collection time periods. Not by a long shot.

So that's the lesson, no matter which stats are being analyzed. What are the underlying assumptions, the underlying data collection methods, and does the analysis accommodate or at least disclose that.