Lies and Stats

I received an email informing me that my blog was included in a list of top stats blogs useful to people interested in health informatics. The following is the relevant excerpt from the full list of health informatics blogs.

Stats is very important when it comes to health informatics. Here are some good statistics blogs to help you with your health informatics learning.

Lies and Stats: It’s been a while since this blog was updated, but it still has a great deal of useful information about how statistics are used.

Overcoming Bias: Learn how to use data and how to overcome the biases that may be in that data.

The Endeavor: A great blog that includes statistics posts and information.

I have no idea who updates the health informatics blog, nor am I in anyway affiliated with them. It’s just a nice reminder and motivator for me to start writing blog posts again.

While on the topic, I decided to look at my blog stats, and was surprised to learn that my blog more than doubled in views from the month of my last post (in February 2009) to the next. It even maintained a steady number of hits per month, even though I haven’t been adding anything to it! Interestingly, note that hits seem to increase before fall and winter exams.

Visits to this blog by month (since May 2008).

Since this is a blog post originally about a list, and we’ve moved on to discuss my blog stats, how about my top five posts for 2009 (by title and number of views in 2009), which also surprised me a bit:

Clearly there are some things that are searched for on a regular basis, and maybe I can come up with a few more popular items while I’m tutoring stats this term. But I don’t think I could ever have predicted this top list, so I’ll probably just write topics of interest as they come up. Happy 2010!

Canadians often wait months or even years for necessary care. For some, the status quo has become so dire that they have turned to the courts for recourse. Several cases currently before provincial courts provide studies in what Americans could expect from government-run health insurance.

At least that’s story told by the Fraser Institute in an op-ed in the Wall Street Journal. “As we inch towards nationalized health care,” reads the subtitle, ” important lessons from north of the border.” With a couple of dire tales, and a couple of national averages, Americans are led to believe that introducing government-run public health insurance will drastically increase wait times in U.S. health care.

Where problems lieMaking an appropriate comparison between wait times in the U.S. and Canada is not trivial. How do you deal with those people that can’t get treatment in the U.S. because of inadequate or nonexistent medical insurance (infinite wait times)? Even comparing specific treatments is tricky because disease coding between the U.S. and Canada differs (ICD-9-CM is currently used in the U.S., and ICD-10-CA in Canada). And then you have to consider subgroups to see how population trends change for socioeconomic classes, say, and to ensure they aren’t reversed entirely (Simpson’s paradox).

Take, for example, a study that found that “socioeconomic status and breast cancer survival were directly associated in the U.S. cohort, but not in the Canadian cohort.” Also note that “this study replicated the finding of advantaged Canadian cancer survival in smaller metropolitan areas that had been consistently observed in larger metropolitan areas.” Although it’s possible there are other (confounding) factors influencing these results, it shows that socioeconomic status needs to be considered when comparing medical treatment and outcomes in the U.S. and Canada. And, therefore, it is likely to affect wait times as well.

Instead of dealing with technical details, however, the article in the WSJ uses stories in which Canadians wait months for treatment. There’s nothing inherently wrong with this—it is, after all, an op-ed piece and not a journal article—but you have to ask yourself about the choice of stories. Are they representative of public health care in Canada, or extreme cases? Also, we don’t know whether the individual that “paid for surgery that may have saved his life”, rather than wait for treatment in Canada, was in immediate need of treatment. These are, nonetheless, compelling stories that should not be disregarded—but they don’t prove a trend.

The basic argument put forward is that Canadians wait a long time for treatment under the public health care system. But what’s considered a “long” wait time, and how does it depend on the condition and severity? Notice that there’s no mention of wait times in the U.S., even for those that have appropriate health coverage. Instead we’re given some specific average wait times, but why cataract surgery or hip and knee replacements, and not others? How much do these wait times vary based on treatment, location, socioeconomic class, and how do they compare with U.S. figures? We’re left with more questions than answers.

The real confounderUltimately, to consider how wait times would increase in the U.S. with the introduction of publicly run, universal health coverage—that is, health coverage for all, as in Canada—there is one factor that would need to be disassociated from wait times in Canada. This factor, not unique to Canada but certainly rare, is not stressed enough in the article.

The Supreme Court of Canada found that Canadians suffer physically and psychologically while waiting for treatment in the public health-care system, and that the government monopoly on essential health services imposes a risk of death and irreparable harm.

Disregarding the inflamed rhetoric, the important point here is that there’s a “government monopoly on essential health services” in Canada. In other words, there’s no competing private system for health services deemed medically necessary, and the government funds and regulates the public health care system (although the government doesn’t operate it). You could probably argue that this monopoly is equivalent to price fixing for those services the government decides it’ll pay for. This is likely the main reason “care is rationed by waiting”—there is, after all, no alternative (besides paying for treatment in the U.S.).

It’s probably only a matter of time before Canada allows for a parallel private system for most, if not all, health services. Private spending currently represents about 30% of the average provinces total health care spending (mostly for medications and services not covered by the public system, such as dentists, optometrists, and physiotherapists). But until a parallel private system exists for all services in Canada, or the monopoly in essential services is taken into account, it’s disingenuous to suggest that wait times are simply because “individuals bear no direct responsibility for paying for their care.”

Bottom lineMany factors impact health care and wait times. You can’t look at just one aspect or descriptive statistic and know whether the system works as intended. It would be like judging a person’s health based on blood pressure alone. I agree with the author regarding comments he made in the past about improving Canada’s health care system. But making inferences into a public health care system in the U.S. based on the results from a couple of average wait times in Canada, where other factors confuse these results and make them unreliable to begin with, is inappropriate and misleading at best.

The current version of Donahue’s text is a 100 pages [PDF], but well worth a casual read (it’s not as bad as it sounds, as a lot of those pages are dedicated to visual displays of the ideas he is describing). If you enjoy reading Tufte’s opinions on the topic of displaying data, and you have to create charts and diagrams of statistical data, then you should enjoy Donahue’s writing as well. Reading Tufte a couple of years ago had a tremendous impact on my view of visual displays. But the focus here is in on statistical data.

The two fundamental acts of science, description and comparison, are facilitated via models. By models, we refer to ideas and explanations that do two things: describe past observations and predict future outcomes. […] Statistical models, then, allow us to describe past observation and predict future within the confines of our understanding of probability and randomness. Statistical models become tools for understanding sources of variation.

Typically, color ought be used for response variables, not design variables—but not always.

We understand the individual responses by comparing them to a distribution of like individuals.

Data presentation layouts and designs should be driven by intended use.

Time series make fine accounting but poor scientific models.

Each datum gets one glob of ink.

Naturally Donahue was also influenced by Tufte. As he says, “the idea of analysis is to understand the whole by decomposing into component parts.” And he therefore reminds the reader of Tufte’s principles of analytical design:

Show comparisons, contrasts, differences.

Show causality, mechanism, structure, explanation.

Show multivariate data; that is, show more than 1 or 2 variables.

Completely integrate words, numbers, images, diagrams.

Thoroughly describe the evidence.

Analytical presentations ultimately stand or fall depending on the quality, relevance, and integrity of their content.

Subscribe

About

Averages and relationships and trends and graphs are not always what they seem. There may be more in them than meets the eye, and there may be a good deal less.
—Darrel Huff in How to Lie With Statistics