The Deceptive Nature Of Averages In Obesity Management

Tuesday, January 12, 2016

Imagine there was a safe and effective treatment that worked great for 10% of people living with obesity.

While there would be no discernible benefit (or say even a small risk) for 90% of people with this chronic disease, the remaining 10% would not only experience a substantial weight loss (say 20% of their initial weight) but would also reap the benefits of the concomitant improvements in health and well-being.

Now imagine that in clinical practice, finding out for whom this treatment works and for whom it doesn’t is simply a matter of trying the treatment for a few weeks – if it works, great – if it doesn’t, well then you’re probably among the 90% for whom the treatment does not work – you simply discontinue the treatment.

The problem is, that even if such a treatment were to be developed, it would never find its way to market or into guideline recommendations.

The math is simple.

If you take 100 people each weighing 100 kg of whom only 10 will experience a 20 kg weight loss, the average weight loss for the group would be a rather modest -2 kg. No regulator would approve such a treatment.

Health economists will happily calculate that the benefits of a 2 kg weight loss do not warrant the cost.

Others may point to the side-effects (experienced even by many of the 90% who do not lose weight) and argue that the “risk/benefit” ratio for this treatment would speak against its use.

Payers would use these arguments to deny payment (even if the treatment does somehow find its way to market).

The only people who stand to lose are the 10% for whom this may well be the treatment that they’ve waited for all their life – a treatment that can reduce their morbidity and greatly improve their quality of life.

While 10% may not sound like a big number, applied to Canadians living with obesity, there would in fact be 700,000 Canadians who would potentially stand to benefit from this treatment (in the US the number would be around 8 million).

Unfortunately, these 700,000 Canadians will probably never see this treatment.

Imagine any other situation, where you potentially had a treatment that would significantly reduce the pain and suffering for 700,000 Canadians.

Now, imagine if this treatment were denied to them simply because the treatment does not work for “everyone” (or even “most”) people living with this condition.

I do fully appreciate the statistical and methodological issues with “responder analyses” – the loss of statistical power, the challenges for trial design, the non-randomized nature of the response, regression to the mean, arbitrary definition of “response” (especially for continuous variables), and more.

These are problems that are recognized and discussed – but so far, this discussion has not led to practical solutions in the clinical trial world (which is often remarkably different from what happens in actual clinical practice).

A logical approach would be to first to screen people for response to enrich the final study population with people for whom the treatment may actually work.

Additional trial design elements could include cross over designs, starting and stopping the treatment (in a blinded fashion if possible).

But the most important step is one of ideology – trialists (and regulators) must ask the question, how do we bring a treatment that only works for 10% of people with the disease (and we don’t know who these are without trying) to the 700,000 Canadians who stand to benefit.

Currently, the ideology appears to be focussed on keeping treatments that don’t work for 90% of patients off the market.

I could say the same for “experts” who belittle the potential benefits of current obesity treatments – by averaging the benefits across all participants (including everyone for whom the treatment does not work), we can ensure that no effective treatments will ever find their way to market.

Incidentally, this issue applies to all types of treatment.

Take for example exercise for weight loss – the overwhelming evidence is that average sustained weight loss with exercise is perhaps 1 or 2 kg at best. Yet, there are 1,000s of patients who will attest to the fact that exercise is what helped them lose weight and keep it off.

However, if anyone tried to get regulatory approval for exercise as a treatment for obesity, regulators would probably simply laugh them out of the room – that’s how ineffective exercise is ON AVERAGE! (the same could be said for most dietary treatments for obesity)

The notion that we will one day find a single treatment that works for every (or even most) patients with obesity is perhaps far too optimistic.

On the other hand, the notion that certain subsets of patients with obesity will benefit from some treatments (while others won’t) only reflects the complex and heterogenous nature of this condition.

4 Comments

Hi, Dr. S. I want to comment on the following line of reasoning (Note that I agree with everything else in the blog post):

“Now imagine that in clinical practice, finding out for whom this treatment works and for whom it doesn’t is simply a matter of trying the treatment for a few weeks – if it works, great – if it doesn’t, well then you’re probably among the 90% for whom the treatment does not work – you simply discontinue the treatment.

The problem is, that even if such a treatment were to be developed, it would never find its way to market or into guideline recommendations.”

Okay. No. The REAL problem is that no such treatment CAN be developed, because the issue isn’t weight loss. The issue is maintenance of weight loss. And that cannot be measured in two weeks. That’s something that has to play out over a long, long time. I would estimate that ninety percent (or more) of people CAN, in fact, lose weight. But according to empirical research, where subjects are weighed in a clinical setting, not surveyed sight unseen, the failure rate at 5 years for maintaining a 10% loss using behavior alone (and ignoring that that’s an unsatisfying goal for most patients to begin with) is 97 percent. In other words, you have a three percent success rate for an unsatisfying goal, and you can only know whether you’ll be one of the success stories if you invest five YEARS of your life. Ack.

Survey research shows a generous 20% success rate at five years.

This makes me ask: Would I agree to, say, a time-consuming heart procedure if it promised only a 20% success rate at five years? Hmmm. I might say, “Well, if it doesn’t hurt me, fine, I’ll try it.” Here’s the problem unique to obesity. Failure hurts like Hell. You lose weight and feel all kinds of wonderful . . . for a few weeks . . . then the weight gradually comes back on, often even if the patient is following doctors’ orders. (Though often doctors accuse their patients of being “noncompliant” based on the “proof” that they are regaining the weight. That’s an additional stinging pain that obese people feel on top of judgement from their families and sideward glances from their friends.) In order to keep weight off, one must restrict food intake and maintain activity levels at a pace comparable to what one was doing in the week the final pound came off, and periodically this regimen must be tweaked to a higher intensity as metabolism slows. Most people (80% or 97%, depending on which research you trust) are unable to do this.

Thanks Debra – yes you are right, sustained weight loss cannot be measured by short-term exposure so that adds a methodological challenge. I also appreciate that unsustained weight loss (or rather weight regain) hurts. And finally, yes, I should perhaps repost the escalator post. http://www.drsharma.ca/running-down-the-up-escalator