Sunday, April 11, 2004

I have written previously about some issues regarding antidepressant
medication. This prompted some questions about the effectiveness
of this kind of medication. It has occurred to me that the best
way to respond would be to explain the apparent paradox: antidepressant
medications (ADMs) are very widely used, yet many studies show only
modest benefit. Some studies show no benefit at all. So if
the drugs don't work very well, why are so many people taking them?

In order to understand this apparent paradox, it first is necessary to
understand where medical knowledge comes from, how it is evaluated for
usefulness, and how useful knowledge is applied to patient care.
This process has been formalized into what is known as Evidence-Based
Medicine. A good review, as concise as any ever could be, can be
found here.
To borrow their opening:

EBM is the
integration of clinical expertise, patient values, and the best
evidence into the decision making process for patient care. Clinical
expertise refers to the clinician's cumulated experience, education and
clinical skills. The patient brings to the encounter his or her own
personal and unique concerns, expectations, and values. The best
evidence is usually found in clinically relevant research that has been
conducted using sound methodology. (Sackett, D.)
[link added]

Much of what has been written about EBM is focused on the
interpretation of clinical evidence. Most good, relevant,
clinical evidence comes from treatment studies. Often, the
results of the studies appear to be fairly easy to interpret.
However, the outcome of the study can only be understood fully by
placing it in a clinical context. That is what Sackett means when
he talks about integrating clinical expertise, patient values, and the
best evidence. The evidence by itself does not mean much.
It has to be interpreted before it can be used. That is were the
challenge comes from: if you do not have clinical experience, how do
you interpret the research in a clinical context?

I will try to explain how a patient can place the clinical research
data into a clinical perspective. When trying to decide whether
to take a medication, the question of greatest interest is this: what will happen to me, in the future, when I
take this medication. The research does not answer this question
directly. Rather, it shows what happened to other people, in the past when
they took the medication. The problem here is obvious. How
do you use the information about other people, in the past, to try to
predict what will happen to you, in the future? In order to do
this, you have to understand something about how research is done, and
how it differs from clinical practice. This still does not enable
you to predict the future, but it can help you assess what is likely to happen (as opposed
to what will happen)
to you, in the future, if you take the medication.

Studies are always done according to a protocol. The protocols
are designed to make it hard to show a positive treatment effect.
The reason for this is that research always start with the assumption
is that the
medication has no effect. The burden of proof is on the
researcher, to show that the medication does, in fact, have an
effect. In practical terms, this means that there are several
aspects of the design of the study that make it hard to show a positive
effect from medication. The idea is that, if the research can
show
an effect -- even with the cards stacked against him or her -- then it
probably is a real effect.

In order to show a positive medication effect, the group of patients in
the study is divided into two groups. This is done by some method
of randomization, in order to control for variables. One group
gets the medication; the other gets a placebo. Otherwise, both
groups then
are treated exactly alike. The patients do not know whether they
are getting active drug or placebo, and the researchers who are
assessing them do know who is getting the drug.

The first method of stacking the deck is this: the research protocol
gives the doctor who prescribes the medication no choice about what
medication to give. Then, after the medication has been started,
any changes in the
dose have to be done according to a pre-established set of
guidelines. In order for the study to be valid, it is necessary
to eliminate as many variables as possible. The rigidity of the
dosing schedule helps to minimize the variables. However, it also
gives rise to the biggest difference between a research study and
routine clinical practice. In a clinical setting, the doctor
evaluates the patient, decides whether or not to give medication,
selects the best medication, and
adjusts it according to outcome.
To use an analogy, the research study is like a W.W.II bazooka: you
pull
the trigger, and whatever happens, happens. Maybe you hit the
target, maybe not. The use of medication in a clinical setting is
more like a optically-tracked,
wire-guided missile. You get to make adjustments after
you pull the trigger, so you are more likely to hit the target.
The rigidity of the design of the research study helps make sure the
results are interpretable, but it makes it harder to show a positive
outcome.

The second method of stacking the deck also is done in order to
minimize variables. Patients are selected for the study according
to rigid guidelines. This factor is more important that it may
seem, at first. Most
studies exclude everyone with multiple medical problems, active or
recent substance abuse, or anyone who is too young or too old.
They exclude people who are already taking certain medications.
Often there are so many selection criteria that only a small percentage
of patients will be enrolled. I once read an article in which the
authors determined that only about 15% of the people who applied to
participate in a study actually met the criteria to be included.
This means that most studies are done on a highly selected group of
people. This group will not be representative of the general
population. There actually were two selection processes: first,
patient select themselves whether to volunteer for the study. As
a result, the group of applicants is already different from the the
general population in a systematic way. Then, the population is
selected according to the study design. This results in
a relatively homogeneous group, but the group of patient selected for
the study differs from the general population in some important ways.

Often, people who volunteer for studies are people who are at least a
little bit desperate. Enrolling in a study entails a willingness
to accept an unknown risk. This means that people with mild cases
of an illness are not likely to apply to participate. Likewise,
people who already have had a good response to an existing treatment
are not likely to sign up for the study. What this means is that
the group of people in the study will, on average, consist of people
who are harder to treat than the average patient would be. In
most kinds of illness, milder form of the illness are more common
than severe forms. The mild cases tend to respond best to
treatment,
but they tend to not be included in the studies. As a
result, the process of selection makes it harder to show a positive
outcome.

So far, we have seen that there are two factors that make it harder to
show a good result with the medication, when a research study is
done. There are two more to consider.

The third method of stacking the deck is to use a statistical method
called a last-observation-carried-forward analysis (LOCF). This
means that if a patient drops out from the study, for any reason, or is
disqualified in the middle of the study, for any reason, the amount of
progress that the patient
had made at the time of the last observation is the amount of progress
used in the final analysis of the data. This tends to make it
harder to show a good treatment outcome, because often the patients who
dropped out would have shown a better response if they had stayed
on the medication for a longer period of time.

The fourth method of stacking the deck is to provide active treatment to
the patients who get placebo. This is something most people
assume is not true. It also is something that applies more in
psychiatry than in other fields. Placebo is supposed to mimic the
absence of treatment, right? No. Because the studies are
done on
humans, and humans have to be treated ethically, it is necessary to
educate all study participants about their diagnosis, the methods of
treatment available, why certain treatments might be chosen over
others, and so forth. In psychiatric illness, this educational
process has a positive
treatment effect by itself. Also, as patients are assessed at
regular intervals throughout the study, they are questioned about their
symptoms. This process tends to improve their insight.
Thus, the very act of asking questions has a therapeutic effect.
This works because the more people look inward to understand what is
happening to them, the more effectively they can devise coping
strategies. A discussion of this is embedded in the article, Finding the Signal
through the Noise: The Use of Surrogate Markers (by Sheldon
Preskorn; for masochists only). You might think that this
would not affect the outcome of the study, since the same factors apply
to the patients getting placebo and the patients getting active
medication. What Sheldon Preskorn point out in his article is
that the introduction of an active treatment into the placebo group
increases the statistical noise, which makes it harder to demonstrate a
treatment effect.

So far, I have written about four factors in the design of research
studies that can introduce a bias in the study results. In most
cases, the bias is going to make it harder to show that the medication
has a positive effect. Let me give some examples.
This should make the article more interesting and, I hope, help clarify
the topic.

My first day after medical school, I began specialty training. I
was assigned to a unit in a veteran's hospital. At the end of
three months there, before going on to my next placement, I sat down
with my supervisor to review the experience. I had counted up all
my patients, and divided them into three groups. I figured that
about a third of them had gotten a lot better, a third had gotten
somewhat better, and the remaining third really had not seemed to get
much benefit. To my surprise, my supervisor said that those
results were pretty good. He pointed out that people don't come
to the hospital, in general, unless they already have shown a lack of
progress to a less intensive treatment. So my entire patient
population had been pre-selected to include only people who had a fair
to poor prognosis.

In contrast, when I finished training, I spent part of my time in a
college counseling service. Only a small proportion of the
students at the clinic were referred to me. Most of them were
people who had seen a therapist for a while, had not shown much
improvement with psychotherapy alone, and who were sent to me as a
result. In that way, the student group also had been
pre-selected, but the starting population was overall a much healthier
group. They were young, usually did not have any medical
problems, did not have to worry about homelessness or hunger, and had
not had anyone shoot at them. Few had any substance abuse
problems; those who did, tended to have a short history of binge
drinking, not a long history of daily drinking. Also, as students
at a competitive college, they were relatively high-achieving
people. They also tended to be intelligent, and to be good
candidates for psychotherapy. Furthermore, because only the ones
who stuck with psychotherapy were referred to me, I tended to see
people who were more likely than average to stick with their
treatment. Also, because they continued to see a therapist while
I was prescribing medication, they had frequent contact, more incentive
to remain on the medication, and any problems that arose could be
spotted quickly. In short, this was an ideal setting in which to
demonstrate that the medication had a positive effect. Although I
did not formally keep track of the outcomes, I had the
impression that almost everyone who took an antidepressant got
significantly better.

These examples represent extreme ends of a spectrum. The veteran
population probably was, on average, more seriously ill than the
population in a typical drug study. The college kids were much
healthier. A drug company could do a study in a population
of college kids, and they could easily show a good result. They
don't do that, though, because nobody would take the study very
seriously. When doctors read the results of the study, they
try to see how closely the study population matches their own patient
population. Then they interpret the results in the context of
their own practice. It would be difficult for a patient to do
this, because it takes a lot of clinical experience to comprehend the
full spectrum of severity of illness.

At this point, we have seen that antidepressant drug studies are
designed to make it difficult to show a positive effect, and that the
probability of a medication response, in a group, depends greatly on
the nature of the group. This helps to explain why so many
studies show a relatively modest treatment effect. We also have
seen that doctors who treat patients in a clinical setting (as opposed
to a research setting) are free to select medications and make
treatment adjustments as they see fit. This gives the patients
treated in a clinical setting a better chance of getting a good
result.

There is one more factor to consider. When a drug is first
released, no one really knows how best to use it. Over time,
doctors learn more about which medications are better for which
patients, what adverse effects to look for, and how to manage those
adverse effects. They also learn more about how to adjust
dosages.

When a drug is first released, the manufacturer declares a certain
dosage range to be the recommended range. It seems as though the
initial recommendations are always wrong. This is because there
has not yet been enough experience with the drug to establish the ideal
range. Also, the manufacturer will always recommended a certain
starting dose. This is usually correct, for the type of patient
who was in the study. But clinicians often find that, with
experience, it is better to start some patients at higher doses, and
some at lower doses. This kind of clinical experience adds to the
effectiveness of the medication. It is still the same medication,
but the person prescribing it is able to prescribe it in a way that is
more effective and which lowers the risk of adverse
effects.

A case in point involves the popular antidepressant, Prozac
(fluoxetine.) What I am about to say is anecdotal, and may not be
100% accurate, but it illustrates the point. When Prozac was
introduced, the only other antidepressants had much higher
probabilities of causing unacceptable adverse effects. Eli
Lilly (the company) knew that their product had a lower adverse effect
burden. They also knew that the most frequent cause of treatment
failure with antidepressants was the use of inadequate doses. So
they calculated an initial recommended dose of 20mg. This was
intended to be enough for the majority of patients. They only
made a 20mg capsule. It was rumored that they knew this was more
than what some people would need, but they did not want to make it easy
to give too small of a dose. This, they thought, would improve
the results -- on average. They probably were right.
It is likely that millions of people were able to get a good result
from the medication as a result of this strategy. Otherwise, they
might have started at a lower dose, and given up before ever getting to
an adequate dose.

The problem was that some patients really need to start at a lower
dose. They get unacceptable adverse effects if started at a
higher dose. This tends to occur mostly in patients who have a
lot of anxiety. Even though Prozac can reduce anxiety, some
people get a transient worsening if they start at the full
dose. Consequently, some people ended up doing poorly with
Prozac, when they would have done better if started at 5 or 10mg.
I believe that the number of people who benefited from starting right
at 20mg was higher than the number who had problems because there were
no smaller doses available. Still, that is no comfort o the
people who had problems at the 20mg starting dose.

Doctors figured this out pretty quickly. For a while, if someone
had a lot of anxiety, I would have them open up the capsule, pour the
contents back and forth between the two halves until they were
approximately equal, plug the half-capsules with peanut butter, and
take the smaller dose that way. I also had some people dissolve
the capsule contents in apple juice (It has to be slightly acidic in
order to dissolve.) One cup of apple hjuice would yield four
2-ounce doses with 5mg in each dose. Some years later, Lilly came
out with a scored 10mg tablet. This made it simpler to start
people at the lower doses.

I do not mean to blame Lilly for this. It is likely that their
premarketing studies screened out people with depression and panic
disorder, or depression and generalized anxiety disorder, or depression
and posttraumatic stress disorder. This would have made sense, as
you want an homogenous group in the study. But such a population
is not typical in a typical setting. In routine office practice,
quite a lot of the depressed patients have an anxiety disorder as
well. There is no way that Lilly could have known about this
problem, because of the way the studies are constructed.
Furthermore, Lilly did not have any choice about the way they did the
early studies. If they had not screened the patients to be a
"pure culture" of depression, the FDA would have rejected the
study. Another complication is that, even after it was learned
that a smaller pill was needed, Lilly had to go through a complex and
expensive process of getting FDA approval for the smaller dose.

The next antidepressant to come out had two strengths right away.
Zoloft was released with a 50mg scored tablet, and a 100mg scored
tablet. The next one, Paxil, come out with 20mg, 30mg, and 40mg
tablets. Pfizer later started making 25mg scored Zoloft
Tablets. Smith-Klein Beecham (now part of GlaxoSmithKlein, or
GSK) came out with a 10mg Paxil tablet. All three now are
available in a liquid form that permits any dosage to be given
easily. No more apple juice.

This has turned out to be longer than I had hoped, but I think that
some of the issues are complex enough that there isn't any quick way to
explain them. Even with the length of this post, it is really an
oversimpification. In particular, the details of research design
and statistical evaluation of result are a lot more compicated than
what I presented here. I am hopeful that I hit the right balance
between simplicity and technical detail to be useful.

The link below goes to a dummy account that automatically forwards email to the Federal Trade Commission's spam reporting service. Don't use it unless
you are a robot. Instead, act like a human and figure out the real address from this: joseph/dot/j7uy5/at-sign/gmail/dot/com

The Corpus Callosum is an occasional journal of armchair musings, by an Ann Arbor reality-based, slightly-left-of-center regular guy who reserves the right to be highly irregular at times.
Topics: social commentary, neuroscience, politics, science news.
Mission: to develop connections between hard science and social science, using linear thinking and intuition; and to explore the relative merits of spontaneity vs. strategy.