Never attribute to malice that which is adequately explained by stupidity.

03 August 2011

JAMA and the NYT don't understand PTSD

The New York Times has a new article about a recent study examining the adjunctive use of Risperdal in treating PTSD. Specifically, patients who were already taking a serotonin reuptake inhibitor either were, or were not, given Risperdal as well. Via a variety of metrics they were then assessed after six months to see if the Risperdal made a difference.

Let's begin where the article does, with its headline, "Drugs Found Ineffective for Veterans’ Stress". First of all, the study only examined one drug (granted, it did so in conjunction with a myriad of others, but its findings all relate to one drug), making the Times' use of the plural deeply wrong. In case you want to excuse the writer (Benedict Carey) and just blame the editor for a careless headline, here's the first sentence, which abuses the plural as well, "Drugs widely prescribed to treat severe post-traumatic stress symptoms for veterans are no more effective than placebos and come with serious side effects, including weight gain and fatigue, researchers reported on Tuesday."

Not only does the sentence abuse its subject to over sensationalize the story, the entire second half exaggerates the research findings upon which this article is based. The research author's only comments on the side effects of Risperdal were, "Adverse events associated with risperidone were not serious."
I don't intend to turn this into one of the hundreds of blog posts written by scientists that lament the sorry state of science "journalism" in the main stream media, though it could certainly be interpreted that way. I'm writing this because PTSD, particularly as it afflicts members of the same military, is often misunderstood and misrepresented in popular discourse. These misapprehensions inform policy and treatment decisions which have an enormous impact on those diagnosed with PTSD.

One of the core problems is what I see as a fundamental misunderstanding of what PTSD is. To begin with let's look at its name (I'm not trying to be pedantic here, but this is necessary). Post traumatic stress disorder. If I tell you someone has PTSD, what have you learned about them, at a basic level? In a 2009 article for Scientific American David Dobbs explained the diagnosis of PTSD thusly:

In the current American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders (DSM-IV), the first diagnostic criterion for post-traumatic stress disorder (PTSD) is having experienced trauma:

“The person has been exposed to a traumatic event in which both of the following have been present: (1) the person experienced, witnessed, or was confronted with an event or events that involved actual or threatened death or serious injury, or a threat to the physical integrity of self or others; (2) the person’s response involved intense fear, helplessness,
or horror.”

The presence of three clusters of symptoms—reexperiencing the event, for example, via
nightmares or flashbacks; numbing or withdrawal; and hyperarousal, evident in irritability, insomnia, aggression or poor concentration—for more than a month and to the extent that they cause “clinically significant distress or impairment in social, occupational or other important areas of functioning” completes the syndrome’s definition.

Critics of this diagnostic construct argue that the symptoms themselves can be characteristic of a wide array of other disorders and may appear together in people who have not experienced trauma.

PTSD was first defined in the DSM-III, published in 1980, in response to anti–Vietnam War psychiatrists and veterans who sought a diagnosis to recognize what they saw as the unique suffering of Vietnam vets.

What makes this construction different from nearly all other medical diagnoses (psychiatric ones included) is that it requires the external, traumatic event. In effect, this places causation for the disorder outside of the afflicted; yet paradoxically, it relies entirely upon the traumatic memory of the afflicted individual in order to inform the diagnosis. Dobbs explains the trouble with relying on memory:

When PTSD was first added to the DSM- III in 1980, traumatic memories were considered reasonably faithful recordings of actual events. But as research since then has repeatedly shown, memory is spectacularly unreliable and malleable. We routinely add or subtract people, details, settings and actions to and from our memories. We conflate, invent and edit.

To get back on track, what I'm trying to illustrate here is how immensely labile the diagnosis and symptoms of PTSD is and are. The reason this is important is illustrated in table 4 of the JAMA paper.

In this study the CAPS score was the main metric that the investigators used to assess whether adding Risperdal helped or not. As you can see, the total CAPS score mean difference does not have a p-value <= 0.05. However, the p-value for two of the subscales does show efficacy (i.e. a p-value <= 0.05). Here's what the study authors say about said finding:

Post hoc analyses of the CAPS, adjusted for multiple comparisons, suggested that risperidone was associated with a significant reduction in reexperiencing and hyperarousal symptoms associated with PTSD with a small effect size. Although the findings were significant statistically, these changes were smaller than the 0.5-SD threshold used to define the minimal important difference in estimating the sample size for this study. Thus, it is questionable whether the observed changes on these subscales would be detected clinically.

What's interesting about the last sentence of that explanation is that these differences were detected clinically. Here's how I know (from their methods section):

The primary outcome measure for this study was the total score on the 34-item CAPS.​ This scale was administered by trained raters who were blind to the randomization status of patients at baseline and weeks 6, 12, and 24. All raters underwent initial training and credentialing to administer and score the primary and secondary outcome measures. […] All raters showed 100% diagnostic accuracy at both sessions, and median scores were within 0.5 points and 3 points at the 2 annual follow-ups, respectively.

To translate that, what it says is: During this study, clinicians were trained to administer and determine CAPS scores. They were continually tested in order to ensure that they were able to accurately administer and score the questionnaire. In other words, the study authors doubt the findings of their own clinicians would be clinically detectable.

The authors, having not sufficiently undermined their hyperbolic abstract quite yet, went on (emphasis mine):

However, this study could not rule out the possibility that risperidone treatment addressed a real clinical need for some patients. The ability of risperidone to reduce reexperiencing and hyperarousal symptoms, such as disrupted sleep and autonomic arousal, is consistent with its ability to block 5-HT2A and α1 adrenergic receptors.​ This hypothesis is supported by the widespread prescription of trazodone, a 5-HT2 receptor antagonist, for sleep impairment associated with PTSD. It is also consistent with the increasing evidence of the efficacy of prazosin, an α1 adrenergic receptor antagonist, for treating reexperiencing and hyperarousal symptoms of PTSD.

Let's translate this paragraph too: An understanding of the pharmacology of Risperdal as well as a review of effective pharmacotherapy for two of the three CAPS subscales would indicate that Risperdal was likely to be an efficacious treatment. Make a note of their mention of trazodone here, we'll come back to it in just a moment. They weren't done undermining their headline quite yet though:

However, the lack of risperidone efficacy on avoidance/emotional numbing symptoms and the relatively greater efficacy for hyperarousal or reexperiencing symptoms appear to be consistent with findings of prior risperidone studies.

Before I move back to discussing the NYT's article, I'd like to briefly summarize the other "limitations" the authors noted about their study (emphasis mine, as usual):

"This study did not achieve the prespecified sample size of 410 patients projected for this study."

"Based on the 247 patients who completed the study and the prespecified factors in the power analysis, this study had 96.6% power to detect a 9-point difference in the ability of risperidone and placebo to reduce CAPS total score during treatment. […] However, even if the full projected sample had been recruited, this study most likely would not have yielded statistical significance for the small differential change in CAPS total scores produced by risperidone and placebo (3.74 points)."

"Entry criteria were relaxed because of recruitment problems; patients were accepted who had long-standing prescriptions of low doses of commonly prescribed sleep medications, particularly trazodone and quetiapine. […] Including these patients may have reduced the expected effects of risperidone in the current study." [I.e. some patients were, effectively, on more Risperdal than others]

"It is not clear that the findings generalize to other SGAs, such as olanzapine or quetiapine, that may have somewhat different clinical profiles in PTSD." [Keep this one in mind, we'll get back to it momentarily.]

As ought to be clear from the sheer number of bullet points here, this study had many flaws. A more accurate conclusion for their abstract would have been, "Among patients with military-related PTSD with SRI-resistant symptoms, 6-month treatment with risperidone compared with placebo only reduced particular types of PTSD symptoms." Naturally, such a conclusion is less headline worthy and less convenient to the overmedication zeitgeist.

Back to the NYT's shoddy reporting. Remember that note about how the results of this study would not be generalize-able to other second generation antipsychotics? Well, the NYT begs to differ with the study authors, citing "experts", "The new study, published in The Journal of the American Medical Association, focused on one medication, Risperdal. But experts said that its results most likely extend to the entire class, including drugs like Seroquel, Geodon and Abilify." If their expert (note the how slippery plurality seems to be in this article?) is Dr. Charles Hoge—mentioned in the next paragraph—I'd draw attention to this portion of his description: "a senior scientist at the Walter Reed Army Institute of Research, who was not involved in the study". To rephrase the part I bolded more appropriately, "who does not seem to have even read the study."

Worse yet, Dr. Hoge's contention that, “It definitely calls into question the use of antipsychotics in general for PTSD” would, in fact, contradict the findings of those same researchers who discussed precisely how antipsychotics could, and seem to, alleviate some PTSD symptoms. As we discussed earlier the panoply of symptoms and presentations in PTSD patients fully explains why these drugs may not be able to treat all of their symptoms.

Furthermore, an enormous, unremarked upon, issue confounding these studies is traumatic brain injury. To quote David Dobbs again, "Recent studies show that traumatic brain injuries from bomb blasts, common among soldiers in Iraq, produce symptoms almost indistinguishable from PTSD." Incidentally, the JAMA study the NYT purports to be basing their article on included 63 Iraq and Afghanistan veterans (23% of the sample size!).

All in all, this JAMA paper's quality is… questionable, at best. The NYT article covering it is quite simply atrocious. It doesn't seem like too much to ask that the NYT at least require their writers to read the paper in question, rather than simply the abstract, but evidently it is.

While it's easy to chalk this up as yet another pair of overhyped journal and newspaper articles; doing so misses a crucial point. Each time we clumsily lump all PTSD symptoms into the same bin we reenforce the notion that it is a discrete illness with a correspondingly discrete treatment. Such an outcome would be ideal, but is far from reality. It would be comical to write a journal (or newspaper!) article decrying cardiologists for not having a single drug with which to treat "heart problems". In the same way that "heart problems" encompasses a constellation of pathologies and symptoms which require different treatments, the same must be acknowledged of PTSD (and other mental health disorders, at large). Until then we can't expect to get an appropriate medical, or societal, handle on these issues.