April 14, 2009

Given that one of the primary functions of Psychotherapy Brown Bag is to discuss empirically supported treatments (EST's) in an effort to raise awareness in the general public and demand for such services, I think it would be a useful exercise to discuss Scott Lilienfeld's (2007) ground breaking article on treatments that cause harm. After all, in making the case for the importance of data driven treatments, it makes sense to emphasize the point that there are many common current practices that not only fail to attain the desired effect, but actually cause harm to clients. Unfortunately, as a field, we have done a poor job of publicizing these facts - that both effective and iatrogenic (harmful) treatments exist - and several of the treatments that appear on Lilienfeld's list are likely to surprise both the general public and practitioners. In writing this paper, Lilienfeld started by making the case for the importance of measuring failure as well as success in treatment. His first point was that, whereas the medical field has the Food and Drug Administration (FDA) to run Phase I and Phase II trials of treatments before they are disseminated to the public, no such policing agency exists for psychotherapies. As such, the field is left to police itself, which human nature tells us is an ineffective means for assuring safe practices. This is not to say that there are legions of therapists out there seeking to harm patients through faulty therapeutic techniques, but rather that we are not particularly good at making accurate subjective evaluations of the efficacy of approaches we have been trained in and have used for years and to which we feel philosophically attached. Given the lack of policing agency, data thus needs to be our judge and studies that examine whether therapies accomplish their goals are imperative. Lilienfeld next addressed the idea of the Dodo Bird Hypothesis (Wampold et al., 1997). The Dodo Bird Hypothesis is a belief, based upon a meta-analysis, that all treatments work equally well. This finding is fueled by a multitude of problems. Primarily, the majority of the studies in the meta-analysis are comparing various forms of cognitive therapy, behavioral therapy, and cognitive behavioral therapy for anxiety disorders to one another. Not surprisingly, a variety of similar treatments for similar disorders performed equally well. It is unclear how such findings generalize to a statement that all treatments work equally well for all disorders. Although Lilienfeld did not go into great detail in criticizing this erroneous verdict (see Crits-Christoph, 1997 for a more thorough description of the problems inherent in the Dodo Bird Hypothesis), he did also mention that the primary point of the meta-analysis was off base. There is a mountain of evidence that particular treatments outperform other treatments for particular disorders in clinical populations. In other words, nobody is arguing that one treatment is superior for everything, but rather, that particular treatments (e.g., CBT for depression, exposure and response prevention for OCD; Chambless & Hollon, 1998) show particularly compelling results for specific disorders and should be seen as the front-line treatment for those particular disorders until controlled trials indicate that another form of therapy performs better. In explaining the importance of measuring harm in treatments, Lilienfeld pointed out that such assessments must cover a variety of domains. The reason for this is that harm can mean many different things. It could mean, among other things, that new symptoms appear, certain symptoms get worse, that the client experiences physical harm, or that the client resists future treatment for mental illness. As such, "harm" is a vague term that refers to a variety of undesirable outcomes. Additionally, before describing the list of treatments known to cause harm, Lilienfeld explained the three criteria that must be met in order for a treatment to be included on the list (list quote from p.57 of article):

They have exhibited harmful psychological or physical effects in clients or others (e.g., relatives)

The harmful effects are enduring and do not merely reflect a short-term exacerbation of symptoms during treatment

The harmful effects have been replicated by independent research teams

Points two and three are important to consider. In order to be included on the list, a treatment must actually cause long-term harm. In other words, minor fluctuations on a path from mental illness to health are not necessarily problematic. Additionally, finding such effects in a single study or in multiple studies run by the same individuals will not suffice. In order to be included on the list, a treatment must be shown to cause harm multiple times by research groups not affiliated with one another. It's actually quite distressing that a number of treatments do, in fact, meet these criteria. The fact that a list like this can include such a variety of interventions further highlights the need for EST's, as the evidence indicates that we are quite capable of making problems worse when we simply follow our gut in treating clients. In no other domains do we simply use gut impulses to produce the best results. Certainly, some individuals may benefit from some of the harmful treatments just as certain individuals may fail to benefit from the treatments with the most empirical support, but those individuals are outliers and we have no systematic way of determining who those individuals will be. By aiming to treat the outliers, we may in fact treat them more effectively, but in doing so would certainly treat the rest of the world less effectively, resulting in a net loss of effectiveness and a greater amount of lingering or even exacerbated psychopathology. Lilienfeld's list of potentially harmful treatments is as follows. For a more thorough description of each therapy and the evidence for its harmful effects, consult the Lilienfeld (2007) article:

It is important to remember that this is not simply a list of treatments that are not effective. That list would be much longer. This is a list of treatments that actually cause harm to some clients. Research like this is alarming, but it also can be seen in a positive light. The bottom line is, we can systematically assess the efficacy and effectiveness of particular therapies for particular disorders. Additionally, it is possible to research why certain individuals benefit from particular interventions and others do not. Determining whether certain aspects of therapist behavior, certain characteristics of clients, or certain clusters of symptoms increase the utility of particular approaches in specific environments would be invaluable. In the meantime, however, there is an obvious need to rely on the data to make treatment decisions.

Joye's series of articles on common factors across therapeutic modalities touches on this topic quite well and the Lilienfeld (2007) article itself provides a much more thorough description of the ideas mentioned in this post, but my hope is that this article prompts some conversation on this troubling situation. Why do you think that individuals (both clients and clinicians) resist the use of empirically supported treatments? Do you think there is validity to that position? Why do you think clinicians continue to use treatments that have been shown to cause harm? Why does the public not know about this phenomenon? What would be the best resolution to this situation?

For more information on empirically supported treatments, we recommend the following products:

Comments

Given that one of the primary functions of Psychotherapy Brown Bag is to discuss empirically supported treatments (EST's) in an effort to raise awareness in the general public and demand for such services, I think it would be a useful exercise to discuss Scott Lilienfeld's (2007) ground breaking article on treatments that cause harm. After all, in making the case for the importance of data driven treatments, it makes sense to emphasize the point that there are many common current practices that not only fail to attain the desired effect, but actually cause harm to clients. Unfortunately, as a field, we have done a poor job of publicizing these facts - that both effective and iatrogenic (harmful) treatments exist - and several of the treatments that appear on Lilienfeld's list are likely to surprise both the general public and practitioners. In writing this paper, Lilienfeld started by making the case for the importance of measuring failure as well as success in treatment. His first point was that, whereas the medical field has the Food and Drug Administration (FDA) to run Phase I and Phase II trials of treatments before they are disseminated to the public, no such policing agency exists for psychotherapies. As such, the field is left to police itself, which human nature tells us is an ineffective means for assuring safe practices. This is not to say that there are legions of therapists out there seeking to harm patients through faulty therapeutic techniques, but rather that we are not particularly good at making accurate subjective evaluations of the efficacy of approaches we have been trained in and have used for years and to which we feel philosophically attached. Given the lack of policing agency, data thus needs to be our judge and studies that examine whether therapies accomplish their goals are imperative. Lilienfeld next addressed the idea of the Dodo Bird Hypothesis (Wampold et al., 1997). The Dodo Bird Hypothesis is a belief, based upon a meta-analysis, that all treatments work equally well. This finding is fueled by a multitude of problems. Primarily, the majority of the studies in the meta-analysis are comparing various forms of cognitive therapy, behavioral therapy, and cognitive behavioral therapy for anxiety disorders to one another. Not surprisingly, a variety of similar treatments for similar disorders performed equally well. It is unclear how such findings generalize to a statement that all treatments work equally well for all disorders. Although Lilienfeld did not go into great detail in criticizing this erroneous verdict (see Crits-Christoph, 1997 for a more thorough description of the problems inherent in the Dodo Bird Hypothesis), he did also mention that the primary point of the meta-analysis was off base. There is a mountain of evidence that particular treatments outperform other treatments for particular disorders in clinical populations. In other words, nobody is arguing that one treatment is superior for everything, but rather, that particular treatments (e.g., CBT for depression, exposure and response prevention for OCD; Chambless & Hollon, 1998) show particularly compelling results for specific disorders and should be seen as the front-line treatment for those particular disorders until controlled trials indicate that another form of therapy performs better. In explaining the importance of measuring harm in treatments, Lilienfeld pointed out that such assessments must cover a variety of domains. The reason for this is that harm can mean many different things. It could mean, among other things, that new symptoms appear, certain symptoms get worse, that the client experiences physical harm, or that the client resists future treatment for mental illness. As such, "harm" is a vague term that refers to a variety of undesirable outcomes. Additionally, before describing the list of treatments known to cause harm, Lilienfeld explained the three criteria that must be met in order for a treatment to be included on the list (list quote from p.57 of article):

They have exhibited harmful psychological or physical effects in clients or others (e.g., relatives)

The harmful effects are enduring and do not merely reflect a short-term exacerbation of symptoms during treatment

The harmful effects have been replicated by independent research teams

Points two and three are important to consider. In order to be included on the list, a treatment must actually cause long-term harm. In other words, minor fluctuations on a path from mental illness to health are not necessarily problematic. Additionally, finding such effects in a single study or in multiple studies run by the same individuals will not suffice. In order to be included on the list, a treatment must be shown to cause harm multiple times by research groups not affiliated with one another. It's actually quite distressing that a number of treatments do, in fact, meet these criteria. The fact that a list like this can include such a variety of interventions further highlights the need for EST's, as the evidence indicates that we are quite capable of making problems worse when we simply follow our gut in treating clients. In no other domains do we simply use gut impulses to produce the best results. Certainly, some individuals may benefit from some of the harmful treatments just as certain individuals may fail to benefit from the treatments with the most empirical support, but those individuals are outliers and we have no systematic way of determining who those individuals will be. By aiming to treat the outliers, we may in fact treat them more effectively, but in doing so would certainly treat the rest of the world less effectively, resulting in a net loss of effectiveness and a greater amount of lingering or even exacerbated psychopathology. Lilienfeld's list of potentially harmful treatments is as follows. For a more thorough description of each therapy and the evidence for its harmful effects, consult the Lilienfeld (2007) article:

It is important to remember that this is not simply a list of treatments that are not effective. That list would be much longer. This is a list of treatments that actually cause harm to some clients. Research like this is alarming, but it also can be seen in a positive light. The bottom line is, we can systematically assess the efficacy and effectiveness of particular therapies for particular disorders. Additionally, it is possible to research why certain individuals benefit from particular interventions and others do not. Determining whether certain aspects of therapist behavior, certain characteristics of clients, or certain clusters of symptoms increase the utility of particular approaches in specific environments would be invaluable. In the meantime, however, there is an obvious need to rely on the data to make treatment decisions.

Joye's series of articles on common factors across therapeutic modalities touches on this topic quite well and the Lilienfeld (2007) article itself provides a much more thorough description of the ideas mentioned in this post, but my hope is that this article prompts some conversation on this troubling situation. Why do you think that individuals (both clients and clinicians) resist the use of empirically supported treatments? Do you think there is validity to that position? Why do you think clinicians continue to use treatments that have been shown to cause harm? Why does the public not know about this phenomenon? What would be the best resolution to this situation?

For more information on empirically supported treatments, we recommend the following products:

What's in a name

Traditionally, brown bag seminars are informal lunchtime meetings held by researchers to update their colleagues on recent research findings. Psychotherapy Brown Bag attempts to serve a similar function, posting new information around lunchtime and hoping to foster intellectual conversations about research topics in an informal setting. The brown bag lunch is optional!
All opinions expressed on this website are those of the contributor(s) and do not necessarily reflect the views and beliefs of the University of Southern Mississippi

Recommended Products

Disclaimer

We ask that all comments are respectful of professional ethical standards such as those published by the American Psychological Association and the American Psychiatric Association. Any comments deemed inflammatory, unethical, or controversial will be removed from the site. That being said, we are not responsible for the views and opinions expressed by contributors.
Furthermore, this is a public website intended to promote conversation between colleagues. Posting of confidential information about specific psychotherapy clients is in violation of HIPAA guidelines and is prohibited. Such comments will be immediately removed from the site.
Finally, this website is not intended as a source of psychotherapy for individuals in need of services. If you are coming to this site in search of services, please visit our "EST clinics" page and contact a service provider near you. If you are currently in suicidal crisis, call 1-800-273-TALK (National Suicide Prevention Lifeline) or 911 immediately or go to the emergency room.