Recent study documents the effectiveness of psychotherapy has been overstated: An example of an RFTM and PEBKAC problem

Not being a computer nerd, I’d never come across these expressions. My 14-year-old son was the first person I heard use the terms. He was referring to a problem I was having with my desktop computer. To be sure, I’m no Luddite. Still, “computer” will always be a second language for me.

With a restart and few clicks of the mouse, he resolved my issue. When I asked him to explain what had caused the problem–hoping either to avoid or be able to resolve a similar occurrences in the future–he quipped, “Dad, it’s an RTFM problem, most likely in the PEBKAC.”

“RTFM problem? In the PEBKAC?”

“Yeah,” he said with a laugh, then walked away.

Consulting Google, I quickly learned what the terms meant: Read the F%$&ing Manual as the Problem Exists Between the Keyboard and the Computer. Swallowing my pride (and a fair bit of irritation), I had to admit my son was right. I had not read the manual. I didn’t want to read the manual. I WANTED MY PROBLEM SOLVED! As a result, I’d spent an increasingly frustrating hour, first tinkering, then on the phone with less than helpful customer service representative.

So, what’s this got to with psychotherapy?

Over the weekend, the twittersphere lit up with posts about a story in the New York Times: Effectiveness of Talk Therapy is Overstated. The article reported on a new study which had found that psychotherapy was “25% less effective…than previously thought.”

The response to the story was swift, questioning, for the most part, motives and methodology:

Who published this study and why?

What kind of therapy was studied?

Why the emphasis on quantitative studies?

Why is the media always so negative about therapy?

Is this the whole picture?

The reaction is understandable. The headline and story are enough to give any practicing therapist pause. More so because, as I reviewed in my recent presentation at the Evolution of Psychotherapyconference, they are already working in an challenging practice environment. Rules and regulation are on the increase. Incomes are on the decline. They know the value of the work they do. They can see it in the people they treat. Instead of recognizing the value of the services offered, the effectiveness of the field, it’s methods, and practitioners are called into question.

The interaction with my son still fresh in my mind, I wondered, “Could this be a RTFM problem located in PEBKAC?”

Said another way, “Had anyone actually read the study?!” Despite assurances that “the facts are always friendly” from the likes of Carl Rogers, we know therapists don’t read research, for example. How do we know? RESEARCH!

In truth, the study is a merely an empirical call for more openness and transparency in publication of psychotherapy research. Inflated estimates of effectiveness help no one. Not practitioners. Not clients. Not the field.

The very same factors that lead the media to highlight the most attention-grabbing aspects of a news story, influence what gets submitted, reviewed, and published in scholarly journals. Sad, but true. To get the full picture–to determine “what really works”–results from all research–whether published or not–must be tracked and reported. Not surprisingly, when you get beyond the headlines, the story is almost always less dramatic and more nuanced.

Additionally, the research article contains some real gems! For example:

Psychotherapy was found clearly superior to a variety of placebo and no-treatment controls, including treatment-as-usual, pill-placebo, and non-specific control conditions.

Consistent with research reported on this blog, no differences in outcome were found between treatment approaches!

No differences in outcome were found between psychological treatments and anti-depressant medication.

Finally, the effect of psychotherapy plus medications was superior to anti-depressant medication alone.

What’s not to like?

And while we’re on the subject, this week another study was published. Click on the link and give it a read. I’m sure there will be no headline in the New York Times, despite the fact that it’s the largest psychotherapy outcome study in history! What are the results? RTFM!

Comments

OK, so your blurb got me reading your blog – well done. Please don’t waste my time in future with stuff that doesn’t add anything to my ability to help people, but promotes populist articles. It just undermines your credibility. (It’s 20.30 and I’ve been working since 09.00 and yes, I’m tired!).

Thanks for taking the time to read my blog. The article was based on a research study that documenting a decline in effectiveness. As I relate in the post, I’d received a number of emails asking me about the piece. Our ability to help our clients is already being limited by the interpretation of such articles by payers, regulatory bodies, and consumers of mental health services. I hope that by making practitioners aware, they will be able to address concerns before they cause problems. Best wishes in your work!

The Times uses ‘talk therapy’ ‘psychological treatment’ interchangeably. If Dreissen, Hollen, Bockting and colleagues included only studies of ‘talk therapy,’ that would be an even bigger problem than publication bias. Show me a clinician that limited and I’ll show you someone who should have never gotten licensed.

Hi Scott, I’d be interested to see what Jackie Sparks made of the first paper – I think she has an astute eye for picking up all sorts of research bias. With regards to the 2nd, well it takes as long as it takes to complete therapy sort of conclusion. But, and I may be wrong, it seems the UK NHS has in these CORE statistics the potential to tell us how many (what percentage of) people are entering the front door of their mental health services and exiting the service ‘recovered’ – versus those who either drop out, die, or become chronic users. Does CORE record if they are symptom & medication free, and in full-time work or study – as Seikkula is recording in his outcome reports?

I really appreciated all of the insight you give to psychotherapy. Specifically, you talk about how this type of therapy was found clearly superior to a variety of placebo and non-treatment controls, which is really surprising to me. I can see, though, that this is a way to get deeper insight on the patients condition and mindset and help the therapist to have a better grasp on how to treat and help the patient. Thus, because there is a better understanding of the problem, there will be better results. Thank you so much for sharing!