Do Psychotherapists Improve with Time and Experience?

The practice known as “routine outcome measurement,” or ROM, is resulting in the publication of some of the biggest and most clinically relevant psychotherapy studies in history. Freed from the limits of the randomized clinical trial, and accompanying obsession with manuals and methods, researchers are finally able to examine what happens in real world clinical practice.

A few weeks ago, I blogged about the largest study of psychotherapy ever published. More than 1,400 therapists participated. The progress of over 26,000 people (aged 16-95) treated over a 12 year period in primary care settings in the UK was tracked on an ongoing basis via ROM. The results? In an average of 8 visits, 60% of those treated by this diverse group of practitioners achieved both reliable and clinically significant change—results on par with tightly controlled RCT’s. The study is a stunning confirmation of the effectiveness of psychotherapy.

This week, another mega-study was accepted for publication in the Journal of Counseling Psychology. Once more,
ROM was involved. In this one, researchers Goldberg, Rousemanier, Miller, Whipple, Nielsen, Hoyt, and Wampold examined a large, naturalistic data set that included outcomes of 6500 clients treated by 170 practitioners whose results had been tracked an average of 5 years.

What might be a bit surprising is that the study found clinicians’ outcomes actually worsened with time and experience. That’s right. On average, the longer a therapist practiced, the less effective they became! Importantly, this finding remained even when controlling for several patient-level, caseload-level, and therapist-level characteristics, as well as when excluding several types of outliers.

Such findings are noteworthy for a number of reasons but chiefly because they contrast sharply with results from other, equally-large studies documenting that therapists see themselves as continuously developing in both knowledge and ability over the course of their careers. To be sure, the drop in performance reported by Goldberg and colleagues wasn’t steep. Rather, the pattern was a slow, inexorable decline from year to year.

Where, one can wonder, does the disconnect come from? How can therapists’ assessments of themselves and their work be so at odds with the facts? Especially considering, in the study by Goldberg and colleagues, participating clinicians had ongoing access to data regarding their effectiveness (or lack thereof) on real-time basis! Even the study I blogged about previously—the largest in history where outcomes of psychotherapy were shown to be quite positive—a staggering 40% of people treated experienced little or no change whatsoever. How can such findings be reconciled with others indicating that clinicians routinely overestimate their effectiveness by 65%?

Turns out, the boundary between “belief in the process” and “denial of reality” is remarkably fuzzy. Hope is a significant contributor to outcome—accounting for as much as 30% of the variance in results. At the same time, it becomes toxic when actual outcomes are distorted in a manner that causes practitioners to miss important opportunities to grow and develop—not to mention help more clients. Recall studies documenting that top performing therapists evince more of what researchers term, “professional self-doubt.” Said another way, they are less likely to see progress where none exists and more likely to values outcomes over therapeutic process.

What’s more, unlike their more average counterparts, highly effective practitioners actually become more effective with time and experience. In the article below, my colleagues and I at the International Center for Clinical Excellence identify several evidence-based steps any practitioner follow to match such results.

Let me know your thoughts.

Until next time,

Scott

Scott D. Miller, Ph.D.Registration is now open for our March Intensives in Chicago. Join colleagues from around the world for the FIT Advanced and the FIT Supervision workshops.

Comments

All true, but you must admit your passion for aggregating all things negative about therapists deserves the title of Therapy Troll yes? Your approach would be in vogue if the system allowed, it is brilliant, but with insurance companies, auditors and rigid policies it will not happen. Why not start there with your critique?

Hi Scott “et. al.”?
From this distance (Denmark) I love and enjoy your cont’d energetic insistence on transmitting good sane safe knowledge on what’s really happening!! As this piece.
I still have your excited “enjoy” in my ears attending my first FIT-seminar with Susanne Bargmann in Hørsholm. Enjoying applies to me still as our knowledge about our own both way possibilities accumulates.
So thanks and go on!
Yours sincerely
Piet

Scott, when you write that 60% ‘treated’ in the UK study showed ‘improvement’, how does that compare with the 2002 Hansen Lambert & Foreman study which gave us figures of 14% ‘recovered’, 21% ‘improved’, 57% ‘no change’, and 8% ‘deteriorated’? That UK study you site from the Brit J Psychiatry didn’t include the drop-outs did it? I accept the main findings of both of these recent studies – that therapy takes as long as therapy takes in the UK study, and that experience doesn’t equal more skilled therapists; but I’m dubious on the percentages improving in general. That new study was from a university counselling clinic – and the authors note that they seldom see SMIs and excluded the ‘early terminators’ from the ‘treated’ stats. I see you use the word ‘treated’ – perhaps wisely – as presumably most of the drop-outs didn’t feel ‘treated’ (although there might have been a % of one-case wonders).
I would like to know the numbers arriving at the front door of mental health services and the numbers leaving the back door ‘recovered’ – as all indications from the numbers now in prison with MH difficulties is suggesting that either a lot are not turning up or have become so disillusioned with MH services they end up elsewhere.

Scott, you’re starting to depress me with these posts! I’d like to think I’m still improving and although I have to admit I did not ready the article (yet) that you’re referencing, I wonder if complacency plays a part in the decline. I remember when I first started doing therapy in private practice, I was so nervous that I was also more alert to what was happening in sessions. I spent a lot of time before and after sessions thinking about what I wanted to do and how it went. Now that I’m more comfortable, I certainly don’t commit the same amount of time and mental energy to one session.

Sorry for that Gary! No, I don;’t think it’s complacency–at least not in the traditional sense. The bottom line is that you have to constantly and consistently work at “your edge” in order to grow and develop. It’s not an easy message, but applying the principles outlined will certainly be rewarding! Don’t despair.

After I read it, I realized that I can be wrong and humility remained to be a virtue.
It is a wake up call to me. I have sensed & experienced these things throughout my journey as a therapist, but I don’t always have enough insight to make sense of it.

1. A balance between Therapeutic outcome & Therapeutic process.
2. System theory: recognizing the small changes that could lead to bigger change. It creates hope and “trusting of the process”.
3. Spirituality: open to other possibilities, therapy is not the “be all and end all” answer, there are miracles of healing beyond one can comprehend. It remains me of faith and believing in the greater power other than myself lol.
4. When I left go of what I believe, and start to see and to understand my client’s perspective. My therapeutic approaches became a better match to my clients’ needs.
5. Treating my believes/clinical judgement/knowledge as hypotheses or theories rather than the absolute “truth/fact”.
6. Feedback informed treatment approach.
7. If it is not working, then it is time to get help from others to change.
8. We are human, we forget sometimes.

Trackbacks

[…] Believing in ourselves might seem like a good idea. Therapists effects exceed treatment effects, accounting for 3-7% of outcome variance. Who we are is more important than what we do. But, wait, therein lies another weird phenomenon. Professional self doubt seems to correlate with effectiveness. The new, fresh student filled with enthusiasm and hope and knowing that they don’t know, is more often more effective than the seasoned therapists who knows that they know. Supershrinks also doubt themselves more. Scott Miller says it more clearly than I can here. […]