“The truth will set you free, but first it will piss you off.”
– Gloria Steinem

by Jean Rhodes

Over the course of my career, I have met many practitioners who have launched what sound like amazing mentoring programs. The practitioners are often new to the field—passionate about their mission and eager to join the broader mentoring movement. As much as I find their commitment inspiring, I also confess to occasional skepticism about the return on their investment of time and resources. My doubts are fueled by the somewhat sobering data which, when plotted over the course of a long career, emerge as an irrefutable trend line.

More than 15 years ago, DuBois et al. (2002) conducted a comprehensive meta-analysis of youth mentoring programs that showed disappointingly small effects. Since then, at least 5 subsequent meta-analyses of youth mentoring have yielded strikingly similar results. Although there remains considerable variation across studies, the average effect size remains small by conventional benchmarks (Cohen, 1988)). This despite a rich and growing body of research and evaluation studies, a strong commitment on the part of MENTOR and other organizations to evidence, and advances in training and classification through the National Mentoring Resource Center. The resistance to improvement is particularly humbling when we compare mentoring program effect sizes with those that have been found in meta-analyses of other prevention programs for children and adolescents. For example, a meta-analysis of 177 prevention studies found effects ranging from moderate to large, depending on program type and target population (Durlak & Wells, 1997).Likewise, meta-analyses of youth psychotherapy, encompassing hundreds of studies, have reported strong mean effects, (Weisz, Sandler, Durlak, & Anton, 2005). Interestingly , however, Weisz et al. (2005) note that, “in settings in which therapists were able to use their clinical judgment to deliver treatment as they saw fit, not constrained by evidence-based interventions or manuals, and in which there was a comparison of their treatment to a control condition. Meta-analyses of these studies of usual clinical care have found effect sizes averaging about zero (see, e.g., Weisz, 2004; Weisz, Donenberg, Han, & Weiss, 1995), indicating no treatment benefit.” I’ll spare you more data, as your confirmation biases may have already kicked in. You may already be thinking “mentoring does work, the data must be wrong!” Such biases are understandable as they are grounded in your lived experiences with caring adults. But there is often a gulf between the influence of a natural mentor and that of a typical volunteer in a typical program.

There are many factors that have contributed to the state of affairs. One is the field’s uneven adherence to evidence-based practice. A recent study by Kupersmidt et al (2016) revealed that only around 40% of programs consistently follow the Elements of Effective Practice (EEP), despite research showing fewer unexpected match closures in programs that adhere to them. And it’s not just the EEP. There are promising strategies that, when carefully exported from other fields can raise the bar. McQuillan et al. (2016), for example, has demonstrated effect sizes of his mentoring approach that are quite impressive, as have others. Despite their availability, many programs continue to rely on home-grown approaches and curricula.

There also the lack of specificity. Many programs will endorse the general goals of promoting healthy development, reducing risk, and ameliorating youth dysfunction and disorder. But those are tall and rather unspecific orders. Depending on their circumstances, youth need dramatically different things from their mentoring experiences. Nonetheless, many programs continue to embrace a one-size fits all friendship approaches. In doing so, they are akin to a pediatrician who, irrespective of presenting problem, prescribes two aspirins to every child that visits. Children with low-grade fevers and muscle aches are likely to benefit from this generalist approach, and they may even be held up as anecdotal evidence that the treatment works. But, in the long-run, this cherry picking of positive effects ignores the many for whom aspirin had little or no lasting effect.

A more effective approach would be to first understand the needs of the youth and then provide targeted, intensive training to the volunteer or paraprofessional. This targeted, evidence-based approach will require precision, fidelity, and, perhaps most importantly, humility. Precision in identifying the needs or goals of the target population and the best strategies for addressing them; fidelity in implementing strategies that adhere as closely as possible to the program and conditions in which they were designed and evaluated; and the humility to refrain from reinventing the wheel when there are already rigorously developed and validated strategies.

8 Comments on "The truth will set you free, but first it will piss you off: Why we need new approaches to youth mentoring"

Thanks, Jean, for challenging our field to think through what are the aspects of mentoring that make it effective and for whom/when. The one size fits all approach is unrealistic and I hope this idea trickles down to the practice world.

For those who know both Jean and myself, you know I’ve been nagging her for a long time about her original theoretical model, the one that is very much in line with a one-size-fits-all approach to mentoring.
My frequent challenges to the traditional, long-term relationship-based view of mentoring were based on my own work that found support for an approach to mentoring that was deliberately shortened, involved re-matching mentors every semester, and was done entirely at the lunch table with a handful of peers constantly interjecting themselves into the conversation.

As we’ve learned more about Lunch Buddy mentoring, we find even greater support for Jean’s recent calls for precise articulation of the proximal factors that mentoring changes and how those changes can benefit youth. Once articulated, we can have a better idea of the type of mentoring program that would work best. I call this the “Other Match” in youth mentoring: The match between our model of risk and our model of change. Sam McQuillin and colleagues have done this quite well in their Academic Mentoring Program and we’re trying to do that with Lunch Buddy mentoring for bullied children.

I should note that traditional, long-term mentoring, in my view, is still an important option in our mentoring toolbox. Next month, my current mentoring match will have spanned 6 years. But a relationship-based approach to mentoring still requires clarity about how the relationship will benefit youth and careful delivery of those relationships through the support of the EEPs. With greater clarity and precision, we will have better success reproducing our outcomes, something that very few of us researchers have been able to do.

This is a fantastic article and I appreciate the comments above, particularly with respect to the notion that given the broad nature of mentoring we should be expecting fairly modest effects. What excites me about this observation is the possibility that research and praxis in mentoring could move in a more focused direction, with more tailored effectiveness and efficacy studies. For example, what about mentoring tailored specifically to, say, children who have lost a parent? Or children in protective services? Part of the challenge here, of course, lies in appropriately selecting and training a volunteer pool of mentors but perhaps we could think about more professionalized or para-professionalized mentors, similar to the approach utilized in Emory Cowen’s primary mental health project. One key aspect of mentoring is that it relies on the development of an effective but otherwise somewhat inchoate mentor-youth relationship where the “match” is everything — but anyone who has worked in youth services knows that some folks are simply better than other folks at developing those sorts of relationships with kids. So one task that falls to the people who study child/adolescent interventions is going to be figuring out how to measure and replicate the characteristics not just of impactful mentoring *programs* but also the characteristics of especially effective mentors themselves. If we can do that, we can starting thinking about more systematically selecting and training effective mentors and perhaps even professionalize the role. I hope that idea doesn’t run too anathema to the spirit of the mentoring approach, of course!

I well remember how exciting it was to see the results of P/PV’s randomized controlled trial of mentoring. Many of us had become numbed by the frequency of no difference findings and this was the first rigorous trial of a widely used youth development program that found it worked. Of course as Jean soon pointed out, the positive effects were mostly from mentored youth not declining as much with age as their control group counterparts. But the subsequent flowering of both mentoring programs and mentoring research has been heartening. One source of small effects, I think and Jean implies, is that mentoring is not a narrowly targeted intervention. Its effects are spread across domains and gains by some youth in one domain are diluted when other youth don’t change in that domain but gain in another.

But Jean is right, as usual. We need to take the challenging findings seriously and use them to guide improvement. We should also build research into program operation via improvement science and related disciplines.

Thanks Steve (so nice to see your comment!). I agree–it’s a broadly targeted intervention so we can expect only small effects. But it’s time to embrace improvement science and be more systematic about what we’re actually doing!

A great reminder. I have seen and heard of wonderful people with great ideas and a passion for youth mentoring who have begun programs without following the Elements of Effective Practice. I think there is a golden rule of never starting with something that ‘might’ mess with the lives of already vulnerable young people. I appreciate most especially that final paragraph. Thanks.

Post a Comment

by Jean Rhodes Early in my career, I developed a conceptual model of youth mentoring that, to my surprise, has been a remarkably durable and useful heuristic. It has been applied to formal and natural mentoring relationships and used to explain everything from short-term, goal-focused relationships with classroom volunteers to lifelong bonds with devoted grandparents. […]

Search the Chronicle:

Welcome to the Chronicle, an online source for sharing new findings and ideas about youth mentoring. The goal of the Chronicle is to encourage active dialogue around evidence-based practice in
... Read More »