This entry is our analysis of a study considered particularly relevant to improving outcomes from drug or alcohol interventions in the UK. The original study was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text The Summary conveys the findings and views expressed in the study. Below is a commentary from Drug and Alcohol Findings.

Marsh J.C., Shin H-C, Cao D.Evaluation and Program Planning: 2010, 33(2), p. 81–90.Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Marsh at jmarsh@uchicago.edu. You could also try this alternative source.

From the comprehensive treatment process data collected by a major national US study emerges the important lesson that retention in itself is not an active ingredient in post-treatment outcomes but reflects influences such having one's needs met (especially important for women) and developing a good relationship with the service and your key worker.

Summary The US National Treatment Improvement Evaluation Study collected data between 1993 and 1995 from a nationally representative sample of treatment programmes funded by the US government, serving mainly vulnerable and underserved populations including minorities, pregnant women, young people, public housing residents, welfare recipients, and those involved in the criminal justice system. It remains one of the few treatment effectiveness studies to have collected detailed information from clients on their relationships with the treatment provider and their receipt of services.

The source study had interviewed 6593 patients when they started treatment. Of these, the featured analysis included 3027 from 59 services who had completed all intake, treatment discharge, and follow-up interviews, were not in prisons or jails, and had expressed some need for services at treatment intake. Mainly they were problem drug users who may also have been drinking heavily. A year after leaving treatment they had told researchers the number of days in the last 30 on which they had used the five most frequently used substances: alcohol; cannabis; crack cocaine; cocaine powder; and heroin. These use days were summed to give an index of the intensity of their polydrug use. The fact that the variables presumed to influence substance use were measured a year before means that they could have had a causal influence, but the study was unable to be sure of this because other influences could not be excluded.

Main findings

Models were constructed of how mechanisms and influences interrelated to influence retention and post-treatment substance use. One was made for the full sample, but the same model could not be applied to men and to women – the processes were too different.

For the full sample, post-treatment substance use was significantly lower when a greater proportion of the patient's needs had been met during treatment, they had a better relationship with the treatment service/keyworker, and they had stayed in treatment longer. Further upstream of the causal chain model, facilitating access to treatment by providing transportation and child care led to receipt of a greater range and intensity of substance abuse counselling and other treatment services, which in turn led to an improved relationship between service and client. Improved relationships then seemed to lead to patients staying longer in treatment, and both were related to having more of one's needs met.

Among the 'fixed' factors affecting these processes were that residential treatment led to longer retention and more counselling and other treatment services being received. The more educated the client the less their post-treatment substance use, while the greater use was before treatment the greater it tended to be afterwards.

The authors' conclusions

Several services and service delivery mechanisms – client-provider relationship, access services, substance use counselling and other treatment services, and services matched to needs – individually and collectively contributed to retention in treatment and reduced post-treatment substance use. While mechanisms affecting outcome are fundamentally the same for men and women, access services are particularly important for women, and a positive client-provider relationship is directly related to reduced substance use for men but only indirectly for women.

The study's modelling techniques helped establish not just which factor is related to another, but also which might be cause and which effect. These analyses suggested that a positive client-provider relationship leads to better matching of services to needs and to longer treatment, but not the other way round. The implication is that when providers develop a constructive relationship with clients, they are able to more effectively identify and meet service needs and encourage clients to remain in treatment. These findings are consistent with a review which concluded that in substance use treatment, client-provider relationships consistently predict retention but less consistently predict substance use outcomes.

Another question is whether the client-provider relationship is therapeutic in its own right, or primarily a vehicle for enhancing access to and impact of specific services. Modelling results suggest this relationship is both directly connected to reduced post-treatment drug use and indirectly related via greater need-service matching and longer treatment stays.

These linkages do however differ for men and women. The client-provider relationship is directly therapeutic for the total sample and for men, but among affects post-treatment substance use only via receipt of needed services, which for women more strongly predicts outcomes than the relationship and more strongly than for men. This may be because women start treatment with more needs, so addressing those is especially important.

The analysis was also consistent with receipt of transportation and child care leading to receipt of more substance use counselling/treatment services, which led to a higher quality client-provider relationship. It seems these access services may be a necessary precondition for receiving more substance use counselling, which in turn gives clients the opportunity to connect with their provider and work on treatment goal-setting, planning and bonding, which in turn means more needs are met, encouraging or enabling longer retention. For the total sample and for men, the result is to reduce post-treatment drinking and drug use.

Access services were identified in the models as especially valuable for women, perhaps because they are more likely to face barriers to treatment entry including lack of transportation and problems with child care. Not only does providing these facilitate access, it also improves treatment in ways which improve substance use outcomes.

Remaining in treatment for more than three months has been considered a robust predictor of reduced post-treatment substance use. The featured analysis found that when other variables are taken in to account and findings for men and women analysed separately, retention is no longer directly related to substance use, but acts via improved matching of services to needs. The implications are that treatment duration may in previous studies have served as a proxy for receipt of services.

These findings emerged from a resicted sample of treatment programmes and from a restricted sample of the patients who started treatment in those programmes; they may not be representative to all substance users in need of treatment. Findings from the source study derive mainly from public sector programmes serving lower-income groups and may not generalise to other programmes and caseloads. They also date from the early '90s, since when treatment services have changed, though perhaps not in ways which invalidate the core conclusions. Finally, the statistical techniques used to construct the models can eliminate some theories about how treatment works, but not confirm that the final model actually reflects cause and effect. This inherent limitation of the techniques is compounded by the fact that the main treatment process information was all collected at one time point (discharge from treatment); ideally the presumed causes would have been assessed before the presumed affects.

commentary Logical, intuitively appealing and in line with other research as the conclusions of the analysis are, it is important (as the authors acknowledge) to remember that the most which can be said is that the cause-effect explanations are consistent with the data. It cannot definitely be said that this is in reality how the treatment process worked for these US clients and services in the early '90s. Only studies which deliberately varied, for example, the quality of the client-provider relationship (while keeping everything else constant) could confirm that directly or indirectly this relationship causes the resulting changes in retention and substance use. Such studies are however not very feasible because they would mean deliberately consigning some patients to a substandard treatment experience and possible substandard outcomes in a situation where lives are at risk from relapse. Failing such studies, careful and sophisticated analyses such as the featured study offers, based on relatively comprehensive data from real-world treatment services, can help elucidate important variables affecting the success of addiction treatment. More methodological considerations below.

Findings from this analysis were foreshadowed by an analysis from the same source study focused on receipt of services matched to needs. It found that receiving services matched to need was associated with greater reductions in illegal drug use generally and use of the drug(s) in relation to which the patient had sought treatment. This was the case for each of the needs separately (except for mental health) and for the extent to which each individual's overall needs had been addressed. The strongest links were with housing and vocational help and among patients at residential services, where these particular needs were most likely to be addressed. Matching services to needs was linked to improved outcomes partly (but not entirely) via a link with increased retention. However, these associations were confined to the half of the patients with multiple needs across at least four out of the five domains.

More generally, research is supportive of the attempt to match the intensity and type of help to patients' needs, but studies are few and usually the impacts on substance use have been moderate. Research is strongest in regard to providing inpatient care and professional psychotherapy for patients with distinct but not disabling psychiatric problems who also have fewer 'recovery resources' in the form of employment opportunities and a supportive family. The relative prominence of research on psychiatric severity and psychotherapy may be a function of the comparative lack of investment in meeting patients' needs for housing and employment, which are also more difficult to engineer. Despite the difficulties, studies do suggest that providing such services improves outcomes in the targeted areas and also in respect of substance use problems.

A Findings review has explored the impact of (among other forms of practical help) the access services found so influential in the featured analysis – transport and help with the child care. It concluded that practical help to overcome access obstacles directly improves retention and also shows that the service is responsive and caring. Transport seemed most important for impoverished populations required to attend methadone services daily for supervised consumption. Direct help in the form of a driver and vehicle worked best, probably because it provides an escort and structures the patient's day. Providing transport was found to augment efforts to link patients to external agencies such as housing and employment services. The review also noted that for many women, child care is essential if they are to be attracted to and retained in treatment, especially in long-term residential care, but may not be used if it is unfamiliar or seems to threaten the mother's custody of the child. Beyond child care and transport, flexible and realistic opening hours and attendance requirements meant patients with unpredictable lives were not set up to fail, and allowed others to maintain normal family and working lives.

Methodological limitations

It is unclear whether characteristics of the patients before they started treatment influenced both retention and outcomes and treatment processes in ways which could have created what looked like a causal relationship between processes and outcomes. This is especially relevant in respect of variables not entirely under the control of the treatment service. For example, clients who were particularly motivated or well placed to overcome their dependence might have developed better therapeutic relationships, been more diligent in attending the service's counselling sessions, and attended 12-step meetings outside the service. They may also have stayed in treatment longer and done better in terms of controlling their substance use after treatment, but perhaps mainly because of their pre-existing attributes rather than treatment processes. Similarly, the measures of child care and transportation services were measures of the receipt of those services, not just their provision by the treatment agency. Receipt may reflect not just organisational concern to help patients come to treatment, but also the eagerness of the patient to ask for these services so they can attend.

Such considerations may explain (for example) why the counselling sessions scheduled by the treatment service and whether it provided on-site access to ancillary services (reflecting the organisation's provision, not their use by the patient) had no impact on retention and substance use a year after leaving treatment, while whether the patient actually attended counselling sessions and whether they actually accessed needed ancillary services (potentially reflecting their impetus to get better) did have direct or indirect impacts.

The restricted sample is also a concern. About 85% of patients starting treatment completed intake interviews for the source study. From this interviewed sample were eliminated another 2179 who did not complete subsequent interviews or expressed no needs at intake. The remaining sample constituted about 57% of all patients starting treatment (from whom were then excluded those in prison or jail). Loss to the sample was unlikely to have been entirely random. In so far as it was related to the factors used to account for treatment outcomes, estimates of the effects of these factors will be biased. For example, assume (not unrealistically) that patients who developed a very poor relationship with their providers not only tended to leave treatment very early but also failed to complete all research interviews. In this scenario the potential impact of this relationship on retention – estimated on the basis of the patients who did complete all interviews – will have been underestimated.

See this web site for more on the National Treatment Improvement Evaluation Study including a list of papers analysing the findings. Also available is the main report on the study published in 1997.