Technology EvaluationCenter (TEC)

Special Report: Early Intensive Behavioral Intervention Based on Applied Behavior Analysis among Children with Autism Spectrum Disorders

EXECUTIVE SUMMARY

Background: In recent years, public attention has focused on the number of children diagnosed with autism spectrum disorders or ASDs, which include autism, Asperger’s syndrome, and “pervasive developmental disorder—not otherwise specified” or PDD-NOS. The estimated prevalence in the U.S. is about 1 in 151 children 8 years of age. In December 2007, Congress authorized $162 million for fiscal year 2008 for autism spectrum disorders, to fund the Combating Autism Act. Traditionally, behavioral and educational interventions for children with ASDs have fallen within the domain of public education systems.

Early Intensive Behavioral Intervention based on Applied Behavior Analysis or ABA (hereafter referred to as “EIBI”) is among the most commonly cited and best-researched interventions for these children. EIBI was championed by researchers led by Lovaas at UCLA in the 1970s and 1980s. This Special Report is a systematic review on the effectiveness of EIBI; the intent is to inform the public discussion. Other uses of ABA are not addressed in this Report.

Children with autism spectrum disorders face many difficulties; this condition is a challenge for them and for their families. This Report systematically reviews the evidence on EIBI because it is important to know which interventions are most effective, especially for conditions like ASDs that have such a profound impact on peoples’ lives. If we are not sure what works in treating any disorder and do not push forward with learning what does work, the people who are affected may potentially be deprived of benefit.

Objective: To conduct a systematic review of the research literature on the use of EIBI among young children with autism, pervasive developmental disorder, or Asperger’s syndrome. A systematic review “aims to identify all relevant primary research, undertake standardized appraisal of study quality, and synthesize the studies of acceptable quality.” Three questions are addressed in this Special Report:

Question 1. How effective is EIBI in improving the functioning of children with autism spectrum disorders, and how does it compare to other early intervention approaches?

Question 2. Can patient characteristics be identified that predict better outcomes from EIBI?

Question 3. Does the effect of EIBI vary with the intensity of treatment?

This Special Report is an update of an earlier version issued in 2009 (Vol. 25, No. 9).

Search Strategy: A literature search was conducted of articles in peer-reviewed journals published between 1966 (MEDLINE®) or 1970 (PsychLit®) and July 2008; this search was updated in October 2010. The search terms were related to autism, pervasive developmental disorders, (applied) behavior therapy, and other behavior modification (for the specific list, see text).

Selection Criteria: Selection criteria for this systematic review were that the study:

1) reported on the use of EIBI compared to another treatment strategy;

2) attempted to identify features of EIBI that had the most impact on its effectiveness; or

3) sought to identify children most likely to benefit from EIBI.

Exclusion criteria were:

1) the sample size was less than 10, including single-subject studies;

2) the interventions were very poorly described;

3) the interventions were not comprehensive, addressing a number of domains affected by ASD, but rather were narrowly focused, e.g., only on speech or play;

4) the intervention within a treatment group was heterogeneous, combining a variety of methods;

5) the experimental intervention was not intensive (i.e., less than 20 hours per week);

6) the study did not directly measure outcomes through a direct assessment of the child’s achievement but relied, for example, on telephone surveys with parents; and

7) the study was published before 1987, when the seminal article on EIBI by Lovaas was released.

Single-subject studies were excluded for two reasons. First, they aim to evaluate the effect of an intervention on a specific individual, rather than on a group of individuals with the condition and diverse characteristics. We are interested in the latter objective: how effective is EIBI among children with ASDs in the U.S.? Group designs, ideally randomized, controlled trials, but also nonrandomized comparative studies, are the only type that can address this question. Second, in a report in 2001 on small clinical trials, the Institute of Medicine identified a set of criteria to consider in deciding whether or not to perform single subject, which they call “n-of-1,” trials. Several aspects of researching EIBI among children with ASDs make it more difficult to derive valid results from the single-subject design, including the inability to use blinding, the variability of the condition over time, and the carryover effect. In single-subject designs, the goal is for the researcher to repeat a task or approach several times, alternated with other tasks, and to measure the impact of each. But if the effect of the first task is long lasting, as one would want it to be in children with ASDs, it makes it impossible to separate the impact of that first task from all subsequent tasks and, thereby, undermines the utility of this approach. As a result, the selection criteria for this Report are limited to group designs.

For the 2010 update, systematic reviews and meta-analyses published in 2008 or later were included if they graded the quality of the literature, since this is a key requirement for such study designs. A revised version of the AMSTAR tool was used to evaluate study quality.

Results: Nineteen studies (22 articles) were abstracted, including 2 randomized, controlled trials; 12 nonrandomized, comparative studies; and 5 single-arm studies. No studies were found that included children with Asperger’s syndrome; 4 studies explicitly included children with PDD or PDD-NOS. The 4 meta-analyses vary in their methodology, tools to assess study quality, studies included, and conclusions. Several of the authors note that combining such disparate studies undermines confidence in their conclusions. The 5 single-arm studies addressed Questions 2 and 3; this study design could not provide information on Question 1.

Overall, the quality and consistency of results of this body of evidence are weak. Consequently, no conclusions can be drawn from this literature on how well EIBI works. Weaknesses in research design and analysis, as well as inconsistent results across studies, undermine confidence in the reported results. It is important to distinguish between certainty about ineffectiveness and uncertainty about effectiveness. Based on the weakness of the available evidence, we are uncertain about the effectiveness of EIBI for ASDs.

Conclusions and recommendations for future studies are as follows:

Randomized, controlled trials: Autism and PPD-NOS are highly variable conditions, with substantial differences both in each child’s symptoms and progression over time. Since it is hard to measure those differences and to predict reliably who will do better in the short and long term, the best way to evaluate the impact of EIBI is to conduct randomized, controlled trials with enough children that any differences are likely to be evenly distributed across treatment groups. If children in one treatment group demonstrate more progress than children in comparison treatment groups, then we can be fairly confident in the results. Unfortunately, only two randomized, controlled trials have been conducted, and they compared different interventions, had small sample sizes, and came to different conclusions. Dawson et al. provide an example of a generally well-done, randomized, controlled trial among children with autism that focuses on a different intervention: the Early Start Denver Model.

Larger sample sizes: Only two studies included more than 50 children, and 11 had fewer than 30, including the 2 randomized, controlled trials. Given the variation in the presentation and progression of autism spectrum disorders, larger sample sizes are necessary to detect small to moderate differences between treatments. They are also required for the analyses needed to deal with potential analytical difficulties when children are not randomly assigned to a treatment.

Greater consistency: The types of treatment vary greatly, both within and across the available studies, especially for the control groups. Adding to the complexity is the use of different approaches to measure symptoms and progress over time. These factors make it difficult to interpret the results and to aggregate evidence across studies.

Longer follow-up: As the American Academy of Pediatrics has pointed out, autism spectrum disorders are generally chronic conditions. Children may progress at differing rates. While results of treatment after a year or two are relevant, only longer follow-up can demonstrate whether durable outcomes—with academic achievement and the ability to function at school and work—are attained. About half of the studies followed children for approximately 2 years or less, and some for only 1 year. This is not sufficient follow-up time to assess the potential impact of an intervention over a lifetime.

Incremental research strategy: For ethical reasons, randomized, controlled trials cannot be performed comparing EIBI to no treatment. However, they can be done comparing various aspects of treatment (e.g., length and intensity, behavioral versus other approaches, different combinations of behavioral strategies, type of person providing the intervention, extent of parental involvement, and setting). The results of such studies can be used to build an understanding of which strategies are most effective for which types of children with ASD, so that the benefit of the intervention can be optimized for each child, and time and effort will not be wasted on less-effective strategies.

The results for each question are summarized below.

Question 1: How effective is EIBI in improving the functioning of children with autism spectrum disorders, and how does it compare to other early intervention approaches?

The strongest evidence is provided by two randomized, controlled trials. However, weaknesses in research design, differences in the treatments and outcomes compared, and inconsistent results mean that the impact of EIBI versus other treatments on outcomes for children with autism cannot be determined. For example, Sallows and Graupner (2005) found that children in the experimental and control groups improved significantly over time, but there was no statistically significant difference in the rate of improvement between groups. Smith et al. (2000), in contrast, found that the experimental group had significantly better cognitive and communication skills than the control group at follow-up, but there was no difference between the groups’ adaptive skills.

The EIBI treatments used in the two studies also varied: For example, Sallows and Graupner compared groups receiving clinic-directed therapy for 39 hours per week in year 1 versus a parent-directed therapy averaging 32 hours per week. So both groups had fairly intensive, ABA-based therapy and differed on the precise hours of treatment and the intensity of supervision by more experienced staff. Smith et al., in contrast, compared a clinic-run program for about 25 hours per week versus special education classes for 10-15 hours per week combined with a parent training program. The evidence is insufficient to determine whether or not EIBI is more effective than alternative approaches for children with ASDs.

Question 2: Can patient characteristics be identified that predict better outcomes from EIBI?

Given the lack of a definitive answer to Question 1 on the relative effectiveness of EIBI, Question 2 on whether there are characteristics of children that predict a greater likelihood of success cannot be answered either. However, researchers have attempted to measure the relationship between specific characteristics and outcomes. The ideal method of identifying characteristics likely to predict treatment outcomes is to examine treatment by covariate (i.e., child characteristics) interaction terms in the context of a randomized, controlled trial. Such analyses can be performed for nonrandomized studies, but the evidence is weaker due to the uncertain influence of residual confounding. Single-arm studies can suggest candidate characteristics that could be investigated in future randomized, controlled trials. Therefore, the reported results should be interpreted with caution. Age and cognitive functioning at intake (usually measured by IQ) were the most commonly studied characteristics in the studies included in this review. Three of the 4 studies examining the impact of pretreatment cognitive functioning found that it significantly predicted outcomes, while one (a randomized, controlled trial) did not. The findings on age were more variable, with some studies suggesting that younger age at the start of therapy is a predictor of better outcomes, while others found no difference based on initial age.

Question 3: Does the effect of EIBI vary with the intensity of treatment?

The findings on whether more intense treatment leads to better outcomes were inconsistent, and no conclusions can be drawn. In a nonrandomized study, Lovaas and colleagues reported that outcomes were better in the more-intensive group (more than 40 hours per week versus 10 or fewer hours per week). This study has a number of limitations, however, including lack of randomization; use of multiple instruments to measure the same outcome; and focus on IQ and school placement, while overlooking other important outcomes such as socialization and communication. Sallows and Graupner randomized children into groups receiving about 39 versus 32 hours per week and detected no difference in outcomes across groups. However, the variation in intensity was not as great in this study, and there were other differences in the programs as well (e.g., clinic- versus parent-directed therapy programs).

Author’s Comments and Conclusions. The variability of presentation and progression among children with autism spectrum disorders, as well as potential differences in delivery of behavioral interventions, make this topic challenging to study. Nevertheless, given the importance of caring for children with ASD, additional research is needed to identify those characteristics of treatment—content, technique, intensity, starting and ending age, etc.—that maximize its effectiveness. Because of the challenges in launching a very large randomized trial and the ethical necessity to provide some treatment to the control group, this body of research needs to be built piece by piece, with a series of studies that investigate different components of the larger research question. For this to be effective, however, the overall quality of studies needs to be improved, including a greater emphasis on randomized, controlled trials, where at all possible; substantially larger sample sizes; uniformity of outcomes evaluated and instruments used to measure them; and consistent treatments that do not vary widely within treatment groups (i.e., experimental or control group).

The cost of continuing the current course of assuming that EIBI works may not be obvious. EIBI is costly financially for society and requires a large time commitment from children, their families, and their teachers or therapists. However, these programs may not appear to pose any harm for the children themselves. Nevertheless, the opportunity costs could be high, indeed, of providing suboptimal care to these children, simply because we as a society do not know what works best. The children may be treated with an intervention that is not as effective as the alternatives. And if we accept an intervention because it seems to work, without solid evidence, research on the alternatives or on how it can be improved is likely to be stifled.

_____________________________________________________________________

NOTICE OF PURPOSE: TEC Assessments are scientific opinions, provided solely for informational purposes. TEC Assessments should not be construed to suggest that the Blue Cross Blue Shield Association, Kaiser Permanente Medical Care Program or the TEC Program recommends, advocates, requires, encourages, or discourages any particular treatment, procedure, or service; any particular course of treatment, procedure, or service; or the payment or non-payment of the technology or technologies evaluated.