Dr. Ann Vander Stoep, Associate Professor, Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Department of Epidemiology, University of Washington School of Public Health, Seattle Children’s Research Institute, (206) 543-1538;

Drs. Vander Stoep, Rockhill, Schoenfelder, and Myers are with the Department of Psychiatry and Behavioral Sciences and Drs. Zhou and McCarty are with the Department of Pediatrics, all at the University of Washington, School of Medicine. Dr. Vander Stoep is with the Department of Epidemiology and Dr. Zhou is with the Department of Health Services, both in the School of Public Health. Drs. Vander Stoep, McCarty, Zhou, Rockhill, Schoenfelder, and Myers, are all affiliated with the Seattle Children’s Research Institute.

If telehealth is to be integrated into mainstream mental health practice for families of children with ADHD, interventions with demonstrated efficacy when conducted in person will need to be replicated using telehealth technologies (Baum, Epstein, & Kelleher, 2013).

The Children’s ADHD Telehealth Treatment Study (CATTS) was a community-based randomized controlled trial (RCT) that compared the effects of two service delivery models for improving the care and outcomes for children with ADHD and their caregivers. The CATTS trial tested the effectiveness of a hybrid telehealth service-delivery model with combined pharmacotherapy and caregiver behavior training for reducing children’s ADHD-related symptoms and caregivers’ distress (CATTS service delivery model) compared to treatment in primary care with a single teleconsultation session (augmented primary care: APC). Children in the CATTS service-delivery model achieved significantly greater improvements than children in the APC model for inattention, hyperactivity/impulsivity, oppositional/defiance, and adaptive functioning (Myers, et al., 2015). The current investigation compares caregiver responses to these two service delivery models. We hypothesized that (a) caregivers of children randomized to the CATTS service-delivery arm would experience greater reductions in distress compared to caregivers of children randomized to the APC arm, and that (b) a significant part of the effects on caregiver distress would be explained by improvement in child ADHD symptoms.

We propose mediation of caregiver outcomes by treatment-related improvements in child symptoms and behaviors – specifically, inattention, hyperactivity/impulsivity, and oppositional/defiant behaviors, as well as adaptive functioning – for several reasons. Oppositional/defiant behaviors in children with ADHD have been linked to parent distress separately from inattentive and hyperactive/impulsive symptoms (Podolski & Nigg, 2001) and typically constitute the reason for seeking treatment (Evans, Owens, Bunford, 2014). Behavioral interventions constitute the best evidence-base for the treatment of oppositional defiant behaviors (Fabiano et al., 2009; McMahon and Forehand, 2003) and transactional models support improvement in parent distress and functioning with improvement in disruptive child behaviors (Chronis, Chacko, Fabiano, Wymbs, & Pelham, 2004; Davies and Cummings, 1998). Therefore, we hypothesized that our caregiver behavior training intervention would be associated with reductions in children’s oppositional-defiant behaviors with a reciprocal reduction in caregiver distress and disaggregate these three types of ADHD-related symptoms in the current study.

Methods

The CATTS trial was approved by the Institutional Review Board of Seattle Children’s Research Institute and monitored by a Data Safety and Monitoring Board. Informed consent and assent were obtained from all caregivers and children who were included as participants in the study. The CATTS trial methods (Vander Stoep & Myers, 2013), interventions (McCarty, Vander Stoep, Violette, & Myers, 2015), and children’s outcomes (Myers, et al., 2015) have been described in detail elsewhere.

Site of Study

Between November 2009 and August 2012, we recruited children from the practices of 88 PCPs in seven communities in underserved areas of Washington and Oregon states (Myers et al., 2015; Vander Stoep & Myers, 2013). All study services were provided at community clinics that had high bandwidth, point-to-point videoconferencing connections (384 kbits/sec to 1.0 MB/sec).

Participant Population

Child participants included boys and girls between ages 5.5 and 12.9 years who met diagnostic criteria for ADHD (American Psychiatric Association, 1994), attended school, and resided with English-speaking caregivers who were their legal guardians. Exclusion criteria included being in state custody, unavailability of the legal guardian, or having medical, developmental, or psychiatric disorders that required interventions beyond the scope of the study (Vander Stoep & Myers, 2013).

Diagnostic Determination

Eligibility, recruitment and randomization procedures were determined in a three step process. First, we reviewed medical records to evaluate exclusionary criteria. Caregivers then completed the Child Behavior Checklist online (CBCL; Achenbach & Rescorla, 2001). If the CBCL diagnostic subscale for ADHD was above the cutoff (T > 65), caregivers met in-person at the community telehealth clinic with a CATTS therapist who administered informed consent and three modules of the Computerized Diagnostic Interview Schedule for Children-IV (CDISC-IV; Shaffer et al., 1996) to confirm a diagnosis of ADHD and to establish the presence of common comorbid conditions. Consistent with other approaches (Jensen et al., 2001; Lewczky, Garland, Hurlburt, Gearity, & Hough, 2003), we considered an anxiety disorder (AD) present if caregivers endorsed criteria on the CDISC module for generalized anxiety disorder or if the anxiety disorder diagnostic subscale on the CBCL showed a T-score > 70. We used the same approach to determine the presence of oppositional defiant disorder (ODD). Children were then administered assent. Caregivers completed a baseline assessment.

Randomization, Enrollment, and Assessment

We carried out block randomization with stratification by age group (5.5–9.9 vs. 10.0–12.9 years) and site (N=7). A statistician generated a set of random numbers with an allocation ratio of 1:1 for assignment in consecutive order to one of two service models: (1) the CATTS delivery model and (2) the APC model (Vander Stoep & Myers, 2013).

Caregivers completed the assessments remotely from personal computers with data entered directly into the outcomes database (Geyer et al., 2011) at baseline, and 4, 10, 19, and 25 weeks post-randomization. The 25-week assessment was timed for completion halfway between the end of the CATTS service delivery (22 weeks) and return to their PCP’s care (27 weeks). Overall, caregivers completed a mean of 4.8 ± .7 assessments; 86.5% of caregivers in the CATTS service arm and 90.2% in the control arm completed all five of the assessments. The CONSORT diagram is available elsewhere (Myers et al., 2015).

Description of the CATTS Service-Delivery Model

The CATTS service delivery model used a hybrid approach with telehealth and in-person services (Hilty & Yellowlees, 2015) to deliver to families six sessions of combined treatment (MTA Cooperative Group, 1999). We used synchronous videoconferencing to provide access to child psychiatry services including pharmacotherapy. We used multiple asynchronous telehealth technologies to enhance the skills of master’s level community therapists to deliver in-person an evidence-based caregiver behavior training intervention for children with ADHD (McCarty, et al., 2015). These skilled therapists could serve as an enduring resource to their communities. All sessions were recorded, and sessions were randomly selected to rate clinician adherence to their intervention protocols using checklists that outlined the essential treatment components. Telepsychiatrists and therapists showed high fidelity to their protocols (Myers, Vander Stoep, & Lobdell, 2013; Vander Stoep & Myers, 2013).

Telepsychiatry

The principal investigator (KM) developed a manual and trained child psychiatrists to deliver evidence-based diagnostic and pharmacological care for ADHD through videoconferencing based on the five ADHD algorithms in the Texas Children’s Medication Algorithm Project (TCMAP; Pliszka, et al., 2006). Telepsychiatrists were instructed to adjust medication to reach a treat-to-target goal of 50% reduction in ADHD-related symptoms as determined by caregiver ratings of the child’s current symptom severity (Rockhill, Tse, Fesinmeyer, Garcia, & Myers, in press). At each session, telepsychiatrists also delivered psychoeducation about the neurobiology of ADHD (Stahl, 2008). The modules included: (1) Role of Medication in ADHD Treatment; (2) Central Nervous System (CNS) Involvement: ADHD Symptoms and Treatment Focus; (3) CNS Involvement: The Prefrontal Cortex and Executive Functioning; (4) Conditions Comorbid with ADHD; (5) Long Term Course and Potential Consequences of ADHD; (6) Review and Implications for the Individual Child.

Caregiver Behavior Training

A PhD-level psychologist co-Investigator (CM) developed a protocol for training community therapists to deliver caregiver behavior training. A training manual described the core elements of each session with sample scripts for teaching skills to caregivers and a series of recorded intervention sessions with two volunteer families (McCarty, et al., 2015). The six-session behavior training protocol incorporated key elements of evidence-based parent training programs that have demonstrated effectiveness in reducing disruptive behaviors within the broad developmental range from 5 to 12 years old (Fabiano et al., 2009; McMahon & Forehand, 2003). Session modules included: (1) Understanding ADHD and Your Child: Understanding Antecedents and Consequences of Behavior; (2) School Advocacy; (3) Praising and Ignoring Skills; (4) Giving Clear Instructions and Follow-through; (5a) Timeout and Other Consequences (5.5–9 years old) and (5b) Point System for Behavior (10–12 years old); and (6) Putting it All Together.

During training, the master’s level therapists in each study community viewed digitized recordings of sessions with the two volunteer families accessed through an asynchronous secure website, reviewed these training cases with the psychologist, and practiced the intervention with two additional volunteer families recruited from their communities. After receiving this training, they began to implement the caregiver behavior training in-person to study participants. During the training sessions, therapists gave caregivers handouts and assigned skills to practice between sessions. The PhD psychologist carried out small group supervision of therapists via telephone.

Prior to each telepsychiatry session, the therapist shared clinical data, such as vital signs, rating scales, homework, and quiz scores with the telepsychiatrist through a website, “WebCATTS;” (Vander Stoep & Myers, 2013). Near the end of each telepsychiatry session, the therapist joined the videoconference to transition the family to the caregiver behavior training component. Between intervention sessions, therapists exchanged documents with the CATTS research staff through an asynchronous web portal and participated in weekly research team meetings through videoconferencing.

At the close of the combined session, the telepsychiatrist and therapist documented their clinical decision-making in WebCATTS which then collated the information into a report that was sent to the family and the PCP. Children returned to the care of the referring PCP following study participation with a final report that contained a summary of the child’s progress and recommendations for follow-up monitoring and treatment.

Primary Care Service Delivery Model Augmented with a Teleconsultation

We included an active teleconsultation control arm (APC model) for several reasons. Because the telepsychiatry consultation model was a well-established service in participating communities, it would not have been ethical to withhold this long-standing option from PCP’s. Further, the availability of two active arms was an attractive offering for PCP’s and families and aided recruitment in participating underserved communities (Baquet, Commiskey, Mullins, & Mishra, 2006). Finally, we anticipated that the comparison of two telehealth models would provide valuable information to stakeholders seeking to establish a telehealth service.

Children randomly assigned to the APC service delivery arm received a single consultation with a telepsychiatrist who then made treatment recommendations to the referring PCP. The PCPs implemented these recommendations at their discretion and were not restricted from implementing any interventions they deemed appropriate or referring children to other services.

Outcome Measures of Caregivers Distress

We assessed four domains: parenting stress, depression, caregiver strain, and family empowerment that reflect the challenges that caregivers experience in raising children with ADHD. All domains were measured with validated instruments (Rockhill, et al., 2013). Parenting stress was measured at the baseline and 25 week assessments. Other domains were measured at all five assessment points (baseline, 4, 10, 19, 25 weeks).

The Parenting Stress Index (PSI) assesses the types of stressors that caregivers experience in caring for children with special needs (Abidin, 1995). Stressors fall into categories of objective life circumstances (e.g., my spouse and I don’t do as many things together), the caregiver’s judgment of the child’s activity level, and the caregiver’s subjective feeling of being trapped by parenting responsibilities (e.g., I feel that my child’s needs control my life). For the CATTS trial, we administered the 20 items that comprise three of the PSI subscales: role restriction (seven items assessing caregiver perception that the child’s demands are a source of frustration and control or restrict their personal freedoms), isolation (six items assessing whether caregivers feel socially isolated from their peers, relatives and other supports); spouse (seven items assessing whether caregivers perceive that they are lacking the emotional and physical support of their significant other to manage the child).

High test-retest reliability and construct and predictive validity have been demonstrated for the PSI in multiple studies (Abidin, 1995), including differentiating between parents of children with and without developmental delays (Lafiosca, 1981) and yielding elevated scores for mothers of children with ADHD (Barkley, 1988; Barkley and Cunningham, 1980).

PSI items were rated on a five point Likert-type scale from 1 (strongly disagree) to 5 (strongly agree) with higher scores indicating more stress. The total score across 20 items was used in analyses. For caregivers who did not have a spouse, we prorated scores for the 13 role restriction and isolation items to a 20-item equivalent. In the CATTS sample, the internal consistency coefficient for the baseline total PSI score was .89.

The Caregiver Strain Questionnaire (CSQ) was developed within a system of care framework to assess the “demands, responsibilities, difficulties and negative psychic consequences of caring for a relative with special needs” (Brannan, Heflinger, & Bickman, 1997). Child behavior problems are proposed to be one of the strongest predictor of caregiver strain. The CSQ developers conceived of strain as resulting from the child’s emotional or behavioral disorders as filtered through the family’s perceptions and resources (Bickman et al. 1995; Brannan, et al., 1997). Thus, strain encompasses both the outwardly observable impact, as well as the less observable emotional impact of caring for a family member with special needs (Brannan & Heflinger, 2002; Sales, Greeno, Shear, & Anderson, 2004).

The CSQ has 21-items and two subscales: objective strain (e.g., missing work due to your child’s problems) and subjective strain (e.g., feeling sad as a result of your child’s problems). Good convergent validity has been demonstrated with regard to other indicators of caregiver well-being (Khanna et al., 2012). CSQ items have five response options ranging from 1 not at all a problem to 5 very much a problem reflecting how much of a problem each potential source of objective or subjective strain the caregiver experienced during the past 6 months due to the child’s problems. Higher scores indicated greater strain. In the CATTS trial, the internal consistency coefficient for the baseline CGSQ was .92.

The Family Empowerment Scale (FES) has 34 items that indicate caregivers’ understanding of their child’s needs and their ability to advocate for their child with mental health problems (Koren, DeChillo, & Friesen, 1992). As with the CSQ, the FES was developed within the system of care framework as a tool to assess the effectiveness of interventions or programs designed to increase the empowerment of parent or other family caregivers. The FES has three subscales: the family subscale queries caregivers about their ability to handle the child within the family context; the services subscale queries caregivers about their sense of competence in working with professionals; the community subscale includes items about caregivers’ ability to advocate for their children. The five response options range from 1 (never) to 5 (very often), with a higher score reflecting greater empowerment. The subscales are highly correlated (Koren, et al., 1992), indicating a core component of the caregiver’s sense of empowerment (Florian and Elad, 1998).

Psychometric testing of the FES has demonstrated high validity and reliability in samples of parents of children with serious emotional disturbance (Koren, et al., 1992) and ADHD (Singh et al., 1995) and covariation with child functioning over the course of mental health treatment (Resendez, Quist, & Matshazi, 2000). In the CATTS trial, the internal consistency coefficient for the baseline FES was .93.

In the CATTS trial, consistency coefficients for baseline ADHD and ODD symptom scores were high, ranging from .84 to .89, and for performance was moderate (.53). Concurrent validity of the item total of the VADRS-Parent and the ADHD symptom scores on the CDISC-IV was high (r = .79). Correlations between the total ADHD scores and the two component scores were very high (r > 0.90).

Statistical Analyses

We conducted intention-to-treat analyses, including all participants, irrespective of the number of treatment sessions they completed. Besides being the analytic approach that we had proposed prior to conducting the study, of children assigned to the CATTS arm, 85% completed five or six sessions of the six session program. Furthermore, when we compared the 15% of children exposed to 2–4 sessions with the 85% children with 5–6 sessions, we found no statistically significant between-group differences in child symptom or functioning scores at baseline, 19-weeks or 25-weeks.

We compared baseline demographics, comorbid conditions, and caregiver distress scores between the two study arms, using two-sample t-tests for continuous variables and Chi-square tests for categorical variables. We adjusted regression analyses by baseline characteristics that were found to differ significantly between the two arms. For each of the four caregiver scales, we discarded a score from analyses if the caregiver answered fewer than 75% of the items contributing to the total scale score. For measures with 0–25% items missing, we assigned to the missing items the mean value of the items the caregiver completed and calculated the total score accordingly.

To test the first hypothesis that caregivers of children assigned to the CATTS service-delivery condition would experience greater reductions in distress, we compared the trajectories of caregiver outcomes in the two arms using longitudinal data analyses. Given that multiple assessments were made within individual participants, and participants were nested within seven study sites, we chose multilevel mixed effects regression models to evaluate primary outcomes. Specifically, subject and site-specific random effects were initially included in the regression models to account for the within-subject and within-site correlations. The site random effect was dropped from the final models because its variance component was nearly zero when the subject random effect was included. In addition, we used robust standard error estimates for the inference. The mixed effects model approach has the advantage of utilizing all available data even if a participant missed assessments. In each regression model we included intervention status (CATTS vs APC), time (week 0, 4, 10, 19, and 25), and an intervention-by-time interaction term. Time was modelled as a discrete predictor. Our evaluation of the effectiveness of the CATTS intervention was based on the coefficient of the interaction terms, which estimated how absolute differences in scores from baseline to each post-baseline follow-up assessment differed across the CATTS intervention and comparison arms. We then visually depicted the outcomes for caregivers of children in the intervention and comparison conditions by plotting trajectories of mean caregiver distress scale scores and point-wise 95% confidence intervals for baseline and 4, 10, 19, 25 weeks post-randomization assessments. We calculated a Cohen’s d (Cohen, 1988) corrected for uneven groups to estimate effect sizes for baseline to 25-week changes in caregiver outcome scores.

To test the second hypothesis that the effects of the CATTS intervention on caregiver distress may be partially mediated by improvements in child clinical outcomes, we operationalized potential mediators as improvements in children’s clinical outcomes as changes from baseline to 19-week follow-up in children’s inattention symptoms, hyperactivity/impulsivity symptoms, oppositional-defiant symptoms, and role performance scores. We modeled each of the 25-week caregiver outcomes separately. Mediation models included the baseline scores on the relevant caregiver scale as covariates. Following the MacKinnon (2008, 2011) framework, we tested for mediation using path analysis and a structural equation modeling (SEM) approach.

McKinnon requires that two conditions be met for establishing mediation (McKinnon & Fairchild, 2009): 1) the intervention is significantly associated with mediator, and 2) the mediator is significantly associated with the outcome while controlling for intervention status. In a previously published paper, we established that the CATTS telehealth service delivery model was associated with greater improvement in each of the child outcomes that we incorporated into the mediation model (Myers, et al., 2015). Results of the test of the first hypothesis (described above) established whether the CATTS service delivery model was associated with an improvement in caregiver outcomes. We then estimated and tested whether improvement in child symptoms, individually and combined, significantly mediated this intervention effect. We used bootstrap estimates and bootstrap confidence intervals on the key coefficients to test the mediated effect of improvements in child symptoms and performance. We considered an intervention effect to be significantly mediated by a child outcome if its associated bootstrap confidence limits did not overlap with zero. We also estimated the percent of the intervention effects mediated by improvements in all the child outcomes combined. Mediation analyses were conducted using the SEM module in Stata 12.1 software.

Results

Participants, Baseline Scores, and Service Engagement

Baseline characteristics of the children and caregivers in the two study sample are summarized in Table 1. Eighty-eight (n=88) PCPs referred 223 children, predominantly boys (n=163; 73.0%), with mean age 9.25 (± 2.0) years, of European/White ancestry (n=204; 91.5%).

The demographic and clinical characteristics of child participants in the CATTS telehealth service delivery and APC arms were comparable at baseline with one exception. A higher proportion of children in the CATTS service delivery arm had ADHD with both comorbid disorders, and, therefore, we adjusted for comorbidity status in the regression analyses of child outcomes.

Caregivers were predominantly biological mothers (n=173; 77.6%), married or cohabitating (n=159; 71.6%), with some post high school education (n=145; 68.1%), and a median annual family income in the range of $35,000 to $75,000. Caregiver demographic characteristics at baseline were well balanced across the two service delivery arms; however, on clinical measures, caregivers in the CATTS service model showed significantly higher baseline PSI score. We adjusted regression analyses by baseline PSI scores. Baseline scores on the four caregiver scales were highly correlated. The bivariate correlation coefficient was particularly high between baseline PSI and CSQ scores (r=.67, p < .001).

Of a possible six combined intervention sessions, participants randomized to the CATTS service delivery model (n=111) attended an average of 5.2 (range 0 to 6) telepsychiatry sessions and an average of 5.1 (range 0 to 6 sessions) caregiver behavior training sessions.

Mediation of Caregiver Outcomes through Improvement in Child Symptoms

Bivariate correlation coefficients between 0–19-week improvements for each pair of child symptom and performance outcomes were statistically significant, ranging from −0.38 to 0.66. As shown in Table 3, a substantial proportion of the CATTS service model effects on caregiver distress occurred through the mechanism of reducing children’s symptoms and improving their role performance. The proportions of intervention effects on 25-week caregiver outcomes that were mediated via baseline to 19-week improvements in child symptoms and performance were: parenting stress (PSI; 41%), caregiver depression (PHQ-9; 48%), caregiver strain (CGSQ; 43%), and family empowerment (FES; 26%). For parenting stress and caregiver strain outcomes, total indirect effects of improvements in all child scales combined were statistically significant. Taking improvements in the four child clinical outcomes independently, only reduction in ODD symptoms made a statistically significant contribution towards the intervention effect on caregiver strain (See Table 3).

Mediation of service delivery effects on 25-wk caregiver outcomes by improvement in child symptoms and performance from baseline to 19-wks

Discussion

This paper reports the results of a randomized controlled trial comparing the effects of two service delivery models on reducing distress of caregivers of children with ADHD living in geographically underserved communities. The children and caregivers assigned to the CATTS service delivery model received multiple sessions of multi-modal treatment delivered remotely via a hybrid service model using videoconferencing with a psychiatrist and in-person services from a community therapist who was trained and supervised remotely using telehealth technologies. Families assigned to the APC model received treatment in primary care augmented by a single teleconsultation session. Over time, caregivers in both service models reported significantly decreased levels of distress and depression symptoms, and increased levels of family empowerment. Effects were significantly greater for caregivers in the CATTS service delivery model. Further, combined child symptom improvements mediated reductions in parenting stress and caregiver strain, and improvements in caregiver strain were significantly mediated by treatment-induced decreases in child ODD symptoms.

Prior studies have shown treatment engagement, continuity, and adherence to be low for families of children with ADHD (Adler & Neirenberg, 2010; Bussing et al., 2003). In the CATTS trial, the majority of families who were assigned to the CATTS service delivery arm attended the full six sessions of combined treatment, reflecting high acceptability and engagement (Myers, et al., 2015; Vander Stoep & Myers, 2013). This finding portends well for the ability of telehealth technologies to extend the reach of empirically-supported psychotherapeutic treatments to children and caregivers in geographically underserved communities. It also suggests next steps to build on the few small studies supporting the delivery of caregiver behavior training through synchronous videoconferencing (Reese, et al., 2012; Tse, McCarty, Vander Stoep, & Myers, 2015; Xie, et al., 2013).

In each of the six CATTS sessions, we used synchronous and asynchronous telehealth technologies to deliver evidence-based treatments directly to children with ADHD and their caregivers. This hybrid model parsimoniously utilized asynchronous telehealth technologies to enhance the skills of local therapists in evidence-based behavioral interventions who could then remain an enduring resource for their communities. The model used synchronous technologies to deliver the scarce resource of expert psychopharmacology from child psychiatrists. The primary target of the CATTS service delivery model was reduction in child ADHD-related symptoms. While the CATTS model did not incorporate interventions that directly targeted caregiver depression, stress, strain or empowerment, it was effective in improving these outcomes. Our findings are consistent with results of studies conducted in-person showing beneficial effects of behavior management training for caregivers of children with ADHD (Anastopoulos et al., 1993; Chronis, Jones, & Raggi, 2006; Owens et al., 2003; Pelham, Wheeler, & Chronis, 1998; Pisterman et al., 1992; Wells et al., 2000) and the benefits for both caregivers and children of providing combined interventions, particularly for children with ADHD and a comorbid disorder (Jensen et al., 2001; MTA Cooperative Group, 1999). These results suggest directions for future research that tests the feasibility and effectiveness of delivering parent-child interventions through telehealth technologies and assesses caregiver implementation of behavior training skills (Chronis et al., 2006; Chronis-Tuscano et al., 2013; Comer at al., 2014).

To provide context for interpreting the meaning of the changes in the caregiver outcomes scales observed in the CATTS trial, we evaluated caregiver distress start and endpoints in light of findings from prior research reporting mean scale scores in normative samples or in outcome studies. The baseline to 25-week decrease in mean item scores from M= 2.55 to M= 2.30 for the 20-item PSI among caregivers participating in the CATTS service delivery model reflected a greater percent change (9.8%) than the 14-month decreases from M= 2.28 to M= 2.12 (7.0%) reported for participants in the combined treatment group in the MTA study (Wells et al., 2000) on the 12-item (Short Form) Parenting Distress (PD) subscale of the PSI (PSI-PD Short Form; Abidin, 1990). More recently, Kolko and colleagues (2014) reported 6-month mean item decreases in the PSI-PD from M= 2.05 to M= 1.92 (6.3%) for participants in a pediatric collaborative care intervention for children with behavioral disorders.

Baseline PHQ-9 mean total scores for caregivers receiving the CATTS service model of M= 7.1 indicated, on average, mild depression. At 25-weeks the mean PHQ-9 score for these caregivers had fallen to M= 4.0, reflecting no depression (Kroenke and Spitzer, 2002).

The CSQ was developed for use in system of care research to characterize caregivers of children with serious emotional disturbance (SED). At baseline, mean item CSQ scores for caregivers in the CATTS service model, M= 2.4, were comparable to those endorsed by families of children with SED in the Fort Bragg study, M= 2.5, reflecting “a little to some” strain” (Brannan, et al., 1997). At 25-week assessment in the CATTS group, the mean item score had dropped to M= 1.8, a significant change, unlike families in the large Fort Bragg study who did not show a significant drop in caregiver strain after participating in a continuum of care service delivery model (Bickman, Lambert, Andrade, Penaloza, 2000). The overall 25.0% decrease from M= 2.4 to M= 1.8 in mean item CSQ scores over 25 weeks for caregivers in the CATTS service model compares favorably to the 20.8% decrease from M= 2.4 to M= 1.9 over 32 weeks for parents enrolled in a publicly funded mental health program (Accurso, Garland, Haines-Schlagel, Brookman-Frazee, & Baker-Ericzen, 2015) and to a small randomized trial that found no benefit for parents enrolled in a parent-to-parent support program for school children designated as having serious emotional disturbance (Kutash, Duchnowski, Green, & Ferron, 2011).

FES mean total baseline scores for caregivers in the CATTS group (M= 115.3) were more similar to the scores of caregivers in the original scale development study (Koren, et al., 1992) who were not involved in any advocacy activities (M= 125.9) than those of caregivers who were engaged in advocacy activities (M= 144.8). The increment of improvement in CATTS participants by 25-weeks to M= 127.9, while significant, did not bring the total mean score to the level observed among caregivers of children with SED who were recruited from parent advocacy organizations. Members of advocacy organizations would be expected to report higher empowerment scores than parents living in underserved communities raising similarly affected children. The final mean total FES score for the CATTS caregivers was comparable to the mean score for parents who received Wraparound services in a public mental health system for six months (M= 132.8) (Bruns, Pullmann, Sather, Brinson, & Ramey, 2015.

Taken together, this evaluation of caregiver outcomes in the CATTS group vis a vis prior studies that used the four scales revealed that caregivers who received services via the CATTS service delivery model experienced statistically and clinically significant improvements in their distress and empowerment.

The CATTS trial has implications beyond the effectiveness of telehealth service delivery. Our results contribute to the small body of research showing the beneficial effects for caregivers of improvement in children’s ADHD symptoms (Accurso, Garland, Haine-Schlagel, Brookman-Frazee, & Baker-Ericzen, 2015; Heath et al., 2015), and the study is the first to formally test these indirect effects. The findings suggest that effective treatment of child ADHD may counter the well-documented detrimental effects of child ADHD on caregiver mental health (Accurso, et al., 2015).

Future research should parse the differential benefits for caregivers of improvements in child ADHD symptoms versus ODD behaviors as our findings indicate that decreased ODD behaviors have particular salience for ameliorating the consequences of caring for a child with ADHD and ODD. As evidence-based interventions for ODD focus on behavior training, rather than pharmacotherapy, our results support the use of telehealth technologies in providing caregiver behavior training.

While not statistically significant, the mediational effect of improvement in child inattentive symptoms on CATTS-induced reduction in parenting stress was quite strong, and appears clinically relevant. When a child has difficulty with focus, forgetfulness, and disorganization during daily tasks, his/her caregiver is required to devote considerable time and resources to instructing and monitoring. Inattentive symptoms often attract less attention or concern than hyperactive or impulsive symptoms or ODD behaviors, but our mediational findings suggest the importance of adequately managing inattentive symptoms.

Over 40% of intervention effects on parenting stress, caregiver strain, and family empowerment were mediated by improvements in child symptoms and functioning, but a large proportion of intervention effects on caregiver distress were not. The caregiver training component of the CATTS intervention focused primarily on helping caregivers to manage child behaviors and to advocate for their children in school settings. Thus, it is plausible that the behavior training itself had a direct effect on caregiver distress through the acquisition of knowledge about ADHD, improved parent-child relationships, and/or perceived social support of the caregiver from the therapist. Such an interpretation is consistent with a recent meta-analysis finding that behavioral interventions have beneficial effects on parenting children with ADHD in studies that did not target parent functioning as a primary intervention goal (Daley et al, 2014). Future research is recommended that is designed to further clarify the effect of behavior training on caregiver distress and to elucidate other child, caregiver, and family-level factors that are potential mediators of caregiver outcomes.

Our study findings must be evaluated in light of several limitations. First, is the choice of comparison condition. Our APC comparison group set a high bar compared to treatment as usual in the community that is commonly used as the comparator in health services and intervention studies. Likely, the increment of improvement in outcomes rendered through the CATTS service delivery model would have been wider had treatment as usual served as the comparison. The benefit of the current approach is the demonstration of the added value for caregivers of providing a short term expert intervention over a single consultation to PCPs. Second, the broad referral base of 88 PCPs across seven sites with varied documentation and tracking methods precluded an accurate quantification of the additional services that families in either group may have received from mental health providers, schools, PCP’s or other community agencies over the course of the study. We did not document other service use on the part of participants. Had this information been available and incorporated into the analyses, it might have affected the interpretation of the contribution to parent well-being of services delivered via the CATTS model. The likelihood is quite low that families in these underserved communities had access to alternative behavioral health services where empirically-supported parent training was delivered with fidelity.

Third, the trial was designed to test the effectiveness of a telehealth service delivery model with a two-component, guideline-based intervention and, therefore, we cannot evaluate the separate contributions of the telepsychiatry and the caregiver behavior training components. The high rate of treatment adherence and completion of intervention sessions yielded insufficient variability in intervention content or dosage to perform post-hoc analyses to parse the relative benefits of the two treatment components. The upside of the current approach is the successful demonstration of use of a hybrid model that delivered both components of guideline-based care (medication and behavior training) in underserved locales.

Fourth, our 25 week assessment provided a short term follow-up half way between the last CATTS session (22 weeks) and the first scheduled PCP appointment (27-weeks) for children in the CATTS condition. The Multi-model Treatment of ADHD (MTA) study demonstrated that intervention groups started to regress to the mean within 24 months of treatment completion (The MTA Cooperative Group, 2004). We have insufficient evidence to suggest a different course for treatment offered through telehealth. Fifth, results may not be generalizable across all populations, as our participants represented a relatively homogeneous sample recruited from underserved communities in the Pacific Northwest (Table 1). Finally, the constructs used in the outcome and mediation analyses were all measured with caregiver-report instruments. Caregiver ratings of their children’s status may not be independent of the current emotional status of the caregiver and do not fully capture functioning of children in settings such as school or extracurricular activities. All parent distress measures were highly correlated. Our study findings did not point towards any differentiation among these measures.

Overall, the CATTS trial contributes new evidence to the emerging body of knowledge that supports the use of telehealth technologies to deliver evidence-based services to children with mental and behavioral disorders. This research highlights the process of how service delivery innovations work and is the first study to show that ADHD intervention leads to changes in caregiver outcomes that are mediated through changes in child symptoms and performance. This new evidence will spark conversations among stakeholders from advocacy organizations and primary care, mental health, and health policy sectors who are working to rectify disparities in access to evidence-based mental health care.

Acknowledgments

Funding

The CATTS study was supported by funding from the U.S. National Institute of Mental Health [1R01MH081997 and R01MH081997-04S1]. The authors declare that they have no conflict of interest.

Funding The design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation of the manuscript was supported by funding from the U.S. National Institute of Mental Health [1R01MH081997 and R01MH081997-04S1]. Funding for pilot data collection was provided by the University of Washington Institute of Translational Health Sciences, Small Pilot Project Grant program [566821]; the University of Washington Royalty Research Fund program [65-4020]; and the American Academy of Child and Adolescent Psychiatry Abramson Fund [506200020101]. The CATTS trial is registered with Clinical Trials: http://clinicaltrials.gov/show/NCT00830700.

Footnotes

Ethical Approval

All procedures performed involving human participants were in accordance with the ethical standards of the Institutional Review Board of Seattle Children’s Research Institute and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

Presentations Earlier versions of this paper were presented at the 16th Biennial Scientific Meeting of the International Society for Research in Child and Adolescent Psychopathology in Leuven, Belgium, June 13–16, 2013 and at the 60th Annual Meeting of the American Academy of Child and Adolescent Psychiatry in Orlando, FL, October 22–27, 2013. This research was supported by the National Institute of Mental Health [1R01MH081997 and R01MH081997-04S1]. The CATTS trial is registered with Clinical Trials: http://clinicaltrials.gov/show/NCT00830700.

Contributor Information

Dr. Ann Vander Stoep, Associate Professor, Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Department of Epidemiology, University of Washington School of Public Health, Seattle Children’s Research Institute, (206) 543-1538.

Dr. Carolyn A. McCarty, Research Associate Professor, Department of Pediatrics, University of Washington School of Medicine, Seattle Children’s Research Institute, Seattle, WA 98145-5005.

Dr. Chuan Zhou, Research Associate Professor, Department of Pediatrics, University of Washington School of Medicine, Adjunct Research Associate Professor, Department of Health Services, School of Pubic Health, Seattle Children’s Research Institute, Seattle, WA 98145.

Archambault R, Mansfield AK, Evans D, Keitner GI. Using the tenets of the Problem- Centered Systems Therapy of the Family (PCSTF) to teach the McMaster approach to family therapists. Family Process. 2014;53(4):640–655.[PubMed]

Barkley RA. The effects of methylphenidate on the interactions of preschool ADHD children with their mothers. Journal of the American Academy of Child & Adolescent Psychiatry. 1988;27(3):336–341.[PubMed]

Brannan AM, Heflinger CA, Bickman L. The Caregiver Strain Questionnaire measuring the impact on the family of living with a child with serious emotional disturbance. Journal of Emotional and Behavioral Disorders. 1997;5(4):212–222.

Brannan AM, Heflinger CA, Foster EM. The role of caregiver strain and other family variables in determining children’s use of mental health services. Journal of Emotional and Behavioral Disorders. 2003;11:77–91.

Evans SW, Sibley M, Serpell ZN. Changes in caregiver strain over time in young adolescents with ADHD: The role of oppositional and delinquent behavior. Journal of Attention Disorders. 2008;12(6):516–524.[PubMed]

Harpin VA. The effect of ADHD on the life of an individual, their family, and community from preschool to adult life. Archives of Disease in Childhood. 2005;90(suppl 1):i2–i7. doi: 10.1136/adc.2004.059006.[PMC free article][PubMed][Cross Ref]

Hilty DM, Yellowlees PM. Collaborative mental health services using multiple technologies: The new way to practice and a new standard of practice? Journal of the American Academy of Child & Adolescent Psychiatry. 2015;54(4):245–246.[PubMed]

Hinshaw SP. Moderators and mediators of treatment outcome for youth with ADHD: Understanding for whom and how interventions work. Ambulatory Pediatrics. 2007;7(1):91–100.[PubMed]

Hoagwood K, Kelleher KJ, Feil M, Comer DM. Treatment services for children with ADHD: A national perspective. Journal of the American Academy of Child & Adolescent Psychiatry. 2000;39(2):198–206.[PubMed]

MTA Cooperative Group. Moderators and mediators of treatment response for children with attention-deficit/hyperactivity disorder: The Multimodal Treatment Study of children with Attention-deficit/hyperactivity disorder. Archives of General Psychiatry. 1999;56(12):1088–1096.[PubMed]

Pliszka S. AACAP Work Group on Quality Issues. Practice parameter for the assessment and treatment of children and adolescents with attention-deficit/hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry. 2007;46(7):894–921.[PubMed]