What’s Known on This Subject:

Integrated or collaborative care intervention models have revealed gains in provider care processes and outcomes in adult, child, and adolescent populations with mental health disorders. However optimistic, conclusions are not definitive due to methodologic limitations and a dearth of studies.

What This Study Adds:

This randomized trial provides further evidence for the efficacy of an on-site intervention (Doctor Office Collaborative Care) coordinated by care managers for children's behavior problems. The findings provide support for integrated behavioral health care using novel provider and caregiver outcomes.

The Services for Kids in Primary-care (SKIP) treatment research program (www.skipprogram.org) integrates personalized behavioral health services in practice settings serving pediatric patients. An initial randomized trial evaluated a protocol for on-site nurse-administered intervention (PONI) relative to enhanced usual care (EUC) in children with behavior problems.13 PONI involved co-location of a nonmental health nurse trained as a care manager (CM) to implement a modular intervention (eg, parenting, child social skills, family problem solving, and communication) with minimal PCP involvement. PONI was superior to EUC in improving service use, child health and individualized behavioral targets, and satisfaction, but both groups showed significant gains on other clinical outcomes. Participating PCPs desired a broader, more interactive, and flexible delivery system.

A second SKIP study sought to enhance the clinical efficacy of PONI by adapting the chronic care model to develop a more collaborative approach (Doctor Office Collaborative Care; DOCC). Mental health clinicians were trained as CMs to administer an expanded set of content modules to manage child anxiety (eg, monitoring, relaxation) to support ADHD medication management in collaboration with the PCP. A pilot study documented the feasibility, fidelity, and acute impact of DOCC for behavior problems, as well as comorbid ADHD and anxiety, relative to EUC.14 However, the study’s scope, sample size, and methods (eg, PCPs were randomly assigned, not practices) were limited.

Using PCP and family feedback, the content and care processes in DOCC were expanded to better address the principles of the chronic care model in the current study (Table 1). DOCC incorporated participatory management for soliciting staff and family input, an expanded curriculum for the management of ADHD and anxiety, training for PCPs in the ADHD care management protocol, and technology-guided assessment and consultation procedures. This effectiveness trial evaluates the benefits of this expanded DOCC model in 8 pediatric practices that were cluster randomized to DOCC or EUC. We hypothesized that DOCC would be associated with gains in service use, child and parent mental health outcomes, and consumer satisfaction, and greater change in pediatrician’s treatment attitudes and practices.

Methods

Settings and Participants

Practices

Study sites included 7 Children’s Community Pediatric practices and 1 general academic pediatric practice affiliated with Children’s Hospital of Pittsburgh. This study was approved by the University of Pittsburgh’s institutional review board. All PCPs and parents/legal guardians provided informed consent, and children provided assent.

Providers

A total of 74 of 75 available PCPs consisting of physicians (n = 67), certified nurse practitioners (n = 6), and physician assistants (n = 1) participated in the study. Most were women and white, with ages from 29 to 69. All but 2 were specialty-certified, and 29% had additional fellowship experience (Table 2), virtually all of which were in pediatrics or a pediatric subspecialty (eg, ambulatory pediatrics, pediatric environmental health).

CMs

Four Masters-level social workers with previous experience in outpatient or residential treatment were hired by the study to serve as CMs. They were trained over 4 months to deliver each treatment condition, and were supervised by a senior clinician with input from the study child and adolescent psychiatrist. Each CM was assigned to 2 practices (1 per condition each) and worked 2 days per week per practice.

Patients

Participating children (n = 321) were mostly boys and white. Ages averaged 8.0 years (Table 2). Most had a primary diagnosis of ADHD (64%) or disruptive behavior disorder (41%); 16% had comorbid anxiety disorder. Few (10%) participants received ADHD medication. Almost half received social assistance (eg, food stamps).

Screening and Recruitment

CMs conducted telephone screens by using the Pediatric Symptom Checklist 17 (PSC-1716) with caregivers of 5- to 12-year-old children referred by PCPs for behavior concerns, and invited those meeting the clinical cutoff (≥6th or 75th percentile) on the externalizing behavior subscale for an intake. Parents and children completed self-reports and clinical interviews identifying exclusions related to diagnosis (eg, bipolar disorder), emergent symptoms (eg, suicidal intent), or parallel treatments. Of 787 children referred for study consideration, 576 completed the screening procedures and met initial eligibility criteria, 353 completed a baseline assessment at intake, and 321 who met inclusion and no exclusion criteria agreed to participate and be randomly assigned (Fig 1). Randomization status was revealed after assessment.

Intervention Conditions

Because 4 of 8 practices were involved in previous outcome studies, practices were stratified by previous participation (no versus yes) and level of patient diversity (low versus high) before cluster-based randomization by the statistician. In both conditions, CMs contacted parents after baseline to identify individualized targets, review findings and treatment recommendations, provide brief psychoeducation, and discuss questions. Both parents and PCPs received written evaluation summaries. The clinical supervisor monitored the integrity of the intake, case presentation, and treatment delivery procedures by reviewing all completed assessments and progress notes on a weekly basis and listening to periodic treatment session audio files. All treatment fidelity feedback was reviewed with the CM each week, and specific suggestions were made to address any questions or performance issues (eg, further role plays, review of materials). Supervisor records indicated that >90% of all CM-delivered sessions received the highest overall fidelity rating on a 4-point scale (1 = poor/incomplete; 4 = very good/complete).

DOCC

These practices offered on-site behavioral health services delivered and/or coordinated by CMs with PCP involvement by using content modules for behavior problems, ADHD, and anxiety13 (Table 2). Most of the content modules targeting behavior problems were adapted from an evidence-based treatment, Alternatives for Families: A Cognitive Behavioral Therapy (www.afcbt.org), designed for families presenting with child behavior problems13,17,18 and/or exposure to physical abuse/discipline.13,19–21 These primary topics were reviewed with all caregivers (eg, psychoeducation, managing stress, promoting positive behavior, home programs) and children (eg, anger control, social skills). As applicable, the ADHD care management module incorporated behavioral and medication guidelines from the American Academy of Pediatrics1,2 (eg, rating scales, medication titration, monitoring of symptoms and side effects) that the CM reviewed with children and caregivers. The PCP was also directly involved with the CM and family in administering ADHD medication. For children with anxiety and fears, we incorporated cognitive behavioral therapy methods from a manual developed for primary care22 (eg, self monitoring, relaxation).

The intervention was designed to be delivered in a minimum of 6 and a maximum of 12 individual (child, caregiver) or joint/family sessions and within 6 months. Each session began with a review of the status of the child’s primary target behaviors (individualized goal attainment rating [IGAR]), which guided the duration and content of treatment. Based on this assessment, the CM’s activities generally included individual goal identification, patient self-management by using psychoeducational materials, delivery of content to children and caregivers, consultation with the PCP, collaboration with the office practice, and linkages with specialty services and the family (eg, calls to monitor treatment response). Services were considered completed if the family received at least 6 sessions and met its agreed upon goals. Families that needed continued care at the end of 12 sessions were referred for aftercare to a recommended provider. Disposition plans, including referral for continuity or aftercare, were reviewed with the child’s PCP and psychiatric consultant (PC). All recommendations were recorded in the medical record and study database.

EUC

After providing brief psychoeducation, the CM made a facilitated referral to a familiar local mental health provider who accepted the child’s insurance. CMs mailed assessment reports to providers and made follow-up calls to parents 2 weeks after referral. Children could also receive ADHD medication from their PCP.

Assessment Procedures

Two bachelors-level research associates unaware of treatment condition administered rating scales, interviews, and treatment response ratings (Table 3). Per intention to treat, all cases were followed. Assessment measures were collected at baseline, 6, 12, and 18 months. Different sources completed service use and treatment measures during and after treatment.

Processes of Care

CMs documented all activities performed for clients on a services provided log14 and completed a treatment summary report to document the parameters of treatment delivered by CMs (DOCC) or outside mental health providers (EUC).13 We computed an “any services” variable on the basis of responses to both measures.

Parents identified treatment goals for up to 4 child problems on an IGAR.14 At pretreatment, each problem and specific behavioral anchors of improvement were defined (eg, 1 = pretreatment severity, 3 = expected or acceptable improvement; 5 = exceeded expected improvement). Goals at pretreatment were rated a “1,” but any 1 to 5 rating could be used later.

The Clinical Global Impression-Severity (CGI-S) and Clinical Global Impression-Improvement (CGI-I) ratings were completed by a study CM who worked in a different practice and had no contact with the family to assess symptom severity at intake (CGI-S) and level of improvement at 6- and 12-month follow-ups (CGI-I)27,28 on a 7-point scale. Treatment response was defined as a CGI-I rating of 1 (very much improved) or 2 (much improved), with high interrater agreement with the treating CM’s rating (r = 0.92, P < .001). Finally, parents completed the Client Satisfaction Questionnaire-8 (CSQ-8) at discharge.

Power Analysis

For hierarchical linear modeling (HLM) analyses, we used power calculation methods from Raudenbush.32–34 A proposed sample size of 300 at baseline (with 20% attrition rate) with 10 clients per PCP, 30 PCPs, 4 time points, and a within-subject correlation of 0.10 to 0.06 (based on Kolko et al 201013) would provide >80% power for finding an effect size (ES) of d = .33 for α = .05 (2-sided) for group differences on outcome measures. ESs of 0.3 to 0.5 were found on key outcomes in our previous studies.13,14

Data Analysis

We first examined the equivalence of DOCC and EUC on demographic and baseline clinical characteristics by using t tests for dimensional variables and χ2 tests for categorical variables (Table 2). Outcome analyses used SPSS (IBM SPSS Statistics, IBM Corporation; Predictive Analytics Software [PASW] 18) and HLM-6.35 For child and parent outcomes, a piecewise growth curve modeling approach36 with an intercept representing baseline levels of functioning and 2 linear slope factors representing change over time was estimated for each family at the model’s first level. Time (assessment) was nested within participants (practitioners). Full maximum likelihood estimate was used. Cases with data for baseline and ≥1 other time point were retained. The level 1 equations for the unconditional models were Yti = π0i + π1i(pre-later) + π0i(follow-up) + eti, where Yti is the observed outcome at time t for participant i. The “pre-later” variable was coded 0, 1, 1, and 1 for the 4 time points. This pre-later slope is the change from baseline to postbaseline, and its coefficient reveals the change due to condition. The “follow-up” variable was coded as 0, 0, 1, and 2 for the 4 time points. This follow-up slope is the change during a 6-month period of the follow-up phase, and its coefficient reveals the change due to condition. We first ran piecewise models of our outcomes unrestricted at level 2 and then examined the effects of training by entering condition (DOCC = “1”; EUC = “0”) at level 2. Pre-later and follow-up are examples of cross-level interactions,37 wherein the level 2 variable, condition, affects the slope of a level 1 predictor.

For PCP outcomes, a simpler growth curve model with a single linear slope representing change over time was estimated for each PCP at the first model level. The level 1 equations for the unconditional models were Yti = π0i + π1i(time) + eti. The time variable was coded 0, 1, 2, and 3 for the 4 time points. This time slope is the change during a 6-month study period. All other characteristics matched the family models.

ES Calculations

ES calculations for cross-sectional analyses used calculations for standardized mean differences (d) that were conducted with the Practical Meta-Analysis Effect Size Calculator.38

Results

Group Equivalence

DOCC and EUC were comparable on all baseline PCP background and outcome variables, and family variables, with 2 exceptions (Table 2). DOCC had a higher proportion of parents who completed at least some college, but a lower proportion of families from practices with experience in a previous study. These findings highlight the initial equivalence of both conditions. Most PCPs in DOCC and EUC enrolled a family (87% vs 79%; P = .37), with a mean of 5.5 patients per PCP (SD = 5.2). Overall study retention was comparable (97% vs 93%; P = .48).

Processes of Care

The rate of any mental health service use was significantly higher for DOCC than EUC (Table 4). CMs in DOCC and EUC averaged 3.5 and 3.0 hours completing intakes, respectively, with more time spent in DOCC on psychoeducation, paperwork, and supervision (all Ps < .001). The mean session length for DOCC cases was 48 minutes (SD = 6.2). Among available reports, DOCC (versus EUC) providers reported more hours of service, longer duration of treatment, more outpatient work with the child and caregiver, and lower rates of referral. More DOCC cases completed treatment goals on time and were on medication at discharge, whereas more EUC cases left treatment early.

Child and Parent Outcomes

Table 5 presents the descriptive statistics for the primary child and parent outcome measures at each time point. ES values are included for 2 of 4 time points: 6-month to illustrate the magnitude of acute differences immediately after intervention, and 18-month to show the magnitude of differences at the study’s conclusion.

We first analyzed the severity of all problems and improvements in child health status and PedsQL. Using the pre-later model, both conditions revealed significant reductions for all 5 outcomes, but DOCC (versus EUC) revealed significantly greater reductions in behavior, hyperactivity, and internalizing problems (Table 6). In the follow-up model, significant changes over time were found only in severity of hyperactivity/impulsivity ratings. DOCC (versus EUC) did not reveal any significant changes over the follow-up phase on any of the 5 outcomes. The absence of significant follow-up differences does not mean differences in the pre-later model have disappeared. Rather, the earlier differences have not been altered during follow-up.

On the PSI-SF, DOCC (versus EUC) parents reported significantly greater reductions on all 3 subscales (parental distress, parent–child dysfunction, difficult child) using the pre-later model, and on the first 2 subscales on the basis of the follow-up model.

HLMs also documented higher VADPRS remission rates for both conditions in ADHD inattention and hyperactivity, but significantly greater remission for DOCC (versus EUC) in behavior problems and internalizing problems on the basis of the pre-later model. Remission rates for DOCC and EUC at posttreatment were as follows: behavior problems (71% vs 51%) and internalizing problems (76% vs 66%). Both conditions revealed greater remission in behavior problems on the basis of the follow-up model, but this was qualified by an interaction revealing higher remission since posttreatment of EUC than DOCC.

Using analysis of variance, the mean IGARs revealed significantly greater improvements for DOCC (versus EUC) at 6-, 12-, and 18-month follow-ups (Table 7). At baseline, DOCC and EUC had comparable proportions of children rated at each severity level on the CGI-S (P = .46), especially at the 2 lowest levels (2% vs 3%), but significantly more DOCC children were treatment responders (CGI-I) at the 6-month follow-up. Parents also reported greater service satisfaction with DOCC.

The provider practices survey revealed more change by DOCC (versus EUC) PCPs in management practices and perceived skill in treating behavior problems and ADHD, and their comfort in addressing comorbidities (Table 9). Perceived obstacles to mental health service availability in the practice were similar in DOCC and EUC on the MH-SKIP. As expected, EUC (versus DOCC) clinicians were significantly more likely to make outside referrals, whereas DOCC (versus EUC) clinicians reported greater perceived competence and effectiveness in delivering on-site behavioral health services over time. There were no significant changes over time or any condition × time interactions on the PBS total score or the 2 derived subsets of OSC subscales (practice cooperation/personal accomplishment, role conflict/overload).

Discussion

This randomized trial provides further support for the feasibility, benefits, and acceptability of an expanded on-site intervention on the basis of the chronic care model (DOCC) for children referred by their PCPs for behavior problems. Like our pilot study,14 implementation of DOCC by trained CMs improved service access, child and caregiver participation, and treatment completion (versus EUC), highlighting the utility of delivering behavioral health services in pediatric offices.10,11 DOCC improved mental health care by making counseling, medication management, and collaboration with PCPs and families more widely available.39,40

Both DOCC and EUC showed improved outcomes,13 but DOCC showed significantly greater reductions in the severity of behavior problems, hyperactivity, and internalizing problems, greater remission of behavior and internalizing problems, and a higher proportion of overall treatment responders. Further, DOCC parents reported significant reductions over time in ratings of child difficulty, parent–child dysfunctional interactions, and parental distress related to child behavior. These findings demonstrating enhanced child and parent benefits associated with collaborative care extend those reported in quality improvement interventions for child behavior problems,13,14 ADHD,6–8 adolescent depression,10,11 and other problems.5 In the follow-up period, EUC showed significantly greater remission since posttreatment in behavior problems than DOCC, which may reflect DOCC patients having achieved greater remission by the end of treatment.

As in our previous trials, individualized treatment goals (IGAR) showed greater improvement for DOCC at all 3 follow-ups. In contrast, fewer improvements were found on other measures, perhaps because the item content of these broad measures is less applicable to a given child.41 This pattern of findings highlights the potential of identifying individualized goals on methods that can compare outcomes across goals and guide selection of personalized intervention content.

PCPs reported no change in perceived burdens to treating mental health problems or adverse aspects of the organizational climate. As expected, DOCC PCPs reported greater treatment involvement in on-site service delivery than those in EUC, who were more likely to refer to outside providers. DOCC PCPs acknowledged greater treatment involvement, competency/effectiveness with behavior problem children, and ADHD medication management skills. Interestingly, these improvements were even more substantial during the follow-up period, suggesting that it may take time to achieve changes in attitudes and practices. The collaborative approach coordinated by CMs promoted PCP service involvement and continuity, especially around ADHD.

Among the study’s limitations, the broad array of clinical content modules (for behavior problems, ADHD, and anxiety) and care processes (eg, meetings with PCPs, weekly progress monitoring) in DOCC precludes evaluation of its components. Given group differences in content, duration, and other treatment parameters, future work could control for relationship or alliance effects. In addition, we had data missing from EUC providers, despite incentives and follow-up calls. The inclusion of more formal fidelity measures and teacher ratings would expand the objectivity of the assessment of provider practices and clinical outcomes, respectively.

We also recognize the need to explore the financing of collaborative care resources, as we chose to use grant funds to pay for the CMs to maximize fidelity to the program when implemented in a real-world clinical setting. Clearly, more research is needed to understand how practices adapt operational and financial strategies for sustaining key program resources, including focused training and technical assistance through the Replicating Effective Programs (REP) program,42 as well as discussions with state and local providers and stakeholders on a reimbursement model for care management activities so the clinics can absorb the costs.43 It is important to point out that the participating pediatric practices in this clinical trial later hired their own clinicians for on-site services after the trial had ended.

Conclusions

A collaborative care management model in pediatric practice (DOCC) enhanced access to and completion of behavioral health services, child and parental outcomes, consumer satisfaction, and provider practices, relative to EUC. The inclusion of standardized assessments with all PCPs provided novel feedback on key implementation outcomes. In 3 clinical trials conducted by the SKIP program, on-site care has shown advantages over facilitated referral to a local mental health provider. Unlike our pilot study, this study included PCP training in an expanded ADHD care management protocol,1 practice-based randomization to optimize PCP participation, technology to collect and share patient progress, and greater communication among CMs, PCPs, and families. Further efforts are needed to enhance primary care’s capacity to integrate and sustain collaborative care models for delivering high quality behavioral health services to children and adolescents.44,45 The incorporation of compelling implementation and financial models may help ensure that these evidence-based practices are transported to scale.39

Acknowledgments

We acknowledge the support of the research and clinical staff of SKIP, the clinical and administrative staff affiliated with the participating pediatric practices from Children’s Community Pediatrics of Children’s Hospital of Pittsburgh, the Advanced Center for Intervention Services Research (David Brent, MD, PI), and V. Robin Weersing, PhD, Kelly Kelleher, MD, Kevin Rumbarger, and James Varni, PhD.

Footnotes

Dr Kolko directed the trial, conceived of the design, designed and interpreted most of the analyses, and was responsible for most of the writing; Dr Campo contributed to the design of the trial, provided consultation during the trial, and helped to write and edit the manuscript; Dr Kilbourne offered recommendations on the organization of the manuscript, and helped to write and edit the manuscript; Mr Hart contributed to the design and conduct of all data analyses; Dr Sakolsky served as the consulting psychiatrist on the trial, offered recommendations on study measures, and helped to edit the manuscript; Dr Wisniewski served as the primary statistician on the project and contributed to the original clinical trial design/randomization scheme and made recommendations for the analytic plan; and all authors approved the final manuscript as submitted.

. The Physician Belief Scale and psychosocial problems in children: a report from the Pediatric Research in Office Settings and the Ambulatory Sentinel Practice Network.J Dev Behav Pediatr. 1999;20(1):24–30pmid:10071942

Terms of Use
The American Academy of Pediatrics (AAP) takes the issue of privacy very seriously. See our Privacy Statement for information about how AAP collects, uses, safeguards and discloses the information collected on our Website from visitors and by means of technology.FAQ