Multi-Level Systems Evaluation: Selected Projects from Hawaii

Synopsis: In the context of Hawaii’s overall system development and evaluation, the present symposium describes selected Hawaii system of care evaluation projects that cut across levels of analysis. At the client outcome level, Nakamura discusses the development, implementation, and initial validation data of aggregated youth progress ratings based on individualized targets. At the service level, Roberts et al. present data regarding the utilization and service outcomes of intensive home-based services. Focusing on system level analyses, Fillman presents a quantitative framework and preliminary data comparing relative efficiencies across operating units based on individually optimized cost-quality functions.

Synopsis: The current investigation utilized the target complaints scoring methodology for examining the relationship between progress ratings on idiographic treatment targets and changes in a standardized measure of functional impairment. Analyses were conducted with a sample of youth with data available at intake into Hawaii’s Child and Adolescent Mental Health Division (CAMHD) system and after six months of treatment. Results indicated degree of improvement on idiographic treatment targets correlated with change (improvements) in global functioning directly, and when controlling for intake scores on both measures. Overall, results support the use and continued development of target progress ratings.

Intensive Home and Community Services: Status of Twelve-Month Follow-Up

Synopsis: The Hawaii Child and Adolescent Mental Health Division (CAMHD) has developed a range of service options within its system of care for youth and their families. Along these lines, Intensive In-Home (IIH) services are frequently provided in its systems array as a level of care that aims to meet the needs of youth and families in the community and avoid the disruption of out-of-home placement. This study followed youth receiving IIH services through CAMHD’s Family Guidance Centers from 2001 through 2003, with a 12-month follow-up after intake. We evaluate and describe the type of services accessed twelve months following system intake and characteristics of youth receiving more restrictive versus less restrictive services. Possible implications for program and system design are discussed.

Synopsis: This presentation overviews and applies the DEA linear programming methodology to examine the relative efficiencies of six case management centers. Five resource input variables were compared to four quality output variables for services to youth with serious mental health problems. The results identified one case management center as relatively inefficient and concluded that the DEA methodology can be an important tool for concurrently focusing quality and financial improvement efforts within an evidence-based mental health delivery system.