Are CAFAS subscales and item weights valid? A preliminary investigation of the Child and Adolescent Functional Assessment Scale

Overview of Ten Child Mental Health Clinical Outcome Measures: Testing of Psychometric Properties with Diverse Client Populations in the U.S

While many standardized assessment measures exist to track child mental health treatment outcomes, the degree to which such tools have been adequately tested for reliability and validity across race, ethnicity, and class is uneven. This paper examines the corpus of published tests of psychometric properties for the ten standardized measures used in U.S. child outpatient care, with focus on breadth of testing across these domains. Our goal is to assist care providers, researchers, and legislators in understanding how cultural mismatch impacts measurement accuracy and how to select tools appropriate to the characteristics of their client populations. We also highlight avenues of needed research for measures that are in common use. The list of measures was compiled from (1) U.S. state Department of Mental Health websites; (2) a survey of California county behavioral health agency directors; and (3) exploratory literature scans of published research. Ten measures met inclusion criteria; for each one a systematic review of psychometrics literature was conducted. Diversity of participant research samples was examined as well as differences in reliability and validity by gender, race or ethnicity, and socio-economic class. All measures showed adequate reliability and validity, however half lacked diverse testing across all three domains and all lacked testing with Asian American/Pacific Islander and Native American children. ASEBA, PSC, and SDQ had the broadest testing.

The Influence of Youth Gender and Complex Trauma on the Relation Between Treatment Conditions and Outcomes in Therapeutic Residential Care

Complex trauma (CT) is the experience, or witness, of prolonged abuse or neglect that negatively affects children's emotional and psychological health. Youth in residential care experience higher incidences of complex trauma than youth in community-based care, with notable gender differences and presentation of psychological symptoms. This study examined the effects of trauma-informed residential care and the relation between CT and gender. A sample (n = 206) from an evaluation of a youth psychiatric residential facility in the Midwest that transitioned from a traditional care model to a trauma-informed care model was used. A hierarchical regression was used to model the main effects of model of care, gender, CT, length of stay, and crisis response on treatment outcomes; and the moderating effects of gender and CT. The results support the high prevalence of CT in residential care populations. The final model explained 30.2% of the variance with a statistically significant interaction between gender and length of stay in treatment, indicating that longer lengths of stay in treatment are associated with less change in functional impairment for girls than boys. Youth gender and prior trauma are important factors to consider when monitoring experiences and treatment outcomes in youth residential care.

Functional Impairment in Children With Externalizing Behavior Disorders: Psychometric Properties of the Weiss Functional Impairment Rating Scale-Parent Report in a German Clinical Sample

OBJECTIVE

To examine the psychometric properties of a German adaptation of the Weiss Functional Impairment Rating Scale-Parent Report (WFIRS-P) in a clinical sample of children (4-12 years) with externalizing behavior disorders.

METHOD

Data were collected within two clinical trials (N = 264). Factorial validity, reliability, and divergent validity from symptoms of ADHD and oppositional defiant disorder (ODD) were assessed.

RESULTS

Confirmatory factor analyses revealed that a bifactor model consistent with the theoretical assumption of a general construct of impairment (total scale) and additional specific factors (subscales) provided satisfactory data fit. Model-based reliability estimates showed that both the general construct and specific factors accounted for item variance. Internal consistencies were >.70, part-whole corrected item-scale correlations mostly >.30. Correlations between the WFIRS-P Scales and ADHD and ODD symptoms were low to moderate.

CONCLUSION

The results support the factorial validity, reliability, and divergent validity of the WFIRS-P.

Predictors of functional improvement in children and adolescents at a publicly funded specialist outpatient treatment clinic in a Canadian Prairie City

Children's mental health problems substantially impact their functioning. For clinically treated children and adolescents, we explored the impact of mental health treatment on functioning and identified predictors of functional improvement. Outpatient clinical data from a regional publicly funded specialist outpatient treatment clinic were analyzed. The Child and Adolescent Functional Assessment Scale (CAFAS) was used to assess outcomes. Non-parametric tests were used to compare baseline and exit scores. Logistic regression analysis was used separately for children and adolescents to examine predictors of improvement. Total CAFAS scores at exit showed a significant decrease from initial scores for both age groups, indicating improvements in clients' functioning. Children and adolescents had shared predictors for initial level of dysfunction, length of treatment and the presence of pervasive behavioral impairment (PBI). Primary presenting problem, caregiver support and area of residence were only associated with outcome among children. Clients with higher initial levels of dysfunction and PBI require longer treatment cycles to reach an acceptable outcome. Shortening the length of treatment cycles may improve the efficiency of resource use but can be detrimental to some clients. Personalized treatment should be tailored to the clients with specific characteristics and needs.

Statewide Implementation of Parenting with Love and Limits Among Youth with Co-Existing Internalizing and Externalizing Functional Impairments Reduces Return to Service Rates and Treatment Costs

Many community mental health (CMH) systems contain inefficiencies, contributing to unmet need for services among youth. Using a quasi-experimental research design, we examined the implementation of an adapted structural-strategic family intervention, Parenting with Love and Limits, in a state CMH system to increase efficiency of services to youth with co-existing internalizing and externalizing functional impairments (PLL n = 296; Treatment-As-Usual n = 296; 54% male; 81% Caucasian). Youth receiving PLL experienced shorter treatment durations and returned to CMH services at significantly lower rates than youth receiving treatment-as-usual. They also demonstrated significant decreases in internalizing and externalizing symptoms over time. Findings lay the foundation for further examination of the role of an adapted structural-strategic family treatment in increasing the efficiency of CMH systems.

Using agency data for evidence-based programming: a university-agency collaboration

Agencies providing residential treatment are encouraged, or even mandated, to collect outcomes data and to implement evidence based practices, yet little guidance has been provided on how to do so using agency administrative data that are collected on an ongoing basis. We examined data on Child and Adolescent Functional Assessment Scale (CAFAS) scores for 1608 admissions to a residential treatment center from 2002 through 2008. CAFAS scores were measured every 90 days, providing multiple CAFAS scores for each individual. Results demonstrated that on average residents improved in functioning over time. Sensitive to the evolving needs of residents who had been entering the program, the treatment center made significant program changes in 2006 to attempt to better serve residents through a broad array of specialized programming. Compared to the overall results, the analysis suggested that residents who entered the program since October 2006 appeared to have made larger improvements in their CAFAS scores. Results were derived by employing multilevel models appropriate for estimating growth trajectories with repeated measures data. Conversations with agency staff suggested that using administrative data, and advanced statistical models, were extremely helpful for organizational decision making and evidence-based programming.