Models of case mix adjustment for Ohio Mental Health Consumer Outcomes among children and adolescents

Randomized Trial of an Organizational Implementation Strategy to Improve Measurement-Based Care Fidelity and Youth Outcomes in Community Mental Health

OBJECTIVE

Measurement-based care (MBC), which collects session-by-session symptom data from patients and provides clinicians with feedback on treatment response, is a highly generalizable evidence-based practice with significant potential to improve the outcomes of mental health treatment in youth when implemented with fidelity; however, it is rarely used in community settings. This study tested whether an implementation strategy targeting organizational leadership and organizational implementation climate could improve MBC fidelity and clinical outcomes for youth in outpatient mental health clinics.

METHOD

In a cluster randomized trial, 21 clinics were assigned to the Leadership and Organizational Change for Implementation strategy plus training and technical assistance in MBC (k = 11, n = 117) or training and technical assistance only (k = 10, n = 117). Primary outcomes of MBC fidelity (assessed via electronic metadata) and youth symptom improvement (assessed via caregiver-reported change on the Shortform Assessment for Children Total Problem Score) were collected for consecutively enrolled youths (ages 4-18 years) who initiated treatment in the 12 months following MBC training. Outcomes of each youth were assessed for 6 months following baseline.

RESULTS

A total of 234 youths were enrolled and included in intent-to-treat analyses. At baseline, there were no significant differences by condition in clinic, clinician, or youth characteristics. Youths in clinics using the Leadership and Organizational Change for Implementation strategy experienced significantly higher MBC fidelity compared with youths in control clinics (23.1% vs 3.4%, p = .014), and exhibited significantly greater reductions in symptoms from baseline to 6 months (d = 0.31, 95% CI: 0.04-0.58, p = .023).

CONCLUSION

Implementation strategies targeting organizational leadership and focused implementation climate can improve fidelity to evidence-based practices and clinical outcomes of youth mental health services.

CLINICAL TRIAL REGISTRATION INFORMATION

Working to Implement and Sustain Digital Outcome Measures (WISDOM); https://clinicaltrials.gov/; NCT04096274.

Organization-level variation in therapists' attitudes toward and use of measurement-based care

PURPOSE

Despite significant interest in improving behavioral health therapists' implementation of measurement-based care (MBC)-and widespread acknowledgment of the potential importance of organization-level determinants-little is known about the extent to which therapists' use of, and attitudes toward, MBC vary across and within provider organizations or the multilevel factors that predict this variation.

METHODS

Data were collected from 177 therapists delivering psychotherapy to youth in 21 specialty outpatient clinics in the USA. Primary outcomes were use of MBC for progress monitoring and treatment modification, measured by the nationally-normed Current Assessment of Practice Evaluation-Revised. Secondary outcomes were therapist attitudes towards MBC. Linear multilevel regression models tested the association of theory-informed clinic and therapist characteristics with these outcomes.

RESULTS

Use of MBC varied significantly across clinics, with means on progress monitoring ranging from values at the 25th to 93rd percentiles and means on treatment modification ranging from the 18th to 71st percentiles. At the clinic level, the most robust predictor of both outcomes was clinic climate for evidence-based practice implementation; at the therapist level, the most robust predictors were: attitudes regarding practicality, exposure to MBC in graduate training, and prior experience with MBC. Attitudes were most consistently related to clinic climate for evidence-based practice implementation, exposure to MBC in graduate training, and prior experience with MBC.

CONCLUSIONS

There is important variation in therapists' attitudes toward and use of MBC across clinics. Implementation strategies that target clinic climate for evidence-based practice implementation, graduate training, and practicality may enhance MBC implementation in behavioral health.

Understanding Racial-Ethnic Disparities in Wraparound Care for Youths With Emotional and Behavioral Disorders

OBJECTIVE

Systems of care (SOCs) were developed to increase access to and quality of care for children with emotional and behavioral difficulties and their families through the provision of coordinated, community-based, culturally competent, family-driven services. SOCs focus on wraparound care that is individualized to meet each family's needs. Previous research has illustrated significant disparities in outcomes of nonwraparound care on the basis of youths' race-ethnicity. This study aimed to fill a research gap by examining disparities in outcomes for families receiving wraparound care coordination within an SOC.

METHODS

This exploratory study examined racial-ethnic disparities in outcomes observed at intake, during service provision, and at 6-month follow-up among 1,138 youths and their caregivers who participated in wraparound care coordination as part of a statewide SOC between 2016 and 2020. Analyses of variance and regression analyses were executed to investigate whether receiving services and/or the youths' racial-ethnic identity predicted differences in behavioral health outcomes and characteristics of or satisfaction with care. Caregiver-reported outcomes were assessed with the Ohio Scales for Youth, the Child Trauma Screen, and the Caregiver Strain Questionnaire.

RESULTS

Results revealed few racial-ethnic disparities in the characteristics and outcomes of care coordination among participants at intake or in family involvement in the wraparound process. Participants across groups reported similar and significant improvement in outcomes. However, the results indicated some disparities in satisfaction with care.

CONCLUSIONS

Results revealed the positive impact of care coordination on the health and well-being of youths and caregivers across racial-ethnic groups.

Predictors of child and adolescent mental health treatment outcome

BACKGROUND

To examine the predictors of treatment outcome or improvement in mental health difficulties for young people accessing child and adolescent mental health services.

METHODS

We conducted a secondary analysis of routinely collected data from services in England using the Mental Health Services Data Set. We conducted multilevel regressions on N = 5907 episodes from 14 services (M_age = 13.76 years, SD_age = 2.45, range = 8-25 years; 3540 or 59.93% female) with complete information on mental health difficulties at baseline. We conduct similar analyses on N = 1805 episodes from 10 services (M_age = 13.59 years, SD_age = 2.33, range = 8-24 years; 1120 or 62.05% female) also with complete information on mental health difficulties at follow up.

RESULTS

Girls had higher levels of mental health difficulties at baseline than boys (β = 0.28, 95% CI = 0.24-0.32). Young people with higher levels of mental health difficulties at baseline also had higher levels of deterioration in mental health difficulties at follow up (β = 0.72, 95% CI = 0.67-0.76), and girls had higher levels of deterioration in mental health difficulties at follow up than boys (β = 0.09, 95% CI = 0.03-0.16). Young people with social anxiety, panic disorder, low mood, or self-harm had higher levels of mental health difficulties at baseline and of deterioration in mental health difficulties at follow up compared to young people without these presenting problems.

CONCLUSIONS

Services seeing higher proportions of young people with higher levels of mental health difficulties at baseline, social anxiety, panic disorder, low mood, or self-harm may be expected to show lower levels of improvement in mental health difficulties at follow up.

A Mixed-Method Analysis on the Impacts of a System-Driven Implementation of Multiple Child Evidence-Based Practices on Community Mental Health Providers

Initiatives to scale up evidence-based practices (EBPs) in routine care are likely to have myriad impacts on community providers, but these impacts have not yet been examined in depth. This is especially true within the context of simultaneous implementation of multiple evidence-based practices. The aim of this study was to characterize the multifaceted impacts on community mental health therapists within a system-driven implementation of multiple EBPs for youth and families. Semistructured interview and survey data were gathered from 60 therapists at 11 agencies contracted with the Los Angeles County Department of Mental Health to deliver EBPs within the Prevention and Early Intervention initiative. Therapists' accounts of impacts varied, and were either predominately negative, predominantly positive, or mixed-valence. Mixed-methods analyses using Kruskal-Wallis tests showed therapist valence groups varied on mean levels of self-reported burnout on surveys. Themes from qualitative data revealed several favorable (e.g., increased EBP knowledge, structure) and unfavorable (e.g., distress, feeling constrained by EBPs) impacts of county-contracted EBP implementation. These findings inform the development and implementation of future system-driven EBP initiatives that consider therapist perspective to optimize positive impacts and minimize negative impacts on therapists.

Linking molar organizational climate and strategic implementation climate to clinicians' use of evidence-based psychotherapy techniques: cross-sectional and lagged analyses from a 2-year observational study

Background

Behavioral health organizations are characterized by multiple organizational climates, including molar climate, which encompasses clinicians’ shared perceptions of how the work environment impacts their personal well-being, and strategic implementation climate, which includes clinicians’ shared perceptions of the extent to which evidence-based practice implementation is expected, supported, and rewarded by the organization. Theory suggests these climates have joint, cross-level effects on clinicians’ implementation of evidence-based practice and that these effects may be long term (i.e., up to 2 years); however, no empirical studies have tested these relationships. We hypothesize that molar climate moderates implementation climate’s concurrent and long-term relationships with clinicians’ use of evidence-based practice such that strategic implementation climate will have its most positive effects when it is accompanied by a positive molar climate.

Methods

Hypotheses were tested using data collected from 235 clinicians in 20 behavioral health organizations. At baseline, clinicians reported on molar climate and implementation climate. At baseline and at a 2-year follow-up, all clinicians who were present in the organizations reported on their use of cognitive-behavioral psychotherapy techniques, an evidence-based practice for youth psychiatric disorders. Two-level mixed-effects regression models tested whether baseline molar climate and implementation climate interacted in predicting clinicians’ evidence-based practice use at baseline and at 2-year follow-up.

Results

In organizations with more positive molar climates at baseline, higher levels of implementation climate predicted increased evidence-based practice use among clinicians who were present at baseline and among clinicians who were present in the organizations at 2-year follow-up; however, in organizations with less positive molar climates, implementation climate was not related to clinicians’ use of evidence-based practice at either time point.

Conclusions

Optimizing clinicians’ implementation of evidence-based practice in behavioral health requires attention to both molar climate and strategic implementation climate. Strategies that focus exclusively on implementation climate may not be effective levers for behavior change if the organization does not also engender a positive molar climate. These findings have implications for the development of implementation theory and effective implementation strategies.

Developing a case mix classification for child and adolescent mental health services: the influence of presenting problems, complexity factors and service providers on number of appointments

Background: Case-mix classification is a focus of international attention in considering how best to manage and fund services, by providing a basis for fairer comparison of resource utilization. Yet there is little evidence of the best ways to establish case mix for child and adolescent mental health services (CAMHS).Aim: To develop a case mix classification for CAMHS that is clinically meaningful and predictive of number of appointments attended and to investigate the influence of presenting problems, context and complexity factors and provider variation.Method: We analysed 4573 completed episodes of outpatient care from 11 English CAMHS. Cluster analysis, regression trees and a conceptual classification based on clinical best practice guidelines were compared regarding their ability to predict number of appointments, using mixed effects negative binomial regression.Results: The conceptual classification is clinically meaningful and did as well as data-driven classifications in accounting for number of appointments. There was little evidence for effects of complexity or context factors, with the possible exception of school attendance problems. Substantial variation in resource provision between providers was not explained well by case mix.Conclusion: The conceptually-derived classification merits further testing and development in the context of collaborative decision making.

Service-level variation, patient-level factors, and treatment outcome in those seen by child mental health services

Service comparison is a policy priority but is not without controversy. This paper aims to investigate the amount of service-level variation in outcomes in child mental health, whether it differed when examining outcomes unadjusted vs. adjusted for expected change over time, and which patient-level characteristics were associated with the difference observed between services. Multilevel regressions were used on N = 3256 young people (53% male, mean age 11.33 years) from 13 child mental health services. Outcome was measured using the parent-reported Strengths and Difficulties Questionnaire. The results showed there was 4-5% service-level variation in outcomes. Findings were broadly consistent across unadjusted vs. adjusted outcomes. Young people with autism or infrequent case characteristics (e.g., substance misuse) had greater risk of poor outcomes. Comparison of services with high proportions of young people with autism or infrequent case characteristics requiring specialist input needs particular caution as these young people may be at greater risk of poor outcomes.

Analysing and reporting UK CAMHS outcomes: an application of funnel plots

BACKGROUND

Patient-reported outcomes measures are increasingly being used in child and adolescent mental health services (CAMHS). League tables are a common way of comparing organizations across health and education but have limitations that are not well known in CAMHS.

METHOD

Parent-rated Strengths and Difficulties Questionnaire (SDQ) outcomes data from 15,771 episodes of care across 51 UK CAMHS were analysed using funnel plots, an alternative to league tables.

RESULTS

While most services were indistinguishable from the national average there was evidence of heterogeneous outcomes and seven services had outcomes below 99.9% limits for SDQ added-value scores.

CONCLUSIONS

Funnel plots are powerful tools for navigating national data and can help prompt investigations using clinical theory and local service context. Examples are provided of factors to consider in these investigations. We argue that analyses of the local context are central to the valid application of funnel plots.

Are children improving? Results from outcome measurement in a large mental health system

Standardized outcome assessment was implemented in a large county mental health system. This study examines changes in children's emotional and behavioral problems after 6 months of treatment and identifies factors associated with treatment improvement. Results from 3,215 youth indicated that externalizing and internalizing problems significantly decreased from intake to 6 months and that ethnicity, gender, and age were associated with caregiver-reported changes in symptomatology. While there was improvement associated with entry into treatment, the magnitude of symptom reduction was small to medium and not associated with amount of care, providing only limited support for the impact of routine out-patient care.

Beyond outcomes monitoring: measurement feedback systems in child and adolescent clinical practice

PURPOSE OF REVIEW

To review the literature published during the past year relevant to identifying the best measures for monitoring progress in the treatment of child and adolescent clients and their families.

RECENT FINDINGS

The current literature shows an increasing focus on clinical utility in measure development as demonstrated by the recent emphasis on evidence-based assessment. However, there is very little research on how the inclusion of monitoring might enhance clinician practice and ultimately youth and family outcomes. There is great promise in expanding our thinking beyond mere outcome measurement to a measurement feedback system that provides timely feedback that is comprehensive and concurrent with treatment.

SUMMARY

Investment in the development of measurement feedback systems is needed to enhance clinical judgment and increase effectiveness of treatment. Clinical utility and consumer appeal need to be key considerations for measures intended to be used in everyday clinical practice. Most importantly, we must harness the power of technology and advances in measurement to provide clinicians with the tools to use effectively the systematic data provided through frequent measurement with measurement feedback systems.