Toward enough of the best for all: research to transform the efficacy, quality, and reach of mental health care for youth

Development and Evaluation of Two Integrated Care Models for Children Using a Partnered Formative Evaluation Approach

Objective

To describe the development and evaluation of two integrated care models using a partnered formative evaluation approach across a private foundation, clinic leaders, providers and staff, and a university-based research center.

Design

Retrospective cohort study using multiple data sources.

Setting

Two federal qualified health care centers serving low-income children and families in Chicago.

Participants

Private foundation, clinic and academic partners.

Interventions

Development of two integrated care models and partnered evaluation design.

Main Outcome Measures

Accomplishments and early lessons learned.

Results

Together, the foundation-clinic-academic partners worked to include best practices in two integrated care models for children while developing the evaluation design. A shared data collection approach, which empowered the clinic partners to collect data using a web-based tool for a prospective longitudinal cohort study, was also created.

Conclusion

Across three formative evaluation stages, the foundation, clinic, and academic partners continued to reach beyond their respective traditional roles of project oversight, clinical service, and research as adjustments were collectively made to accommodate barriers and unanticipated events. Together, an innovative shared data collection approach was developed that extends partnered research to include data collection being led by the clinic partners and supported by the technical resources of a university-based research center.

Pilot to policy: statewide dissemination and implementation of evidence-based treatment for traumatized youth

BACKGROUND

A model for statewide dissemination of evidence-based treatment (EBT) for traumatized youth was piloted and taken to scale across North Carolina (NC). This article describes the implementation platform developed, piloted, and evaluated by the NC Child Treatment Program to train agency providers in Trauma-Focused Cognitive Behavioral Therapy using the National Center for Child Traumatic Stress Learning Collaborative (LC) Model on Adoption & Implementation of EBTs. This type of LC incorporates adult learning principles to enhance clinical skills development as part of training and many key implementation science strategies while working with agencies and clinicians to implement and sustain the new practice.

METHODS

Clinicians (n = 124) from northeastern NC were enrolled in one of two TF-CBT LCs that lasted 12 months each. During the LC clinicians were expected to take at least two clients through TF-CBT treatment with fidelity and outcomes monitoring by trainers who offered consultation by phone and during trainings. Participating clinicians initiated treatment with 281 clients. The relationship of clinician and client characteristics to treatment fidelity and outcomes was examined using hierarchical linear regression.

RESULTS

One hundred eleven clinicians completed general training on trauma assessment batteries and TF-CBT. Sixty-five clinicians met all mastery and fidelity requirements to meet roster criteria. One hundred fifty-six (55%) clients had fidelity-monitored assessment and TF-CBT. Child externalizing, internalizing, and post-traumatic stress symptoms, as well as parent distress levels, decreased significantly with treatment fidelity moderating child PTSD outcomes. Since this pilot, 11 additional cohorts of TF-CBT providers have been trained to these roster criteria.

CONCLUSION

Scaling up or outcomes-oriented implementation appears best accomplished when training incorporates: 1) practice-based learning, 2) fidelity coaching, 3) clinical assessment and outcomes-oriented treatment, 4) organizational skill-building to address barriers for agencies, and 5) linking clients to trained clinicians via an online provider roster. Demonstrating clinician performance and client outcomes in this pilot and subsequent cohorts led to legislative support for dissemination of a service array of EBTs by the NC Child Treatment Program.

The Contextualized Technology Adaptation Process (CTAP): Optimizing Health Information Technology to Improve Mental Health Systems

Health information technologies have become a central fixture in the mental healthcare landscape, but few frameworks exist to guide their adaptation to novel settings. This paper introduces the contextualized technology adaptation process (CTAP) and presents data collected during Phase 1 of its application to measurement feedback system development in school mental health. The CTAP is built on models of human-centered design and implementation science and incorporates repeated mixed methods assessments to guide the design of technologies to ensure high compatibility with a destination setting. CTAP phases include: (1) Contextual evaluation, (2) Evaluation of the unadapted technology, (3) Trialing and evaluation of the adapted technology, (4) Refinement and larger-scale implementation, and (5) Sustainment through ongoing evaluation and system revision. Qualitative findings from school-based practitioner focus groups are presented, which provided information for CTAP Phase 1, contextual evaluation, surrounding education sector clinicians' workflows, types of technologies currently available, and influences on technology use. Discussion focuses on how findings will inform subsequent CTAP phases, as well as their implications for future technology adaptation across content domains and service sectors.

The implementation research institute: training mental health implementation researchers in the United States

BACKGROUND

The Implementation Research Institute (IRI) provides two years of training in mental health implementation science for 10 new fellows each year. The IRI is supported by a National Institute of Mental Health (NIMH) R25 grant and the Department of Veterans Affairs (VA). Fellows attend two annual week-long trainings at Washington University in St. Louis. Training is provided through a rigorous curriculum, local and national mentoring, a 'learning site visit' to a federally funded implementation research project, pilot research, and grant writing.

METHODS

This paper describes the rationale, components, outcomes to date, and participant experiences with IRI.

RESULTS

IRI outcomes include 31 newly trained implementation researchers, their new grant proposals, contributions to other national dissemination and implementation research training, and publications in implementation science authored by the Core Faculty and fellows. Former fellows have obtained independent research funding in implementation science and are beginning to serve as mentors for more junior investigators.

CONCLUSIONS

Based on the number of implementation research grant proposals and papers produced by fellows to date, the IRI is proving successful in preparing new researchers who can inform the process of making evidence-based mental healthcare more available through real-world settings of care and who are advancing the field of implementation science.

Improving practice in community-based settings: a randomized trial of supervision - study protocol

Background

Evidence-based treatments for child mental health problems are not consistently available in public mental health settings. Expanding availability requires workforce training. However, research has demonstrated that training alone is not sufficient for changing provider behavior, suggesting that ongoing intervention-specific supervision or consultation is required. Supervision is notably under-investigated, particularly as provided in public mental health. The degree to which supervision in this setting includes ‘gold standard’ supervision elements from efficacy trials (e.g., session review, model fidelity, outcome monitoring, skill-building) is unknown. The current federally-funded investigation leverages the Washington State Trauma-focused Cognitive Behavioral Therapy Initiative to describe usual supervision practices and test the impact of systematic implementation of gold standard supervision strategies on treatment fidelity and clinical outcomes.

Methods/Design

The study has two phases. We will conduct an initial descriptive study (Phase I) of supervision practices within public mental health in Washington State followed by a randomized controlled trial of gold standard supervision strategies (Phase II), with randomization at the clinician level (i.e., supervisors provide both conditions). Study participants will be 35 supervisors and 130 clinicians in community mental health centers. We will enroll one child per clinician in Phase I (N = 130) and three children per clinician in Phase II (N = 390). We use a multi-level mixed within- and between-subjects longitudinal design. Audio recordings of supervision and therapy sessions will be collected and coded throughout both phases. Child outcome data will be collected at the beginning of treatment and at three and six months into treatment.

Discussion

This study will provide insight into how supervisors can optimally support clinicians delivering evidence-based treatments. Phase I will provide descriptive information, currently unavailable in the literature, about commonly used supervision strategies in community mental health. The Phase II randomized controlled trial of gold standard supervision strategies is, to our knowledge, the first experimental study of gold standard supervision strategies in community mental health and will yield needed information about how to leverage supervision to improve clinician fidelity and client outcomes.

Trial registration

ClinicalTrials.gov NCT01800266

Observational measure of implementation progress in community based settings: the Stages of Implementation Completion (SIC)

BACKGROUND

An increasingly large body of research is focused on designing and testing strategies to improve knowledge about how to embed evidence-based programs (EBP) into community settings. Development of strategies for overcoming barriers and increasing the effectiveness and pace of implementation is a high priority. Yet, there are few research tools that measure the implementation process itself. The Stages of Implementation Completion (SIC) is an observation-based measure that is used to track the time to achievement of key implementation milestones in an EBP being implemented in 51 counties in 53 sites (two counties have two sites) in two states in the United States.

METHODS

The SIC was developed in the context of a randomized trial comparing the effectiveness of two implementation strategies: community development teams (experimental condition) and individualized implementation (control condition). Fifty-one counties were randomized to experimental or control conditions for implementation of multidimensional treatment foster care (MTFC), an alternative to group/residential care placement for children and adolescents. Progress through eight implementation stages was tracked by noting dates of completion of specific activities in each stage. Activities were tailored to the strategies for implementing the specific EBP.

RESULTS

Preliminary data showed that several counties ceased progress during pre-implementation and that there was a high degree of variability among sites in the duration scores per stage and on the proportion of activities that were completed in each stage. Progress through activities and stages for three example counties is shown.

CONCLUSIONS

By assessing the attainment time of each stage and the proportion of activities completed, the SIC measure can be used to track and compare the effectiveness of various implementation strategies. Data from the SIC will provide sites with relevant information on the time and resources needed to implement MTFC during various phases of implementation. With some modifications, the SIC could be appropriate for use in evaluating implementation strategies in head-to-head randomized implementation trials and as a monitoring tool for rolling out other EBPs.