Making Implementation Last: The Impact of Training Design on the Sustainability of an Evidence-Based Treatment in a Randomized Controlled Trial

Predictors of the sustainability for an evidence-based eating disorder prevention program delivered by college peer educators

BACKGROUND

Despite ongoing efforts to introduce evidence-based interventions (EBIs) into mental health care settings, little research has focused on the sustainability of EBIs in these settings. College campuses are a natural place to intervene with young adults who are at high risk for mental health disorders, including eating disorders. The current study tested the effect of three levels of implementation support on the sustainability of an evidence-based group eating disorder prevention program, the Body Project, delivered by peer educators. We also tested whether intervention, contextual, or implementation process factors predicted sustainability.

METHODS

We recruited 63 colleges with peer educator programs and randomly assigned them to (a) receive a 2-day Train-the-Trainer (TTT) training in which peer educators were trained to implement the Body Project and supervisors were taught how to train future peer educators (TTT), (b) TTT training plus a technical assistance (TA) workshop (TTT + TA), or (c) TTT plus the TA workshop and quality assurance (QA) consultations over 1-year (TTT + TA + QA). We tested whether implementation support strategies, perceived characteristics of the intervention and attitudes towards evidence-based interventions at baseline and the proportion of completed implementation activities during the implementation year predicted three school-level dichotomous sustainability outcomes (offering Body Project groups, training peer educators, training supervisors) over the subsequent two-year sustainability period using logistic regression models.

RESULTS

Implementation support strategies did not significantly predict any sustainability outcomes, although a trend suggested that colleges randomized to the TTT + TA + QA strategy were more likely to train new supervisors (OR = 5.46, 95% CI [0.89-33.38]). Colleges that completed a greater proportion of implementation activities were more likely to offer Body Project groups (OR = 1.53, 95% CI [1.19-1.98]) and train new peer educators during the sustainability phase (OR = 1.39, 95% CI [1.10-1.74]). Perceived positive characteristics of the Body Project predicted training new peer educators (OR = 18.42, 95% CI [1.48-299.66]), which may be critical for sustainability in routine settings with high provider turnover.

CONCLUSIONS

Helping schools complete more implementation activities and increasing the perceived positive characteristics of a prevention program may result in greater sustainment of prevention program implementation.

TRIAL REGISTRATION

This study was preregistered on 12/07/17 with ClinicalTrials.gov, ID NCT03409809, https://clinicaltrials.gov/ct2/show/NCT03409809 .

Multi-level alignment processes in the sustainment of a youth substance use treatment model following a federal implementation initiative: A mixed method study

INTRODUCTION

Government agencies have identified evidence-based practice (EBP) dissemination as a pathway to high-quality behavioral health care for youth. However, gaps remain about how to best sustain EBPs in treatment organizations in the U.S., especially in resource-constrained settings like publicly-funded youth substance use services. One important, but understudied, determinant of EBP sustainment is alignment: the extent to which multi-level factors that influence sustainment processes and outcomes are congruent, consistent, and/or coordinated. This study examined the role of alignment in U.S. states' efforts to sustain the Adolescent Community Reinforcement Approach (A-CRA), an EBP for youth substance use disorders, during the COVID-19 pandemic.

METHODS

In this mixed methods study, the qualitative investigation preceded and informed the quantitative investigation. We interviewed state administrators and providers (i.e., supervisors and clinicians) from 15 states that had completed a federal A-CRA implementation grant; providers also completed surveys. The sample included 50 providers from 35 treatment organizations that reported sustaining A-CRA when the COVID-19 pandemic began, and 20 state administrators. In qualitative thematic analyses, we applied the EPIS (Exploration, Preparation, Implementation, Sustainment) framework to characterize alignment processes that interviewees described as influential on sustainment. We then used survey items to quantitatively explore the associations described in qualitative themes, using bivariate linear regressions.

RESULTS

At the time of interview, staff from 80 % of the treatment organizations (n = 28), reported sustaining A-CRA. Providers from both sustainer and non-sustainer organizations, as well as state administrators, described major sources of misalignment when state agencies ceased technical assistance post-grant, and because limited staff capacity conflicted with A-CRA's training model, which was perceived as time-intensive. Participants described the pandemic as exacerbating preexisting challenges, including capacity issues. Sustainer organizations reported seeking new funding to help sustain A-CRA. Quantitative associations between self-rated extent of sustainment and other survey items largely followed the pattern predicted from the qualitative findings.

CONCLUSIONS

The COVID-19 pandemic amplified longstanding A-CRA sustainment challenges, but treatment organizations already successfully sustaining A-CRA pre-pandemic largely continued. There are missed opportunities for state-level actors to coordinate with providers on the shared goal of EBP sustainment. A greater focus on alignment processes in research and practice could help states and providers strengthen sustainability planning.

Optimizing implementation: elucidating the role of behavior change techniques and corresponding strategies on determinants and implementation performance: a cross-sectional study

INTRODUCTION

Behavior change techniques (BCTs) are considered as active components of implementation strategies, influencing determinants and, ultimately, implementation performance. In our previous Delphi study, experts formulated 'implementation hypotheses', detailing how specific combinations of BCTs and strategies (referred to as BCT-strategy combinations) might influence determinants and guideline implementation within youth care. For example, educational meetings providing instructions on guideline use were hypothesized to enhance practitioners' knowledge and, consequently, guideline implementation. However, these hypotheses have not been verified in practice yet.

METHOD

We conducted a cross-sectional study involving practitioners and management professionals from youth (health)care organizations. Using questionnaires, we obtained data on the presence of BCT-strategy combinations and their perceived influence on determinants and implementation performance. Chi-squared tests and regression analyses were employed to determine the influence of specific BCT-strategy combinations on determinants and implementation performance.

RESULTS

Our analyses included data from 104 practitioners and 34 management professionals. Most of the management professionals indicated that the BCT-strategy combinations positively influenced or had the potential to influence their implementation performance. At the practitioner level, half of the combinations were perceived to have a positive influence on determinants and implementation performance. Furthermore, practitioners who reported the absence of BCT-strategy combinations were more skeptical about their potential influence on determinants and implementation performance.

CONCLUSION

Several BCT-strategy combinations were perceived to improve or potentially improve implementation performance of both practitioners and management professionals. In the development and evaluation of implementation efforts, we advocate for clearly describing the implementation effort's objective and using frameworks that detail the BCTs inducing behavior change, the strategy employed, and the processes driving the observed changes. Understanding these interconnected processes is important in designing targeted, evidence-based behavior change interventions. This understanding optimizes resource allocation and contributes to the overall success of implementation efforts in youth care.

Implementation fidelity, student outcomes, and cost-effectiveness of train-the-trainer strategies for Masters-level therapists in urban schools: results from a cluster randomized trial

BACKGROUND

Little is known about the effectiveness and cost-effectiveness of train-the-trainer implementation strategies in supporting mental health evidence-based practices in schools, and about the optimal level of support needed for TT strategies.

METHODS

The current study is part of a larger type 2 hybrid cluster randomized controlled trial. It compares two train-the-trainer strategies, Train-the-Trainer (TT) and Train-the-Trainer plus ongoing consultation for trainers (TT +) on the delivery of a group cognitive behavioral treatment protocol for anxiety disorders. Participants were 33 therapists, 29 supervisors, and 125 students who were at risk for anxiety disorders from 22 urban schools. Implementation outcomes were implementation fidelity and treatment dosage. Student outcomes were child- and parent-reported symptoms of anxiety, child-reported symptoms of depression, and teacher-reported academic engagement. We estimated the cost of implementing the intervention in each condition and examined the probability that a support strategy for supervisors (TT vs TT +) is a good value for varying values of willingness to pay.

RESULTS

Therapists in the TT and TT + conditions obtained similarly high implementation fidelity and students in the conditions received similar treatment dosages. A mixed effects modeling approach for student outcomes revealed time effects for symptoms of anxiety and depression reported by students, and emotional disaffection reported by teachers. There were no condition or condition × times effects. For both conditions, the time effects indicated an improvement from pre-treatment to post-treatment in symptoms of anxiety and depression and academic emotional engagement. The average cost of therapist, supervisor, and consultant time required to implement the intervention in each condition was $1002 for TT and $1431 for TT + (p = 0.01). There was a greater than 80% chance that TT was a good value compared to TT + for all values of willingness to pay per one-point improvement in anxiety scores.

CONCLUSIONS

A TT implementation approach consisting of a thorough initial training workshop for therapists and supervisors as well as ongoing supervision for therapists resulted in adequate levels of fidelity and student outcomes but at a lower cost, compared to the TT + condition that also included ongoing external expert consultation for supervisors.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT02651402.

Sustainability of the Collaborative Chronic Care Model in Outpatient Mental Health Teams Three Years Post-Implementation: A Qualitative Analysis

Our goal was to investigate the sustainability of care practices that are consistent with the collaborative chronic care model (CCM) in nine outpatient mental health teams located within US Department of Veterans Affairs (VA) medical centers, three to four years after the completion of CCM implementation. We conducted qualitative interviews (N = 30) with outpatient mental health staff from each of the nine teams. We based our directed content analysis on the six elements of the CCM. We found variable sustainability of CCM-based care processes across sites. Some care processes, such as delivery of evidence-based psychotherapies (EBPs) and use of measurement-based care (MBC) to guide clinic decision-making, were robustly maintained or even expanded within participating teams. In contrast, other care processes-which had in some cases been developed with considerable effort-had not been sustained. For example, care manager roles were diminished in scope or eliminated completely in response to workload pressures, frontline care needs, or the COVID-19 pandemic. Similarly, processes for engaging Veterans more fully in decision-making had generally been scaled back. Leadership support in the form of adequate team staffing and time to conduct team meetings were seen as crucial for sustaining CCM-consistent care. Given the potential impact of leadership turnover on sustainability in mental health, future efforts to implement CCM-based mental health care should strive to involve multiple leaders in implementation and sustainment efforts, lest one key departure undo years of implementation work. Our results also suggest that implementing CCM processes may best be conceptualized as a partnership across multiple levels of medical center leadership.

Learning Collaboratives: a Strategy for Quality Improvement and Implementation in Behavioral Health

Learning collaboratives are increasingly used in behavioral health. They generally involve bringing together teams from different organizations and using experts to educate and coach the teams in quality improvement, implementing evidence-based practices, and measuring the effects. Although learning collaboratives have demonstrated some effectiveness in general health care, the evidence is less clear in behavioral health and more rigorous studies are needed. Learning collaboratives may contain a range of elements, and which elements are included in any one learning collaborative varies widely; the unique contribution of each element has not been established. This commentary seeks to clarify the concept of a learning collaborative, highlight its common elements, review evidence of its effectiveness, identify its application in behavioral health, and highlight recommendations to guide technical assistance purveyors and behavioral health providers as they employ learning collaboratives to improve behavioral health access and quality.

Online training for evidence-based child trauma treatment: evaluation of the German language TF-CBT-Web

BACKGROUND

Evidence-based trauma-focused interventions for treating PTSD in children and youth are barely used in practice. Web-based training has proven to be an effective way of transferring knowledge to healthcare professionals.

OBJECTIVE

TF-CBT Web is a web-based training programme designed to foster the dissemination of Trauma Focused Cognitive Behavioural Therapy (TF-CBT) for children and youth, and is run by the Medical University of South Carolina. This paper describes the characteristics of healthcare professionals who registered for the adapted German language version of TF-CBT Web. It evaluates the effectiveness and user friendliness of the programme.

METHOD

: Similar to the TF-CBT treatment manual, the German language TF-CBT Web contains 12 modules. Between 2018 and 2020, 4,020 users registered for the programme. During the registration process users provided demographic information. The knowledge of users regarding the TF-CBT components was assessed via pre-tests and post-tests in each module.

RESULTS

The programme was accessed by a sample of mostly German users with varying professional health care backgrounds and a wide-ranging spread of work experience. The results indicated a significant knowledge gain and high rates of user satisfaction with the programme.

CONCLUSIONS

In summary, the results of this study suggested that web-based training is an effective and well-accepted method for knowledge gain in trauma-focused interventions. Future research should evaluate the actual application of the taught methods in clinical practice.

HIGHLIGHTS

Children and adolescents with PTSD require trauma-focused treatment. However, evidence-based interventions for this patient population are barely used. Therefore, it is necessary to expand professional training for the treatment of traumatised children and adolescents.Our evaluation showed the effectiveness and feasibility of a web-based training programme for mental health practitioners in an evidence-based treatment for children and youth with PTSD (TF-CBT). Results show a significant knowledge gain of users who participate in the web-based training programme.The user satisfaction survey also revealed that users found the modus and content of the web-based training applicable and relevant for their clinical practice.