Influence of an Implementation Support Intervention on Barriers and Facilitators to Delivery of a Substance Use Prevention Program

What is known about the role of external facilitators during the implementation of complex interventions in healthcare settings? A scoping review

OBJECTIVE

To synthesise current knowledge about the role of external facilitators as an individual role during the implementation of complex interventions in healthcare settings.

DESIGN

A scoping review was conducted. We reviewed original studies (between 2000 and 2023) about implementing an evidence-based complex intervention in a healthcare setting using external facilitators to support the implementation process. An information specialist used the following databases for the search strategy: MEDLINE, CINAHL, APA PsycINFO, Academic Search Complete, EMBASE (Scopus), Business Source Complete and SocINDEX.

RESULTS

36 reports were included for analysis, including 34 different complex interventions. We performed a mixed thematic analysis to synthesise the data. We identified two primary external facilitator roles: lead facilitator and process expert facilitator. Process expert external facilitators have specific responsibilities according to their role and expertise in supporting three main processes: clinical, change management and knowledge/research management.

CONCLUSIONS

Future research should study processes supported by external facilitators and their relationship with facilitation strategies and implementation outcomes. Future systematic or realist reviews may also focus on outcomes and the effectiveness of external facilitation.

Organizational Capacity for Sexual Assault Prevention Within a U.S. Army Installation

INTRODUCTION

Preventing military sexual assault (SA) is a DoD priority. Building prevention capacity could strengthen the impact of prevention programs and improve outcomes. Capacity was conceptualized as implementation knowledge and skills using the Getting To Outcomes (GTO) process and organizational-level capacities using the Prevention Evaluation Framework, a framework that applies best practices in prevention and implementation science to prevention at the program and organizational level. The present study assesses implementation knowledge and skills currently possessed by SA prevention personnel in one U.S. Army installation with the goal of identifying preexisting proficiencies and potential gaps. The study uses the Prevention Evaluation Framework organizational-level domains to identify organizational capacities that could be improved.

MATERIALS AND METHODS

Fifteen SA prevention program personnel were recruited for individual 1-hour interviews from one U.S. Army installation. A semi-structured interview guide was developed by the study team to assess knowledge and skills within the GTO framework. Applied thematic analysis guided data coding and analysis.

RESULTS

Participants reported a range of proficiencies and gaps in capacity for SA prevention. Within the prevention workforce capacity, proficiencies were skill in tailoring training, and an existing train-the-trainer model. Gaps were training to facilitate difficult conversations, and an inability to tailor trainings to audience needs. Two proficiencies related to the comprehensive approach were having a victim advocate present during trainings, and providing training across leadership levels. The final proficiency involved data. Existing data sources were viewed as useful for identifying the nature and extent of local problems. However, participants reported a lack of confidence and skill in selecting appropriate outcome measures and collecting and using primary data. Gaps in leadership capacity were cultural norms, lack of trust, leader investment, logistical difficulty working with leaders, and comfort engaging with leaders. Gaps in collaborative relationships were the lack of informal networking within the SA prevention area to include civilians.

CONCLUSIONS

The GTO-specific knowledge and skills of SA prevention personnel at one U.S. Army installation revealed proficiencies and gaps in organizational capacities influencing the prevention mission. Findings indicate that GTO support could be useful for improving the quality of program activities. However, results also indicate that GTO support would be more effective if they were paired with sufficient prevention infrastructure at the organizational level.

Adaptive Designs in Implementation Science and Practice: Their Promise and the Need for Greater Understanding and Improved Communication

The promise of adaptation and adaptive designs in implementation science has been hindered by the lack of clarity and precision in defining what it means to adapt, especially regarding the distinction between adaptive study designs and adaptive implementation strategies. To ensure a common language for science and practice, authors reviewed the implementation science literature and found that the term adaptive was used to describe interventions, implementation strategies, and trial designs. To provide clarity and offer recommendations for reporting and strengthening study design, we propose a taxonomy that describes fixed versus adaptive implementation strategies and implementation trial designs. To improve impact, (a) futureimplementation studies should prespecify implementation strategy core functions that in turn can be taught to and replicated by health system/community partners, (b) funders should support exploratory studies that refine and specify implementation strategies, and (c) investigators should systematically address design requirements and ethical considerations (e.g., randomization, blinding/masking) with health system/community partners.

External relationships as implementation determinants in community-engaged, equity-focused COVID-19 vaccination events

Background

While relationships and connectedness among organizations have been included in implementation theories, models, and frameworks, the increased attention to health equity in implementation science raises the urgency of understanding the role of relationships external to the implementing organization. This paper addresses this gap through an exploration of the role of external relationships in community-based, equity-focused interventions.

Methods

This study focuses on an equity-focused, community-based COVID-19 vaccination intervention in Arkansas, drawing upon long-term community-engaged relationships among University of Arkansas for Medical Sciences and the Hispanic and Marshallese Islander communities. We used an exploratory qualitative descriptive design to examine barriers and facilitators to implementation of COVID-19 vaccination events analyzing in-depth qualitative interviews with implementation team members (n = 17).

Results

All participants described pre-existing relationships among the implementing organization, partner organizations, and communities as a key implementation determinant for this equity-focused program. At the inter-organizational level, external relationships included formal connections and informal relationships among staff (e.g., communication channels from prior partnerships). At the individual level, strong external relationships with the community were facilitators leveraging long-term engagement, community familiarity, and staff from the communities of focus. Strong external relationships facilitated program reach in underserved communities through three mechanisms: (1) reduced time required to establish functional working relationships among partners; (2) accessibility and cultural congruence of health services; and (3) increased trust among community members. Barriers to implementation also existed in external relationships, but had less influence than facilitators.

Conclusions

Achieving health equity in implementation science requires greater understanding of external relationships as implementation determinants. This exploratory study makes a significant contribution to the literature by describing the types of external relationships that facilitate equitable implementation and identifying the mechanisms through which they may work. We argue that approaches to community engagement drawn from community-engaged research approaches may be useful, as these processes require investment in building/maintaining formal and informal organizational and interpersonal relationships. Further research is needed to understand connections among external relationships and other implementation determinants.

Model for Doctor of Nursing Practice Projects Based on Cross-Fertilization Between Improvement and Implementation Sciences: Protocol for Quality Improvement and Program Evaluation Studies

BACKGROUND

Hundreds of nursing professionals graduate each year from Doctor of Nursing Practice (DNP) programs, entrusted with roles as practice scholars and leaders. Graduates are tasked to lead multidisciplinary knowledge implementation projects to improve safety, quality, and key performance metrics. Nevertheless, there is a continued lack of agreement and faculty dissatisfaction with the format, focus, and results of the DNP graduation projects. The use of a wide range of models and methodologies from different sciences for knowledge implementation introduces challenges to DNP students; affects the scientific rigor of the projects; and results in the overuse, superficial use, or misuse of the models. Quality improvement (QI) and program evaluation studies are substantial investments that may lead to waste and even harm if not well conducted. Traditional QI methodologies, commonly used in DNP projects, were found to be uncertain in improving health care outcomes. The complexity of health care systems calls for cross-fertilization between improvement and implementation sciences to improve health care outcomes.

OBJECTIVE

This study describes the development, implementation, and evaluation of a hybrid model for QI and program evaluation studies to guide scholarship in the DNP program.

METHODS

The hybrid model was based on cross-fertilization between improvement and implementation sciences. The model adapted the Getting to Outcome (GTO) and Knowledge to Action (KTA) models as the overarching process models for knowledge implementation. Within each phase of the GTO and KTA models, expected barriers and facilitators for the implementation and adoption of innovation were identified based on the CFIR (Consolidated Framework for Implementation Research). Accordingly, strategies to facilitate the implementation and adoption of innovations were identified based on a refined list of implementation strategies and QI tools. The choice of these models was based on the top 5 criteria for selecting implementation science theories and frameworks. Seven DNP students used the hybrid model to conduct QI projects. Students evaluated their experiences by responding to a Qualtrics survey.

RESULTS

The hybrid model encouraged a comprehensive systematic way of thinking, provided tools essential to implementation success, emphasized the need for adaptability in implementation, maintained rigor in QI, and guided the sustainability of change initiatives. Some of the challenges faced by students included finding reliable and valid measures, attaining and maintaining staff buy-in, and competing organizational priorities.

CONCLUSIONS

Cross-fertilization between improvement and implementation sciences provided a roadmap and systematic thinking for successful QI projects in the DNP program. The integration of the CFIR with the GTO or KTA process models, enforced by the use of evidence-based implementation strategies and QI tools, reflected the complexity of health care systems and emphasized the need for adaptability in implementation.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR1-10.2196/54213.

Improving Sexual Assault and Sexual Harassment Prevention from the Bottom-up: a Pilot of Getting To Outcomes in the US Military

While the Department of Defense (DoD) has given increased attention and priority to preventing sexual assault and sexual harassment (SA/SH), it remains a problem. To build its prevention capacity, DoD piloted Getting To Outcomes® (GTO®) from 2019 to 2022 at 10 military installations. GTO is an evidence-based planning and implementation support that has been used in many civilian contexts but has only recently been adapted for military SA/SH. The purpose of this study was to describe GTO use, identify its benefits and challenges, and discuss lessons the GTO effort yielded for prevention more broadly using a framework of organizational and program-level capacities needed for successful prevention in the military context, called the Prevention Evaluation Framework (PEF). GTO was piloted with 10 military installations ("sites") representing all Military Services, plus the Coast Guard and National Guard. GTO is comprised of a written guide, training, and ongoing coaching. The pilot's goal was for each site to use GTO to implement a SA/SH prevention program twice. Participants from each site were interviewed and data was collected on GTO steps completed, whether GTO spurred new evaluation activities and collaborations, and the degree of leadership support for GTO. Most sites completed all GTO steps at least once. Interviews showed that DoD participants believe GTO improved prevention understanding, planning, and evaluation capacity; strengthened confidence in chosen programs; and helped sites tailor programs to the military context. Barriers were the complexity of GTO, DoD personnel turnover, and the disruption that the COVID pandemic caused in sexual assault prevention program delivery. Many respondents were unsure if they would continue all of GTO after the coaching ended, but many believed they would continue at least some parts. According to the PEF, the GTO pilot revealed several additional prevention system gaps (e.g., need for leadership support) and changes needed to GTO (e.g., stronger leader and champion engagement), to support quality prevention. The military and other large organizations will need to focus on these issues to ensure prevention implementation and evaluation are conducted with quality.

A scoping review of implementation of health-focused interventions in vulnerable populations

Vulnerable populations face significant challenges in getting the healthcare they need. A growing body of implementation science literature has examined factors, including facilitators and barriers, relevant to accessing healthcare in these populations. The purpose of this scoping review was to identify themes relevant for improving implementation of healthcare practices and programs for vulnerable populations. This scoping review relied on the methodological framework set forth by Arksey and O’Malley, and the Consolidated Framework for Implementation Research (CFIR) to evaluate and structure our findings. A framework analytic approach was used to code studies. Of the five CFIR Domains, the Inner Setting and Outer Setting were the most frequently examined in the 81 studies included. Themes that were pertinent to each domain are as follows—Inner Setting: organizational culture, leadership engagement, and integration of the intervention; Outer Setting: networks, external policies, and patients’ needs and resources; Characteristics of the Individual: knowledge and beliefs about the intervention, self-efficacy, as well as stigma (i.e., other attributes); Intervention Characteristics: complexities with staffing, cost, and adaptations; and Process: staff and patient engagement, planning, and ongoing reflection and evaluation. Key themes, including barriers and facilitators, are highlighted here as relevant to implementation of practices for vulnerable populations. These findings can inform tailoring of implementation strategies and health policies for vulnerable populations, thereby supporting more equitable healthcare.

Inner and outer setting factors that influence the implementation of the National Diabetes Prevention Program (National DPP) using the Consolidated Framework for Implementation Research (CFIR): a qualitative study

BACKGROUND

Scaling evidence-based interventions are key to impacting population health. The National DPP lifestyle change program is one such intervention that has been scaled across the USA over the past 20 years; however, enrollment is an ongoing challenge. Furthermore, little is known about which organizations are most successful with program delivery, enrollment, and scaling. This study aims to understand more about the internal and external organization factors that impact program implementation and reach.

METHODS

Between August 2020 and January 2021, data were collected through semi-structured key informant interviews with 30 National DPP delivery organization implementers. This study uses a qualitative cross-case construct rating methodology to assess which Consolidated Framework for Implementation Research (CFIR) inner and outer setting constructs contributed (both in valence and magnitude) to the organization's current level of implementation reach (measured by average participant enrollment per year). A construct by case matrix was created with ratings for each CFIR construct by interviewee and grouped by implementation reach level.

RESULTS

Across the 16 inner and outer setting constructs and subconstructs, the interviewees with greater enrollment per year provided stronger and more positive examples related to implementation and enrollment of the program, while the lower reach groups reported stronger and more negative examples across rated constructs. Four inner setting constructs/subconstructs (structural characteristics, compatibility, goals and feedback, and leadership engagement) were identified as "distinguishing" between enrollment reach levels based on the difference between groups by average rating, the examination of the number of extreme ratings within levels, and the thematic analysis of the content discussed. Within these constructs, factors such as organization size and administrative processes; program fit with existing organization services and programs; the presence of enrollment goals; and active leadership involvement in implementation were identified as influencing program reach.

CONCLUSIONS

Our study identified a number of influential CFIR constructs and their impact on National DPP implementation reach. These findings can be leveraged to improve efforts in recruiting and assisting delivery organizations to increase the reach and scale of the National DPP as well as other evidence-based interventions.

Organizational Leaders Perceptions of Barriers to Accessing Behavioral Health Services in a Low-Resource Community

Little is known about how to effectively implement behavioral health programs in low-resource communities. Leaders from 20 community-serving behavioral health organizations in Flint, MI, were asked about their organizations and the barriers that they, and the populations they serve, face in providing and accessing behavioral health services. Barriers are reported using a mixed-methods analysis, reporting the number and percentage of organizations that experienced the barrier along with example quotations from the organization leaders. The most frequently reported barrier to providing services was finding adequate funding (50%) while the most frequently reported barrier for accessing services was finding adequate and reliable transportation (30%). Comparisons of these findings with barriers reported by providers in different settings and those seeking services are discussed. These comparisons may provide an important next step in identifying areas where providers perceptions and the needs of the population are misaligned and for systemic improvements more broadly.

Measuring capacity to use evidence-based interventions in community-based organizations: A comprehensive, scoping review

Introduction:

Community-based organizations (CBOs) are well-positioned to incorporate research evidence, local expertise, and contextual factors to address health inequities. However, insufficient capacity limits use of evidence-based interventions (EBIs) in these settings. Capacity-building implementation strategies are popular, but a lack of standard models and validated measures hinders progress in the field. To advance the literature, we conducted a comprehensive scoping review.

Methods:

With a reference librarian, we executed a comprehensive search strategy of PubMed/Medline, Web of Science Core Collection, and EBSCO Global Health. We included articles that addressed implementation science, capacity-building, and CBOs. Of 5527 articles, 99 met our inclusion criteria, and we extracted data using a double-coding process

Results:

Of the 99 articles, 47% defined capacity explicitly, 31% defined it indirectly, and 21% did not define it. Common concepts in definitions were skills, knowledge/expertise, and resources. Of the 57 articles with quantitative analysis, 48 (82%) measured capacity, and 11 (23%) offered psychometric data for the capacity measures. Of the 99 studies, 40% focused exclusively on populations experiencing inequities and 22% included those populations to some extent. The bulk of the studies came from high-income countries.

Conclusions:

Implementation scientists should 1) be explicit about models and definitions of capacity and strategies for building capacity, 2) specify expected multi-level implementation outcomes, 3) develop and use validated measures for quantitative work, and 4) integrate equity considerations into the conceptualization and measurement of capacity-building efforts. With these refinements, we can ensure that the necessary supports reach CBO practitioners and critical partners for addressing health inequities.

Getting To Implementation (GTI)-Teach: A seven-step approach for teaching the fundamentals of implementation science

Introduction:

Implementation Science (IS) is a complex and rapidly evolving discipline, posing challenges for educators. We developed, implemented, and evaluated a novel, pragmatic approach to teach IS.

Methods:

Getting To Implementation (GTI)-Teach was developed as a seven-step educational model to guide students through the process of developing, conducting, and sustaining an IS research project. During the four-week online course, students applied the steps to self-selected implementation problems. Students were invited to complete two online post-course surveys to assess course satisfaction and self-reported changes in IS knowledge and relevance of GTI-Teach Steps to their work. Results were summarized using descriptive statistics; self-reported post-course changes in IS knowledge were compared using paired t-tests.

Results:

GTI-Teach was developed to include seven Steps: 1. Define the implementation problem; 2. Conceptualize the problem; 3. Prioritize implementation barriers and facilitators; 4. Select and tailor implementation strategies; 5. Design an implementation study; 6. Evaluate implementation; 7. Sustain implementation. Thirteen students, ranging in experience from medical students to full professors, enrolled in and completed the first GTI-Teach course. Of the seven students (54%) completing an end-of course survey, six (86%) were very satisfied with the course. Ten students (77%) responded to the tailored, 6-month post-course follow-up survey. They retrospectively reported a significant increase in their knowledge across all steps of GTI-Teach (1.3–1.8 points on a 5-point Likert scale) and rated each of the Steps as highly relevant to their work.

Conclusions:

GTI-Teach is a seven-step model for teaching IS fundamentals that students reported increased their knowledge and was relevant to their work.

Lessons Learned from Implementing the SHIELD Intervention: A Peer Education Intervention for People Who Use Drugs

HIV prevention and care peer education interventions have demonstrated effectiveness at changing HIV risk and care behaviors among a variety of at-risk populations in different settings. However, little is known about the implementation of this type of intervention in community-based settings. Further, there is limited information available regarding the facilitators and barriers to implementing peer education interventions in community-based settings. In this study, we explore implementation facilitators, barriers, and strategies to overcome these barriers among 12 organizations that implemented the SHIELD intervention, an evidenced-based peer education intervention for people who use drugs. Guided by the Consolidated Framework for Implementation Research, we identified several facilitators and barriers at the outer, inner individuals, and intervention level of the implementation process. Future evidence-based public health programs should, in addition to addressing effectiveness, be relevant to the needs and lives of clients.

Evaluation of a school-based participatory intervention to improve school environments using the Consolidated Framework for Implementation Research

Background

Participatory research offers a promising approach to addressing health inequities and improving the social determinants of health for diverse populations of adolescents. However, little research has systematically explored factors influencing the implementation of participatory health interventions targeting health disparities.

Objective

This study examined the utility of the Consolidated Framework for Implementation Research (CFIR) in identifying and comparing barriers and facilitators influencing implementation of participatory research trials by employing an adaptation of the CFIR to assess the implementation of a multi-component, urban public school-based participatory health intervention.

Methods

We collected qualitative data over a one-year period through weekly team meeting observational field notes and regular semi-structured interviews with five community-based participatory researchers, one school-based partner, and four school principals involved in implementing a participatory intervention in five schools. Adapted CFIR constructs guided our largely deductive approach to thematic data analysis. We ranked each of the three intervention components as high or low implementation to create an overall implementation effectiveness score for all five schools. Cross-case comparison of constructs across high and low implementation schools identified constructs that most strongly influenced implementation.

Results

Ten of 30 assessed constructs consistently distinguished between high and low implementation schools in this participatory intervention, with five strongly distinguishing. Three additional constructs played influential, though non-distinguishing, roles within this participatory intervention implementation. Influential constructs spanned all five domains and fit within three broad themes: 1) leadership engagement, 2) alignment between the intervention and institutional goals, priorities, demographics, and existing systems, and 3) tensions between adaptability and complexity within participatory interventions. However, the dynamic and collaborative nature of participatory intervention implementation underscores the artificial distinction between inner and outer settings in participatory research and the individual behavior change focus does not consider how relationships between stakeholders at multiple levels of participatory interventions shape the implementation process.

Conclusions

The CFIR is a useful framework for the assessment of participatory research trial implementation. Our findings underscore how the framework can be readily adapted to further strengthen its fit as a tool to examine project implementation in this context.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-021-11644-5.

Applying a Race(ism)-Conscious Adaptation of the CFIR Framework to Understand Implementation of a School-Based Equity-Oriented Intervention

Objectives

To use the Consolidated Framework for Implementation Research (CFIR) adapted to a race-conscious frame to understand ways that structural racism interacts with intervention implementation and uptake within an equity-oriented trial designed to enhance student-school connectedness.

Design

Secondary analysis of qualitative implementation data from Project TRUST (Training for Resiliency in Urban Students and Teachers), a hybrid effectiveness-implementation, community-based participatory intervention.

Setting

Ten schools across one urban school district.

Methods

We analyzed qualitative observational field notes, youth and parent researcher reflections, and semi-structured interviews with community-academic researchers and school-based partners within CFIR constructs based on framing questions using a Public Health Critical Race Praxis approach.

Results

Within most CFIR constructs and sub-constructs, we identified barriers to implementation uptake not previously recognized using standard race-neutral definitions. Themes that crossed constructs included: 1) Leaders' willingness to examine Black, Indigenous, People of Color (BIPOC) student and parent experiences of school discrimination and marginalization had a cascading influence on multiple factors related to implementation uptake; 2) The race/ethnicity of the principals was related to intervention engagement and intervention uptake, particularly at the extremes, but the relationship was complex; 3) External change agents from BIPOC communities facilitated intervention uptake in indirect but significant ways; 4) Highly networked implementation champions had the ability to enhance commitment to intervention uptake; however, perceptions of these individuals and the degree to which they were networked was highly racialized.

Conclusions

Equity-oriented interventions should consider structural racism within the CFIR model to better understand intervention uptake.

Getting to implementation: a protocol for a Hybrid III stepped wedge cluster randomized evaluation of using data-driven implementation strategies to improve cirrhosis care for Veterans

BACKGROUND

Cirrhosis is a rapidly increasing cause of global mortality. To improve cirrhosis care, the Veterans Health Administration (VHA) developed the Hepatic Innovation Team (HIT) Collaborative to support VA Medical Centers (VAMCs) to deliver evidence-based cirrhosis care. This randomized HIT program evaluation aims to develop and assess a novel approach for choosing and applying implementation strategies to improve the quality of cirrhosis care.

METHODS

Evaluation aims are to (1) empirically determine which combinations of implementation strategies are associated with successful implementation of evidence-based practices (EBPs) for Veterans with cirrhosis, (2) manualize these "data-driven" implementation strategies, and (3) assess the effectiveness of data-driven implementation strategies in increasing cirrhosis EBP uptake. Aim 1 will include an online survey of all VAMCs' use of 73 implementations strategies to improve cirrhosis care, as defined by the Expert Recommendations for Implementing Change taxonomy. Traditional statistical as well as configurational comparative methods will both be employed to determine which combinations of implementation strategies are associated with site-level adherence to EBPs for cirrhosis. In aim 2, semi-structured interviews with high-performing VAMCs will be conducted to operationalize successful implementation strategies for cirrhosis care. These data will be used to inform the creation of a step-by-step guide to tailoring and applying the implementation strategies identified in aim 1. In aim 3, this manualized implementation intervention will be assessed using a hybrid type III stepped-wedge cluster randomized design. This evaluation will be conducted in 12 VAMCs, with four VAMCs crossing from control to intervention every 6 months, in order to assess the effectiveness of using data-driven implementation strategies to improve guideline-concordant cirrhosis care.

DISCUSSION

Successful completion of this innovative evaluation will establish the feasibility of using early evaluation data to inform a manualized, user-friendly implementation intervention for VAMCs with opportunities to improve care. This evaluation will provide implementation support tools that can be applied to enhance the implementation of other evidence-based practices.

TRIAL REGISTRATION

This project was registered at ClinicalTrials.Gov ( NCT04178096 ) on 4/29/20.

Sustaining an Evidence-Based Program Over Time: Moderators of Sustainability and the Role of the Getting to Outcomes® Implementation Support Intervention

Problematic rates of alcohol, e-cigarette, and other drug use among US adolescents highlight the need for effective implementation of evidence-based programs (EBPs), yet schools and community organizations have great difficulty implementing and sustaining EBPs. Although a growing number of studies show that implementation support interventions can improve EBP implementation, the literature on how to improve sustainability through implementation support is limited. This randomized controlled trial advances the literature by testing the effects of one such implementation intervention-Getting To Outcomes (GTO)-on sustainability of CHOICE, an after-school EBP for preventing substance use among middle-school students. CHOICE implementation was tracked for 2 years after GTO support ended across 29 Boys and Girls Club sites in the greater Los Angeles area. Predictors of sustainability were identified for a set of key tasks targeted by the GTO approach (e.g., goal setting, evaluation, collectively called "GTO performance") and for CHOICE fidelity using a series of path models. One year after GTO support ended, we found no differences between GTO and control sites on CHOICE fidelity. GTO performance was also similar between groups; however, GTO sites were superior in conducting evaluation. Better GTO performance predicted better CHOICE fidelity. Two years after GTO support ended, GTO sites were significantly more likely to sustain CHOICE implementation when compared with control sites. This study suggests that using an implementation support intervention like GTO can help low-resource settings continue to sustain their EBP implementation to help them get the most out of their investment. ClinicalTrials.gov Identifier: NCT02135991.