Development and Preliminary Evaluation of an Implementation Facilitation Training Program

Building a sharable literature collection to advance the science and practice of implementation facilitation

Background

Implementation science seeks to produce generalizable knowledge on strategies that promote the adoption and sustained use of evidence-based innovations. Literature reviews on specific implementation strategies can help us understand how they are conceptualized and applied, synthesize findings, and identify knowledge gaps. Although rigorous literature reviews can advance scientific knowledge and facilitate theory development, they are time-consuming and costly to produce. Improving the efficiency of literature review processes and reducing redundancy of effort is especially important for this rapidly developing field. We sought to amass relevant literature on one increasingly used evidence-based strategy, implementation facilitation (IF), as a publicly available resource.

Methods

We conducted a rigorous systematic search of PubMed, CINAHL, and Web of Science citation databases for peer-reviewed, English-language articles with "facilitation" and a combination of other terms published from January 1996 to December 2021. We searched bibliographies of articles published from 1996 to 2015 and identified articles during the full text review that reported on the same study. Two authors screened 3,168 abstracts. After establishing inter-rater reliability, they individually conducted full-text review of 786 relevant articles. A multidisciplinary team of investigators provided recommendations for preparing and disseminating the literature collection.

Findings

The literature collection is comprised of 510 articles. It includes 277 empirical studies of IF and 77 other articles, including conceptual/theoretical articles, literature reviews, debate papers and descriptions of large-scale clinical initiatives. Over half of the articles were published between 2017 and 2021. The collection is publicly available as an Excel file and as an xml file that can be imported into reference management software.

Conclusion

We created a publicly accessible collection of literature about the application of IF to implement evidence-based innovations in healthcare. The comprehensiveness of this collection has the potential to maximize efficiency and minimize redundancy in scientific inquiry about this strategy. Scientists and practitioners can use the collection to more rapidly identify developments in the application of IF and to investigate a wide range of compelling questions on its use within and across different healthcare disciplines/settings, countries, and payer systems. We offer several examples of how this collection has already been used.

Getting to Fidelity: Consensus Development Process to Identify Core Activities of Implementation Facilitation

Transferring successful implementation strategies from research to practice requires approaches for assessing fidelity to the strategy's core components. Implementation facilitation (IF) is a strategy involving an interactive process of problem-solving, enabling, and supporting individuals in efforts to implement clinical innovations that occurs in the context of a recognized need for improvement and supportive interpersonal relationships. Because IF is a dynamic strategy involving numerous activities, our objective was to conduct a rigorous consensus development process to identify core activities for monitoring fidelity to IF when applied in clinical settings. We first conducted a scoping literature review to identify the range of activities used when IF has been applied in clinical settings, searching multiple citation databases for English-language articles including "facilitation" or other commonly-used terms for the strategy published from 1996-2015. Through multi-stage screening, 135 articles (from 94 studies) were identified for data extraction on IF activities, frequency with which IF activities were identified as 'core' by study authors, and study outcomes. From the literature review, we identified 32 distinct IF activities and developed definitions/examples for each. Next, we conducted a 3-stage, modified-Delphi expert panel consensus development process to identify core IF activities across three implementation phases (i.e., Pre-Implementation, Implementation, Sustainment). The expert panel identified 8 core activities for the Pre-Implementation Phase, 8 core activities for the Implementation Phase, and 4 core activities for the Sustainment Phase. This work provides an important foundation for developing measures/tools to assess use of core IF activities to ensure the strategy is delivered with fidelity.

Impact of Implementation Facilitation on the REACH VET Clinical Program for Veterans at Risk for Suicide

OBJECTIVE

In 2017, the Veterans Health Administration (VHA) implemented a national suicide prevention program, called Recovery Engagement and Coordination for Health-Veterans Enhanced Treatment (REACH VET), that uses a predictive algorithm to identify, attempt to reach, assess, and care for patients at the highest risk for suicide. The authors aimed to evaluate whether facilitation enhanced implementation of REACH VET at VHA facilities not meeting target completion rates.

METHODS

In this hybrid effectiveness-implementation type 2 program evaluation, a quasi-experimental pre-post design was used to assess changes in implementation outcome measures evaluated 6 months before and 6 months after onset of facilitation of REACH VET implementation at 23 VHA facilities. Measures included percentages of patients with documented coordinator and provider acknowledgment of receipt, care evaluation, and outreach attempt. Generalized estimating equations were used to compare differences in REACH VET outcome measures before and after facilitation. Qualitative interviews were conducted with personnel and were explored via template analysis.

RESULTS

Time had a significant effect in all outcomes models (p<0.001). An effect of facilitation was significant only for the outcome of attempted outreach. Patients identified by REACH VET had significantly higher odds of having a documented outreach attempt after facilitation of REACH VET implementation, compared with before facilitation. Site personnel felt supported and reported that the external facilitators were helpful and responsive.

CONCLUSIONS

Facilitation of REACH VET implementation was associated with an improvement in outreach attempts to veterans identified as being at increased risk for suicide. Outreach is critical for engaging veterans in care.

Evaluation of implementation facilitation integrated into a national mentoring programme to improve access to evidence-based psychotherapy for post-traumatic stress disorder within the veterans health administration: a quality improvement report

Despite the resources dedicated to specialised mental healthcare for patients with post-traumatic stress disorder (PTSD) within the US Veterans Health Administration, evidence-based psychotherapies (EBPs) for PTSD have been underutilised, as evidenced by low EBP reach to patients. A research-operation collaboration evaluated whether implementation facilitation delivered by regional PTSD mentors as part of a national mentoring programme improved EBP reach compared with less-intensive quality improvement interventions. We used a non-equivalent comparison-group design that included all PTSD clinics with low EBP reach at baseline (n=51). Clinics were grouped into one of four quality improvement conditions according to self-selection by regional PTSD mentors: facilitation (n=6), learning collaborative (n=15), mentoring as usual in the regions that had facilitation-target clinics (n=15) and mentoring as usual in other regions (n=15). The primary outcome was EBP reach among therapy patients with PTSD at preintervention baseline and postintervention sustainment periods. We used the ratio of odds ratios (ROR) between the two time periods to evaluate the effectiveness of facilitation compared with the other conditions, adjusting for patient-level and clinic-level confounders. 26 126 veterans with PTSD received psychotherapy in one of 51 low-reach PTSD clinics during preintervention baseline and postintervention sustainment periods. The odds of a patient receiving an EBP increased over time across conditions. The adjusted ORs of a patient receiving an EBP from baseline to sustainment were 1.35-1.69 times larger in clinics that received facilitation compared with the three comparison conditions (adjusted RORs of comparison condition versus facilitation ranged from 0.59 (95% CI 0.47 to 0.75) to 0.74 (95% CI 0.58 to 0.94)). Implementation facilitation can be integrated into a national programme for quality improvement for PTSD specialty care and may be particularly useful when less-intensive approaches are not sufficiently effective.

Evaluation of a brief virtual implementation science training program: the Penn Implementation Science Institute

BACKGROUND

To meet the growing demand for implementation science expertise, building capacity is a priority. Various training opportunities have emerged to meet this need. To ensure rigor and achievement of specific implementation science competencies, it is critical to systematically evaluate training programs.

METHODS

The Penn Implementation Science Institute (PennISI) offers 4 days (20 h) of virtual synchronous training on foundational and advanced topics in implementation science. Through a pre-post design, this study evaluated the sixth PennISI, delivered in 2022. Surveys measures included 43 implementation science training evaluation competencies grouped into four thematic domains (e.g., items related to implementation science study design grouped into the "design, background, and rationale" competency category), course-specific evaluation criteria, and open-ended questions to evaluate change in knowledge and suggestions for improving future institutes. Mean composite scores were created for each of the competency themes. Descriptive statistics and thematic analysis were completed.

RESULTS

One hundred four (95.41% response rate) and 55 (50.46% response rate) participants completed the pre-survey and post-survey, respectively. Participants included a diverse cohort of individuals primarily affiliated with US-based academic institutions and self-reported as having novice or beginner-level knowledge of implementation science at baseline (81.73%). In the pre-survey, all mean composite scores for implementation science competencies were below one (i.e., beginner-level). Participants reported high value from the PennISI across standard course evaluation criteria (e.g., mean score of 3.77/4.00 for overall quality of course). Scores for all competency domains increased to a score between beginner-level and intermediate-level following training. In both the pre-survey and post-survey, competencies related to "definition, background, and rationale" had the highest mean composite score, whereas competencies related to "design and analysis" received the lowest score. Qualitative themes offered impressions of the PennISI, didactic content, PennISI structure, and suggestions for improvement. Prior experience with or knowledge of implementation science influenced many themes.

CONCLUSIONS

This evaluation highlights the strengths of an established implementation science institute, which can serve as a model for brief, virtual training programs. Findings provide insight for improving future program efforts to meet the needs of the heterogenous implementation science community (e.g., different disciplines and levels of implementation science knowledge). This study contributes to ensuring rigorous implementation science capacity building through the evaluation of programs.

Getting to implementation: Adaptation of an implementation playbook

Introduction

Implementation strategies supporting the translation of evidence into practice need to be tailored and adapted for maximum effectiveness, yet the field of adapting implementation strategies remains nascent. We aimed to adapt "Getting To Outcomes"® (GTO), a 10-step implementation playbook designed to help community-based organizations plan and evaluate behavioral health programs, into "Getting To Implementation" (GTI) to support the selection, tailoring, and use of implementation strategies in health care settings.

Methods

Our embedded evaluation team partnered with operations, external facilitators, and site implementers to employ participatory methods to co-design and adapt GTO for Veterans Health Administration (VA) outpatient cirrhosis care improvement. The Framework for Reporting Adaptations and Modifications to Evidenced-based Implementation Strategies (FRAME-IS) guided documentation and analysis of changes made pre- and post-implementation of GTI at 12 VA medical centers. Data from multiple sources (interviews, observation, content analysis, and fidelity tracking) were triangulated and analyzed using rapid techniques over a 3-year period.

Results

Adaptations during pre-implementation were planned, proactive, and focused on context and content to improve acceptability, appropriateness, and feasibility of the GTI playbook. Modifications during and after implementation were unplanned and reactive, concentrating on adoption, fidelity, and sustainability. All changes were collaboratively developed, fidelity consistent at the level of the facilitator and/or implementer.

Conclusion

GTO was initially adapted to GTI to support health care teams' selection and use of implementation strategies for improving guideline-concordant medical care. GTI required ongoing modification, particularly in steps regarding team building, context assessment, strategy selection, and sustainability due to difficulties with step clarity and progression. This work also highlights the challenges in pragmatic approaches to collecting and synthesizing implementation, fidelity, and adaptation data.

Trial registration

This study was registered on ClinicalTrials.gov (Identifier: NCT04178096).