Adapting a Compilation of Implementation Strategies to Advance School-Based Implementation Research and Practice

Promoting 21st Century Health and Wellness Skills in Elementary School Children: a Group Randomized Trial

In a group randomized trial of 45 elementary schools, a whole-child health and wellness curriculum introduced as a regular part of the educational programming was compared to education as usual over 2 years with a 2-year follow-up. The curriculum focused on integrating multiple SEL skills-mindfulness, compassion, and physical awareness-with the intent to advantage developmental patterns for these skills, academic engagement, personal well-being, and student behavior. The program design and trial implementation were launched with end-use delivery and long-term sustainability as integral considerations. Effects were shown for several SEL skills and behavioral indicators previously robustly correlated to long-term outcomes. Effects were demonstrated more broadly in schools serving high-poverty communities, suggesting the impact was stronger where the need was greater. Results are interpreted regarding supporting school-based SEL, the potential of applying group randomized trials with end-use conditions of implementation, and the value of fitting innovation efforts to school system operations, mandates, and priorities for promoting sustainability.

Student and caregiver acceptability of a school-based intervention to improve sugar-sweetened beverage behaviors: a mixed methods study

Kids SIPsmartER is a 6-month behavioral and health literacy intervention effective at reducing sugar-sweetened beverage (SSB) intake among middle school students and their caregivers in the rural Appalachian region. This exploratory mixed methods study utilized a convergent parallel design to assess participant acceptability of a school-based curriculum for students and a text messaging program for caregivers. Acceptability was assessed using surveys (873 students and 453 caregivers), five focus groups (34 students) and telephone interviews (22 caregivers). Quantitative data were analyzed descriptively, and qualitative data were content coded. On a 5-point scale, average quantitative survey acceptability ratings ranged from 2.7 to 3.3 among three student-rated questions and 4.1 to 4.2 among four caregiver-rated questions. Qualitative focus group findings suggested that students preferred curricular activities that were hands-on and involved social interaction, while caregiver interview results showed high acceptability of the text messaging program's design, including usability, content messages and personalization. Students and caregivers reported similar program benefits: increased knowledge of SSBs and health risks, increased awareness of SSB behaviors and support to make beverage behavior changes. Results from this study can be used to understand Kids SIPsmartER's effectiveness data, communicate the intervention's acceptability with stakeholders and plan for future implementation studies.

Building on Strong Foundations: Deploying Enhanced Replicating Effective Programs for evidence-based prevention curriculum adaptation

Schools frequently adopt new interventions for each new public health issue, but this is both time- and resource-intensive. Adversity exposure is an example of a pervasive public health issue that emerged during Coronavirus Disease 2019 (COVID-19) with notable consequences, including an elevated risk of developing substance use disorders and mental illnesses. Adapting existing, universal, evidence-based interventions, such as the Michigan Model for HealthTM (MMH), by incorporating trauma-sensitive content is a promising approach to meet this need. We examined critical steps in promoting MMH adaptability as part of the Enhanced REP (Replicating Effective Programs) implementation strategy during the COVID-19 pandemic. We share usability testing from the 2020 to 2021 school year and describe how we apply the results to inform the group randomized trial pilot study. We applied key steps from implementation adaptation frameworks to integrate trauma-sensitive content as COVID-19 unfolded, documenting the process through field notes. We conducted initial usability testing with two teachers via interviews and used a rapid qualitative analysis approach. We conducted member checking by sharing the information with two health coordinators to validate results and inform additional curriculum refinement. We developed an adapted MMH curriculum to include trauma-sensitive content, with adaptations primarily centered on adding content, tailoring content, substituting content, and repeating/reinforcing elements across units. We designed adaptations to retain the core functional elements of MMH. Building foundational relationships and infrastructure supports opportunities to user-test intervention materials for Enhanced REP that enhance utility and relevance for populations that would most benefit. Enhanced REP is a promising strategy to use an existing evidence-based intervention to meet better the needs of youth exposed to adversity. Building on the foundations of existing evidence-based interventions, is vital to implementation success and achieving desired public health outcomes.

Pragmatic Adaptations to Kids SIPsmartER's Implementation Protocol Supported Its Delivery During the COVID-19 Pandemic

OBJECTIVE

Describe the adaptation, implementation, and perceptions of Kids SIPsmartER's classroom component during the coronavirus disease-impacted 2020-2021 school year.

DESIGN

Mixed methods process evaluation.

SETTING

Seven rural Appalachian middle schools (US).

PARTICIPANTS

Middle-school teachers (n = 14) and principals (n = 6).

INTERVENTION

Kids SIPsmartER was a multilevel, school-based intervention designed to decrease sugar-sweetened beverage intake. The 12-lesson classroom component was supported by an implementation protocol.

MEASURES

Implementation protocol adaptations, program perceptions, and the school context were assessed using teacher and principal interviews, teacher-completed fidelity checklists, and researcher-maintained field notes. Adaptations were mapped to the Framework for Reporting Adaptations and Modifications-Enhanced (FRAME).

ANALYSIS

Qualitative data were content coded. Quantitative data were summarized using descriptive statistics.

RESULTS

All schools maintained Kids SIPsmartER and delivered 100% of lessons. Ten adaptations were made to the implementation protocol. Schools used adapted delivery approaches to meet individual needs. Teachers and principals identified more benefits than barriers to implementing the program.

CONCLUSIONS AND IMPLICATIONS

Using a strategically adapted implementation protocol that was flexible to schools' individual needs allowed all middle schools to deliver Kids SIPsmartER during the 2020-2021 school year. Findings identify adaptation considerations that other school-based evidence-based interventions could incorporate to facilitate delivery during high-stress times.

Optimization of implementation strategies using the Multiphase Optimization STratgey (MOST) framework: Practical guidance using the factorial design

The Multiphase Optimization STrategy (MOST) is a framework that uses three phases-preparation, optimization, and evaluation-to develop multicomponent interventions that achieve intervention EASE by strategically balancing Effectiveness, Affordability, Scalability, and Efficiency. In implementation science, optimization of the intervention requires focus on the implementation strategies-things that we do to deliver the intervention-and implementation outcomes. MOST has been primarily used to optimize the components of the intervention related to behavioral or health outcomes. However, innovative opportunities to optimize discrete (i.e. single strategy) and multifaceted (i.e. multiple strategies) implementation strategies exist and can be done independently, or in conjunction with, intervention optimization. This article details four scenarios where the MOST framework and the factorial design can be used in the optimization of implementation strategies: (i) the development of new multifaceted implementation strategies; (ii) evaluating interactions between program components and a discrete or multifaceted implementation strategies; (iii) evaluating the independent effects of several discrete strategies that have been previously evaluated as a multifaceted implementation strategy; and (iv) modification of a discrete or multifaceted implementation strategy for the local context. We supply hypothetical school-based physical activity examples to illustrate these four scenarios, and we provide hypothetical data that can help readers make informed decisions derived from their trial data. This manuscript offers a blueprint for implementation scientists such that not only is the field using MOST to optimize the effectiveness of an intervention on a behavioral or health outcome, but also that the implementation of that intervention is optimized.

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): a modified Delphi study

BACKGROUND

Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources.

METHOD

MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability and inclusion (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis.

RESULTS

In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application.

DISCUSSION

MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

Academic and behavior combined support: A single-case practice-based replication study

The purpose of this research was to conduct a practice-based replication of Academic and Behavior Combined Support (ABC Support), a previously developed and experimentally evaluated supplemental intervention that merges a combined focus on reading fluency and academic engagement. In the present study, a school-based interventionist and data collector had access to implementation resources online and participated in virtual training and coaching. Four Grade 2 students received the ABC Support intervention for 6 weeks in their school. Students' oral fluency on training and non-training reading passages, as well as occurrence of engagement and disruptive behaviors during universal reading instruction, were measured repeatedly across baseline, intervention, and follow-up phases in a multiple-baseline design. In concert with prior empirical findings on ABC Support, analyses revealed improvement from baseline to intervention for both reading and behavior outcomes, as well as from baseline to follow-up assessments. Empirical contributions of the study are offered within the context of replication research and an implementation science perspective. We also emphasize the importance of telecommunication for practice-based research evaluation of interventions.

Associations between a technical assistance model to build school organizational capacity and improvements in student perceptions of school climate

Organizational capacity building-the process of developing leadership, collecting and analyzing data, building buy-in, and implementing programming-is foundational to effectively changing schools, and frequently relies on technical assistance. This study employed a quasi-experimental, repeated measured design to evaluate the role of technical assistance provided through Safe School Certification model in improving school climate. Schools worked through an eight-element framework, using data from a sample of six middle and high schools in Washington, D.C. that completed data collection in all years of the evaluation. Students in schools receiving technical assistance for implementing the SSC Framework had more positive changes in perceptions of school climate than students in schools that did not receive support, but those differences were small. The results from this study offer limited evidence that providing schools with technical assistance to improve organizational capacity is associated with more positive student perceptions of school climate.

Antibullying Interventions in Schools: Assessing the Evidence Base

OBJECTIVE

This systematic review used established rating criteria to describe the level of evidence for interventions aimed at preventing or reducing bullying perpetration and victimization in schools, synthesized the evidence for students from racially and ethnically diverse backgrounds, and reviewed the literature for available information to conduct an economic analysis of the interventions.

METHODS

Major databases, gray literature, and evidence-base registries were searched to identify studies published from 2008 through 2022. The authors rated antibullying intervention models as having high, moderate, or low evidence depending on the number and rigor of studies with positive findings.

RESULTS

Overall, 80 articles reporting on 71 original research studies describing a total of 48 antibullying interventions met the inclusion criteria for this review. Two schoolwide interventions received a high-evidence rating: the KiVa (Kiusaamista Vastaan) Antibullying Program and the Friendly Schools program. Multilevel interventions with components at the levels of school, classroom, and individual student most consistently showed strong evidence for reducing bullying behavior in elementary and middle school grades. Four interventions yielded positive effects in reducing bullying and victimization among diverse samples of students.

CONCLUSIONS

Antibullying interventions can reduce bullying in schools. Some interventions show effectiveness with students from racially and ethnically diverse backgrounds. The gains relative to per-student costs were in the range that is considered cost-effective. Most implementation costs are spent on staff training and support. Research on successful implementation of whole-school interventions and additional synthesis of evidence pertaining to program structures would further advance the antibullying evidence base.

Development of an implementation plan for a school-based multimodal approach for depression and suicide prevention in adolescents

Strong Teens and Resilient Minds (STORM) is a multimodal, school-based approach for depression and suicide prevention in adolescents that is currently implemented in a region in the Netherlands. The STORM approach will be implemented in new regions in the coming years. This study used the implementation mapping protocol to report on the development of the STORM implementation plan. First, a needs assessment was conducted through semi-structured interviews with stakeholders and brainstorming sessions with regional programme leaders in the two regions that started implementing STORM in 2023. This led to the identification of six main barriers to implementation: high level of demands for schools, insufficient understanding of the programme content, insufficient network collaboration, no perceived relative advantage of STORM by stakeholders, lack of attention to sustainability, and high work pressure. Second, performance and change objectives were formulated based on these barriers. For example, a performance objective for potential providers was that they felt supported by STORM. Third, implementation strategies were selected from theory and translated into practical applications through brainstorming sessions with programme leaders. The following strategies were included in the implementation plan: collaborate with similar initiatives within the region, free up time for STORM tasks, tailor strategies, identify and prepare STORM champions, and promote network weaving. Last, a plan to evaluate the implementation of STORM and the application of the STORM implementation plan was formulated. Planned evaluation research will provide more insight into the usefulness and impact of the STORM implementation plan.

Building a sharable literature collection to advance the science and practice of implementation facilitation

Background

Implementation science seeks to produce generalizable knowledge on strategies that promote the adoption and sustained use of evidence-based innovations. Literature reviews on specific implementation strategies can help us understand how they are conceptualized and applied, synthesize findings, and identify knowledge gaps. Although rigorous literature reviews can advance scientific knowledge and facilitate theory development, they are time-consuming and costly to produce. Improving the efficiency of literature review processes and reducing redundancy of effort is especially important for this rapidly developing field. We sought to amass relevant literature on one increasingly used evidence-based strategy, implementation facilitation (IF), as a publicly available resource.

Methods

We conducted a rigorous systematic search of PubMed, CINAHL, and Web of Science citation databases for peer-reviewed, English-language articles with "facilitation" and a combination of other terms published from January 1996 to December 2021. We searched bibliographies of articles published from 1996 to 2015 and identified articles during the full text review that reported on the same study. Two authors screened 3,168 abstracts. After establishing inter-rater reliability, they individually conducted full-text review of 786 relevant articles. A multidisciplinary team of investigators provided recommendations for preparing and disseminating the literature collection.

Findings

The literature collection is comprised of 510 articles. It includes 277 empirical studies of IF and 77 other articles, including conceptual/theoretical articles, literature reviews, debate papers and descriptions of large-scale clinical initiatives. Over half of the articles were published between 2017 and 2021. The collection is publicly available as an Excel file and as an xml file that can be imported into reference management software.

Conclusion

We created a publicly accessible collection of literature about the application of IF to implement evidence-based innovations in healthcare. The comprehensiveness of this collection has the potential to maximize efficiency and minimize redundancy in scientific inquiry about this strategy. Scientists and practitioners can use the collection to more rapidly identify developments in the application of IF and to investigate a wide range of compelling questions on its use within and across different healthcare disciplines/settings, countries, and payer systems. We offer several examples of how this collection has already been used.

Maintaining Program Fidelity in a Changing World: National Implementation of a School-Based HIV Prevention Program

Large-scale, evidence-based interventions face challenges to program fidelity of implementation. We developed implementation strategies to support teachers implementing an evidence-based HIV prevention program in schools, Focus on Youth in The Caribbean (FOYC) and Caribbean Informed Parents and Children Together (CImPACT) in The Bahamas. We examined the effects of these implementation strategies on teachers' implementation in the subsequent year after the initial implementation during the COVID-19 pandemic. Data were collected from 79 Grade 6 teachers in 24 government elementary schools. Teachers completed training workshops and a pre-implementation questionnaire to record their characteristics and perceptions that might affect their program fidelity. School coordinators and peer mentors provided teachers with monitoring, feedback, and mentoring. In Year 1, teachers on average taught 79.3% of the sessions and 80.8% of core activities; teachers in Year 2 covered 84.2% of sessions and 72.9% of the core activities. Teachers with "good" or "excellent" school coordinators in the second year taught significantly more sessions on average (7.8 vs. 7.0, t = 2.04, P < 0.05) and more core activities (26.3 vs. 23.0, t = 2.41, P < 0.05) than teachers with "satisfactory" coordinators. Teachers who had a "good" or "satisfactory" mentor taught more sessions than teachers who did not have a mentor (7.9 vs. 7.3; t = 2.22; P = 0.03). Two-level mixed-effects model analysis indicated that teachers' program fidelity in Year 1, confidence in the execution of core activities, and school coordinators' performance were significantly associated with Year 2 implementation dose. Implementation of FOYC + CImPACT was significantly associated with improved student outcomes. Teachers maintained high fidelity to a comprehensive HIV prevention program over 2 years during the COVID-19 pandemic. Future program implementers should consider additional implementation support to improve the implementation of school-based programs.

Using personal network analysis to understand the interaction between programmes' facilitators and teachers in psychoeducational interventions

In this study we use personal network analysis to examine the social relationships taking place throughout programme implementation. Previous literature on psychosocial intervention has used network analysis techniques to examine: (a) the interaction between participants, (b) the facilitators' link to the intervention target group, (c) the transfer of knowledge between experts and facilitators, and (d) the interaction of facilitators with each other. However, there has been little research on how facilitators connect with other figures in their organisational context, impacting both intervention fidelity and the fit of the programme to the local context. In this study we combine the analysis of personal networks with qualitative interviews with 102 teachers in 72 schools in Barranquilla (Colombia), with whom we describe the implementation of the psychoeducational programmes Pisotón and Metodologías Flexibles. The results show that programme implementation networks not only rely on facilitators but also on the contribution of the schools' director of studies, the coupling with the regular classroom teachers, and occasional collaborations from the rest of the school staff. After conducting a cluster analysis, we detected the existence of two types of personal networks, some based on the functioning of highly cohesive teams and others with a greater level of centralisation around the head of studies. Implementation networks not only enable the implementation of programme activities, but also the integration of the programme into educational organisations. In the discussion we reflect on how the analysis of facilitators' personal networks can be used to improve the process of programme implementation.

Core Components and Implementation Determinants of Multilevel Service Delivery Frameworks Across Child Mental Health Service Settings

Multilevel service delivery frameworks are approaches to structuring and organizing a spectrum of evidence-based services and supports, focused on assessment, prevention, and intervention designed for the local context. Exemplar frameworks in child mental health include positive behavioral interventions and supports in education, collaborative care in primary care, and systems of care in community mental health settings. Yet, their high-quality implementation has lagged. This work proposes a conceptual foundation for multilevel service delivery frameworks spanning diverse mental health service settings that can inform development of strategic implementation supports. We draw upon the existing literature for three exemplar multilevel service delivery frameworks in different child mental health service settings to (1) identify core components common to each framework, and (2) to highlight prominent implementation determinants that interface with each core component. Six interrelated components of multilevel service delivery frameworks were identified, including, (1) a systems-level approach, (2) data-driven problem solving and decision-making, (3) multiple levels of service intensity using evidence-based practices, (4) cross-linking service sectors, (5) multiple providers working together, including in teams, and (6) built-in implementation strategies that facilitate delivery of the overall model. Implementation determinants that interface with core components were identified at each contextual level. The conceptual foundation provided in this paper has the potential to facilitate cross-sector knowledge sharing, promote generalization across service settings, and provide direction for researchers, system leaders, and implementation intermediaries/practitioners working to strategically support the high-quality implementation of these frameworks.

The perspective of school leaders on the implementation of evidence-based practices: A mixed methods study

Background

School leaders play an integral role in the use of implementation strategies, which in turn support special education teachers in the implementation of evidence-based practices (EBPs). In this convergent mixed methods study, we explored school leaders' perceptions of the facilitators and barriers to EBP implementation, particularly for students receiving special education, as well as the importance and feasibility of 15 implementation strategies.

Method

School leaders (N  =  22, principals, assistant principals, school psychologists, etc.) participated in a semistructured interview that included three parts-qualitative questions, quantitative ratings of strategies' importance and feasibility, and discussion of the top three implementation strategies. Data strands were analyzed independently and then integrated to generate meta-inferences.

Results

The qualitative data identified facilitators such as access to resources about the intervention (e.g., professional development) and collaboration and teamwork, while barriers centered on lack of school supports, culture/climate, and organizational factors (e.g., lack of communication). The quantitative data indicated that the implementation strategy provide ongoing consultation/coaching was rated as important and feasible. Monitor the progress of the implementation effort was rated as important but less feasible, while conduct educational meetings and change the environment were rated as feasible, but less important. Build partnerships to support implementation was rated as less important and feasible. There was convergence and divergence in mixed methods findings.

Conclusion

This study underscores the critical need to increase school leaders' knowledge and skills related to implementation science to better leverage implementation strategies that address the confluence of relevant implementation determinants.

Using ERIC to Assess Implementation Science in Drowning Prevention Interventions in High-Income Countries: A Systematic Review

This systematic review identifies and describes the use of the Expert Recommendation for Implementing Change (ERIC) concepts and strategies using public health approaches to drowning prevention interventions as a case study. International calls for action have identified the need to better understand the implementation of drowning prevention interventions so that intervention design and implementation is improved. In high-income countries (HICs), interventions are sophisticated but still little is known or written about their implementation. The review was registered on PROSPERO (number CRD42022347789) and followed the PRISMA guidelines. Eight databases were searched. Articles were assessed using the Public Health Ontario Meta-tool for quality appraisal of public health evidence. Forty-nine articles were included. Where ERIC strategies were reported, the focus was on evaluative and iterative strategies, developing partnerships and engaging the target group. The review identified few articles that discussed intervention development and implementation sufficiently for strategies to be replicated. Findings will inform further research into the use and measurement of implementation strategies by practitioners and researchers undertaking work in drowning prevention in HICs and supports a call to action for better documentation of implementation in public health interventions.

The Longitudinal Implementation Strategy Tracking System (LISTS): feasibility, usability, and pilot testing of a novel method

BACKGROUND

Systematic approaches are needed to accurately characterize the dynamic use of implementation strategies and how they change over time. We describe the development and preliminary evaluation of the Longitudinal Implementation Strategy Tracking System (LISTS), a novel methodology to document and characterize implementation strategies use over time.

METHODS

The development and initial evaluation of the LISTS method was conducted within the Improving the Management of SymPtoms during And following Cancer Treatment (IMPACT) Research Consortium (supported by funding provided through the NCI Cancer MoonshotSM). The IMPACT Consortium includes a coordinating center and three hybrid effectiveness-implementation studies testing routine symptom surveillance and integration of symptom management interventions in ambulatory oncology care settings. LISTS was created to increase the precision and reliability of dynamic changes in implementation strategy use over time. It includes three components: (1) a strategy assessment, (2) a data capture platform, and (3) a User's Guide. An iterative process between implementation researchers and practitioners was used to develop, pilot test, and refine the LISTS method prior to evaluating its use in three stepped-wedge trials within the IMPACT Consortium. The LISTS method was used with research and practice teams for approximately 12 months and subsequently we evaluated its feasibility, acceptability, and usability using established instruments and novel questions developed specifically for this study.

RESULTS

Initial evaluation of LISTS indicates that it is a feasible and acceptable method, with content validity, for characterizing and tracking the use of implementation strategies over time. Users of LISTS highlighted several opportunities for improving the method for use in future and more diverse implementation studies.

CONCLUSIONS

The LISTS method was developed collaboratively between researchers and practitioners to fill a research gap in systematically tracking implementation strategy use and modifications in research studies and other implementation efforts. Preliminary feedback from LISTS users indicate it is feasible and usable. Potential future developments include additional features, fewer data elements, and interoperability with alternative data entry platforms. LISTS offers a systematic method that encourages the use of common data elements to support data analysis across sites and synthesis across studies. Future research is needed to further adapt, refine, and evaluate the LISTS method in studies with employ diverse study designs and address varying delivery settings, health conditions, and intervention types.

Using Patient-Centered Dissemination and Implementation Frameworks and Strategies in Palliative Care Settings for Improved Quality of Life and Health Outcomes: A Scoping Review

BACKGROUND

There is a need for patient-provider dissemination and implementation frameworks, strategies, and protocols in palliative care settings for a holistic approach when it comes to addressing pain and other distressing symptoms affecting the quality of life, function, and independence of patients with chronic illnesses. The purpose of this scoping review is to explore patient-centered D&I frameworks and strategies that have been adopted in PC settings to improve behavioral and environmental determinants influencing health outcomes through evidence-based programs and protocols.

METHODS

The five step Arksey and O'Malley's (2005) York methodology was adopted as a guiding framework: (1) identifying research questions; (2) searching for relevant studies; (3) selecting studies relevant to the research questions; (4) charting the data; and (5) collating, summarizing, and reporting results.

RESULTS

Only 6 out of the 38 (16%) included studies applied a D&I theory and/or framework. The RE-AIM framework was the most prominently cited (n = 3), followed by the Diffusion of Innovation Model (n = 2), the CONNECT framework (n = 1), and the Transtheoretical Stages of Change Model (n = 1). The most frequently reported ERIC strategy was strategy #6 "Develop and organize quality monitoring systems", as it identified in all 38 of the included studies.

CONCLUSION

This scoping review identifies D&I efforts to translate research into practice in U.S. palliative care settings. Results may contribute to enhancing future D&I initiatives for dissemination/adaptation, implementation, and sustainability efforts aiming to improve patient health outcomes and personal satisfaction with care received.

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): A modified Delphi study

Background

Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources.

Method

MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis.

Results

In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application.

Discussion

MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

A Generic Implementation Framework for School-Based Research and Practice

PURPOSE

One of our biggest challenges is integrating evidence-based research into practice to serve students with communication disorders. To encourage the systematic application of research findings into practice, implementation science offers frameworks and tools, many of which have a narrow scope. It is important to have comprehensive frameworks that encompass all essential implementation concepts to support implementation in schools.

METHOD

Guided by the generic implementation framework (GIF; Moullin et al., 2015), we reviewed implementation science literature to identify and tailor frameworks and tools covering all core concepts of implementation: (a) the process of implementation, (b) the domains and determinants of practice, (c) implementation strategies, and (d) evaluations.

RESULTS

We created a version of the GIF for school settings, called the GIF-School, to bring together frameworks and tools that sufficiently cover core concepts of implementation. The GIF-School is accompanied by an open access toolkit, which lists selected frameworks, tools, and useful resources.

CONCLUSION

Researchers and practitioners, in speech-language pathology and education more broadly, who seek to apply implementation science frameworks and tools to improve school services for students with communication disorders may turn to the GIF-School as a resource.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.23605269.

Identifying Implementation Strategies to Increase Augmentative and Alternative Communication Adoption in Early Childhood Classrooms: A Qualitative Study

PURPOSE

The goal of this study was to identify promising implementation strategies that may increase teachers' and speech-language pathologists' (SLPs) adoption of augmentative and alternative communication (AAC) during routine classroom activities.

METHOD

A grounded theory approach was used to collect and code online, semistructured interviews with six special education teachers and 14 SLPs who taught children aged 3-8 years who used AAC in inclusive and self-contained special education classrooms. Interviews were transcribed and coded verbatim using the constant comparative method. During a member checking process, six participants (n = 30%) confirmed their transcripts and commented on emerging themes.

RESULTS

Participants shared a range of implementation strategies to increase AAC use during typical classroom activities, which were organized into seven themes: distribute modern AAC equipment, provide foundational training, share video examples, develop a systematic plan for adoption, deliver practice-based coaching, alter personnel obligations to provide protected time, and connect with professionals. A preliminary thematic map was created to link implementation barriers, strategies, and potential outcomes.

CONCLUSIONS

AAC interventions effective in clinical research can be difficult to translate into routine practice unless investigators directly explore the needs of and demands on educational professionals. Future research should define implementation strategies clearly, solicit feedback from school-personnel, and match implementation strategies to the needs of local schools to support the uptake of AAC interventions in routine classroom settings.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.23699757.

Implementation strategies in the Exploration and Preparation phases of a colorectal cancer screening intervention in community health centers

BACKGROUND

Adoption of colorectal cancer (CRC) screening has lagged in community health center (CHC) populations in the USA. To address this implementation gap, we developed a multilevel intervention to improve screening in CHCs in our region. We used the Exploration, Preparation, Implementation, Sustainment (EPIS) framework to guide this effort. Here, we describe the use of implementation strategies outlined in the Expert Recommendations for Implementing Change (ERIC) compilation in both the Exploration and Preparation phases of this project. During these two EPIS phases, we aimed to answer three primary questions: (1) What factors in the inner and outer contexts may support or hinder colorectal cancer screening in North Carolina CHCs?; (2) What evidence-based practices (EBPs) best fit the needs of North Carolina CHCs?; and (3) How can we best integrate the selected EBPs into North Carolina CHC systems?

METHODS

During the Exploration phase, we conducted local needs assessments, built a coalition, and conducted local consensus discussions. In the Preparation phase, we formed workgroups corresponding to the intervention's core functional components. Workgroups used cyclical small tests of change and process mapping to identify implementation barriers and facilitators and to adapt intervention components to fit inner and outer contexts.

RESULTS

Exploration activities yielded a coalition of stakeholders, including two rural CHCs, who identified barriers and facilitators and reached consensus on two EBPs: mailed FIT and navigation to colonoscopy. Stakeholders further agreed that the delivery of those two EBPs should be centralized to an outreach center. During Preparation, workgroups developed and refined protocols for the following centrally-delivered intervention components: a registry to identify and track eligible patients, a centralized system for mailing at-home stool tests, and a process to navigate patients to colonoscopy after an abnormal stool test.

CONCLUSIONS

This description may be useful both to implementation scientists, who can draw lessons from applied implementation studies such as this to refine their implementation strategy typologies and frameworks, as well as to implementation practitioners seeking exemplars for operationalizing strategies in early phases of implementation in healthcare.

Exploring Classroom Factors and Augmentative and Alternative Communication Use in Qualitative Interviews

PURPOSE

This qualitative study explores classroom contextual factors influencing augmentative and alternative communication (AAC) use in early childhood classrooms.

METHOD

Semistructured interviews were conducted with six special education teachers, 13 speech-language pathologists (SLPs), and one SLP assistant. Interviews were analyzed inductively in a systematic, multistep coding process using the constant comparative method with member checking.

RESULTS

Seven themes were identified that affected AAC use in the classroom. Each theme was organized into three categories: individual-, classroom-, or school-level factors. Three themes were categorized as individual-level factors: Students' AAC systems are unavailable, professionals face challenges obtaining AAC knowledge and skills, and students' disruptive behaviors reduce access to AAC. One theme was a classroom-level factor: Classroom demands prevent AAC use. Finally, three themes were considered school-level factors: The culture of AAC influences AAC acceptance, team collaboration is essential for successful implementation, and teams need administrative support to deliver AAC interventions effectively. Conclusions: Teachers and SLPs felt that collaborative teams with AAC expertise, a school culture that endorsed AAC, and solid administrative support accelerated AAC adoption by normalizing AAC use in the classroom. Due to classroom demands, they struggled to use strategies such as aided AAC modeling and responsiveness during group instruction. Finally, participants emphasized that while it was necessary to customize AAC systems for each student, this individualization makes it harder for staff to learn each system and integrate them into classroom activities.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.23681718.

Applying hybrid effectiveness-implementation studies in equity-centered policy implementation science

Policy implementation science (IS) is complex, dynamic, and fraught with unique study challenges that set it apart from biomedical or clinical research. One important consideration is the ways in which policy interacts with local contexts, such as power and social disadvantage (e.g., based on ability, race, class, sexual identity, geography). The complex nature of policy IS and the need for more intentional integration of equity principles into study approaches calls for creative adaptations to existing implementation science knowledge and guidance. Effectiveness-implementation hybrid studies were developed to enhance translation of clinical research by addressing research questions around the effectiveness of an intervention and its implementation in the same study. The original work on hybrid designs mainly focused on clinical experimental trials; however, over the last decade, researchers have applied it to a wide range of initiatives and contexts, including more widespread application in community-based studies. This perspectives article demonstrates how effectiveness-implementation hybrid studies can be adapted for and applied to equity-centered policy IS research. We draw upon principles of targeted universalism and Equity in Implementation Research frameworks to guide adaptations to hybrid study typologies, and suggest research and engagement activities to enhance equity considerations; for example, in the design and testing of implementing strategies. We also provide examples of equity-centered policy IS studies. As the field of policy IS rapidly evolves, these adapted hybrid type studies are offered to researchers as a starting guide.

An unexplored equity factor: differential beliefs and attitudes toward contingency management by providers' ethnicity

BACKGROUND

Although considered one of the most effective interventions for substance use disorders (SUD), the widespread implementation of contingency management (CM) has remained limited. In more recent years there has been surge in the implementation of CM to address increasing rates of substance use. Prior studies at the provider-level have explored beliefs about CM among SUD treatment providers and have tailored implementation strategies based on identified barriers and training needs, to promote implementation of CM. However, there have been no implementation strategies that have actively sought to identify or address potential differences in the beliefs about CM that could be influenced by the cultural background (e.g., ethnicity) of treatment providers. To address this knowledge gap, we examined beliefs about CM among a sample of inpatient and outpatient SUD treatment providers.

METHODS

A cross-sectional survey of SUD treatment providers was completed by 143 respondents. The survey asked respondents about their attitudes toward CM using the Contingency Management Beliefs Questionnaire (CMBQ). Linear mixed models examined the effect of ethnicity (non-Hispanic White and Hispanic) on CMBQ subscale (general barriers, training-related barriers, CM positive-statements) scores.

RESULTS

Fifty-nine percent of respondents to the CMBQ self-identified as non-Hispanic White and 41% as Hispanic. Findings revealed that treatment providers who identified as Hispanic had significantly higher scores on the general barriers (p < .001) and training-related barriers (p = .020) subscales compared to the non-Hispanic White treatment providers. Post-hoc analyses identified differences in the endorsement of specific individual scale items on the general barriers (e.g., CM interventions create extra work for me) and training-related (e.g., I want more training before implementing CM) subscales.

CONCLUSIONS

Dissemination and implementation strategies for CM need to consider equity-related factors at the provider-level that may be associated with the adoption and uptake of CM.

Acceptability, appropriateness, and feasibility of Rural School Support Strategies for behavioral interventions: a mixed methods evaluation over two years of a hybrid type 3 implementation-effectiveness trial

BACKGROUND

Positive Behavioral Interventions and Supports (PBIS) is a framework for implementing evidence-based interventions for preventing behavioral issues and improving climate in schools. The implementation of school-wide PBIS with fidelity is complex, requiring leadership commitment, teaming, and coordination of systems for tracking behaviors and consequences. Putting these components in place while ensuring alignment with the values and needs of the school community can be difficult for schools with fewer resources, such as rural schools. Implementation supports are needed, including strategies such as technical assistance, but it is unclear whether lower-cost modalities such as virtual support are acceptable, appropriate, and feasible and whether perceptions vary throughout the implementation process.

METHODS

A type 3 hybrid implementation-effectiveness trial is taking place in 40 Idaho schools, testing a bundle of implementation supports selected to meet the needs of schools in rural areas. Supports include technical assistance from an implementation support practitioner (ISP), didactic trainings, virtual learning sessions, and an online resource portal. Surveys and interviews in the first 2 years of implementation (fall 2019 to spring 2021) explored outcomes of acceptability, appropriateness, and feasibility regarding the implementation supports among more than 150 school stakeholders.

RESULTS

Evaluations showed high acceptability and appropriateness of the PBIS concepts and training. The 20 schools receiving additional implementation support rated the technical assistance and support from the project's ISPs as the most acceptable and appropriate resource. Reasons for acceptability were the relationship built with the ISP, the ISP's expertise, and being a "neutral party." Although in-person support from the ISP was preferred, remote support was acceptable and increased feasibility of attendance. Virtual learning sessions were acceptable for learning and collaboration, particularly in the second year of implementation, once ISPs had developed closer relationships with school teams.

CONCLUSIONS

School staff found training, technical assistance, and virtual learning sessions to be acceptable and appropriate. Virtual formats of training and technical assistance decreased in acceptability but increased feasibility of attendance. In-person support was preferred during initial implementation, and virtual support was more acceptable thereafter.

TRIAL REGISTRATION

This trial was prospectively registered on ClinicalTrials.gov ( NCT03736395 ), on November 9, 2018.

Overcoming challenges in conducting systematic reviews in implementation science: a methods commentary

Consolidation of the literature using systematic reviews is a critical way to advance a discipline and support evidence-based decision-making in healthcare. However, unique challenges exist that impact the conduct of systematic reviews in implementation science. In this commentary, we reflect on our combined experience to describe five key challenges unique to systematic reviews of primary implementation research. These challenges include (1) descriptors used in implementation science publications, (2) distinction between evidence-based interventions and implementation strategies, (3) assessment of external validity, (4) synthesis of implementation studies with substantial clinical and methodological diversity, and (5) variability in defining implementation 'success'. We outline possible solutions and highlight resources that can be used by authors of primary implementation research, as well as systematic review and editorial teams, to overcome the identified challenges and optimise the utility of future systematic reviews in implementation science.

Initial insights into the impact and implementation of Creating Active Schools in Bradford, UK

BACKGROUND

Few whole-school physical activity programmes integrate implementation science frameworks within the design, delivery, and evaluation. As a result, knowledge of the key factors that support implementation at scale is lacking. The Creating Active Schools (CAS) programme was co-designed and is underpinned by the Capability, Opportunity, Motivation and Behaviour (COM-B) model and the Consolidated Framework for Implementation Research (CFIR). The study aims to understand the initial impact and implementation of CAS in Bradford over 9 months using McKay's et al.'s (2019) implementation evaluation roadmap.

METHODS

Focus groups and interviews were conducted with school staff (n = 30, schools = 25), CAS Champions (n = 9), and the CAS strategic lead (n = 1). Qualitative data were analysed both inductively and deductively. The deductive analysis involved coding data into a priori themes based on McKay et al's implementation evaluation roadmap, using a codebook approach to thematic analysis. The inductive analysis included producing initial codes and reviewing themes before finalising.

RESULTS

Identified themes aligned into three categories: (i) key ingredients for successful adoption and implementation of CAS, (ii) CAS implementation: challenges and solutions, and (iv) the perceived effectiveness of CAS at the school level. This included the willingness of schools to adopt and implement whole-school approaches when they are perceived as high quality and aligned with current school values. The programme implementation processes were seen as supportive; schools identified and valued the step-change approach to implementing CAS long-term. Formal and informal communities of practice provided "safe spaces" for cross-school support. Conversely, challenges persisted with gaining broader reach within schools, school staff's self-competence and shifting school culture around physical activity. This resulted in varied uptake between and within schools.

CONCLUSIONS

This study provides novel insights into the implementation of CAS, with outcomes aligning to the adoption, reach, and sustainability. Successful implementation of CAS was underpinned by determinants including acceptability, intervention complexity, school culture and school stakeholders' perceived self-efficacy. The combination of McKay's evaluation roadmap and CFIR establishes a rigorous approach for evaluating activity promotion programmes underpinned by behavioural and implementation science. Resultantly this study offers originality and progression in understanding the implementation and effectiveness of whole-school approaches to physical activity.

Implementing a Digital Depression Prevention Program in Australian Secondary Schools: Cross-Sectional Qualitative Study

BACKGROUND

Depression is common during adolescence and is associated with adverse educational, employment, and health outcomes in later life. Digital programs are increasingly being implemented in schools to improve and protect adolescent mental health. Although digital depression prevention programs can be effective, there is limited knowledge about how contextual factors influence real-world delivery at scale in schools.

OBJECTIVE

The purpose of this study was to examine the contextual factors that influence the implementation of the Future Proofing Program (FPP) from the perspectives of school staff. The FPP is a 2-arm hybrid type 1 effectiveness-implementation trial evaluating whether depression can be prevented at scale in schools, using an evidence-based smartphone app delivered universally to year 8 students (13-14 years of age).

METHODS

Qualitative interviews were conducted with 23 staff from 20 schools in New South Wales, Australia, who assisted with the implementation of the FPP. The interviews were guided by our theory-driven logic model. Reflexive thematic analysis, using both deductive and inductive coding, was used to analyze responses.

RESULTS

Staff perceived the FPP as a novel ("innovative approach") and appropriate way to address an unmet need within schools ("right place at the right time"). Active leadership and counselor involvement were critical for planning and engaging; teamwork, communication, and staff capacity were critical for execution ("ways of working within schools"). Low student engagement and staffing availability were identified as barriers for future adoption and implementation by schools ("reflecting on past experiences").

CONCLUSIONS

Four superordinate themes pertaining to the program, implementation processes, and implementation barriers were identified from qualitative responses by school staff. On the basis of our findings, we proposed a select set of recommendations for future implementation of digital prevention programs delivered at scale in schools. These recommendations were designed to facilitate an organizational change and help staff to implement digital mental health programs within their schools.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2020-042133.

An external facilitation intervention to increase uptake of an adverse drug event reporting intervention

Background

Adverse drug events (ADEs) are a leading cause of emergency department visits and hospital admissions in Canada. ActionADE prevents repeat ADEs by enabling clinicians to document and communicate standardized ADE information across care settings. We used an external facilitation intervention to promote the uptake of ActionADE in four hospitals in British Columbia, Canada. This study examined whether, how and in what context external facilitation influenced the uptake of ActionADE.

Methods

In this convergent-parallel mixed-methods study, an external facilitator used a four-step iterative process to support site champions using context-specific implementation strategies to increase the ADE reporting rate at their sites. We extracted archival data to assess implementation determinants before and after the implementation of the external facilitation and implementation strategies. We also retrieved data on the mean monthly counts of reported ADEs for each user from the ActionADE server. Zero-inflated Poisson models were used to examine changes in mean monthly counts of reported ADEs per user between pre-intervention (June 2021 to October 2021) and intervention (November 2021 to March 2022) periods.

Results

The external facilitator and site champions co-created three functions: (1) educate pharmacists about what and how to report in ActionADE, (2) educate pharmacists about the impact of ActionADE on patient outcomes, and (3) provide social support for pharmacists to integrate ADE reporting into clinical workflows. Site champions used eight forms to address the three functions. Peer support and reporting competition were the two common strategies used by all sites. Sites' responses to external facilitation varied. The rate of mean monthly counts of reported ADEs per user significantly increased during the intervention period compared to the pre-intervention period at LGH (RR: 3.74, 95% CI 2.78 to 5.01) and RH (RR: 1.43, 95% CI 1.23 to 1.94), but did not change at SPH (RR: 0.68, 95% CI: 0.43 to 1.09) and VGH (RR: 1.17, 95% CI 0.92 to 1.49). Leave of absence of the clinical pharmacist champion and failure to address all identified functions were implementation determinants that influenced the effectiveness of external facilitation.

Conclusion

External facilitation effectively supported researchers and stakeholders to co-create context-specific implementation strategies. It increased ADE reporting at sites where clinical pharmacist champions were available, and where all functions were addressed.

Aligning Organizational Priorities and System Policies to Support Implementation Scale-Up of a Tailored Classroom-Based Physical Activity Intervention in Low-Resource Schools

BACKGROUND

A mismatch between organizational priorities and system-level policies can negatively impact implementation and sustainment of classroom-based physical activity (PA) interventions. The purpose of this study was twofold: (1) present methods to systematically identify organization- and system-level implementation barriers, and (2) align organizational priorities and system policies by designing multi-level implementation strategies. This alignment will support implementation scale-up of a tailored PA intervention in one low-resource intermediate school district (ISD; 16 districts, 32 schools) in central Michigan.

METHODS

Multi-level assessments of organizational readiness were conducted using the Hexagon Discussion and Analysis Tool to assess intervention-context fit, the Wellness School Assessment Tool 3.0 to evaluate district PA policy strength and comprehensiveness, and semi-structured interviews were conducted to assess administrative support and priorities related to PA programming.

RESULTS

Our assessments revealed three implementation barriers: limited structural capacity to sustain teacher training, limited resources across districts and school buildings to support teachers, and misalignment of ISD and district PA policies and priorities.

CONCLUSIONS

Greater attention to organizational capacity and existing infrastructure should be considered a priori to support effective implementation and sustainment of PA interventions in low-resource schools.

Refining Expert Recommendations for Implementing Change (ERIC) strategy surveys using cognitive interviews with frontline providers

BACKGROUND

The Expert Recommendations for Implementing Change (ERIC) compilation includes 73 defined implementation strategies clustered into nine content areas. This taxonomy has been used to track implementation strategies over time using surveys. This study aimed to improve the ERIC survey using cognitive interviews with non-implementation scientist clinicians.

METHODS

Starting in 2015, we developed and fielded annual ERIC surveys to evaluate liver care in the Veterans Health Administration (VA). We invited providers who had completed at least three surveys to participate in cognitive interviews (October 2020 to October 2021). Before the interviews, participants reviewed the complete 73-item ERIC survey and marked which strategies were unclear due to wording, conceptual confusion, or overlap with other strategies. They then engaged in semi-structured cognitive interviews to describe the experience of completing the survey and elaborate on which strategies required further clarification.

RESULTS

Twelve VA providers completed surveys followed by cognitive interviews. The "Engage Consumer" and "Support Clinicians" clusters were rated most highly in terms of conceptual and wording clarity. In contrast, the "Financial" cluster had the most wording and conceptual confusion. The "Adapt and Tailor to Context" cluster strategies were considered to have the most redundancy. Providers outlined ways in which the strategies could be clearer in terms of wording (32%), conceptual clarity (51%), and clarifying the distinction between strategies (51%).

CONCLUSIONS

Cognitive interviews with ERIC survey participants allowed us to identify and address issues with strategy wording, combine conceptually indistinct strategies, and disaggregate multi-barreled strategies. Improvements made to the ERIC survey based on these findings will ultimately assist VA and other institutions in designing, evaluating, and replicating quality improvement efforts.

Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy

BACKGROUND

Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers' implementation of evidence-based psychosocial interventions-(1) audit and feedback and (2) active learning.

METHODS

Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples.

RESULTS

In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ2=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ2=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples.

CONCLUSIONS

Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04096274 . Registered on 19 September 2019.

A Pre-Implementation Enhancement Strategy to Increase the Yield of Training and Consultation for School-Based Behavioral Preventive Practices: a Triple-Blind Randomized Controlled Trial

As the most common setting where youth access behavioral healthcare, the education sector frequently employs training and follow-up consultation as cornerstone implementation strategies to promote the implementation of evidence-based practices (EBPs). However, these strategies alone are not sufficient to promote desirable implementation (e.g., intervention fidelity) and youth behavioral outcomes (e.g., mitigated externalizing behaviors). Theory-informed pragmatic pre-implementation enhancement strategies (PIES) are needed to prevent the lackluster outcomes of training and consultation. Specifically, social cognitive theory explicates principles that inform the design of PIES content and specify mechanisms of behavior change (e.g., "intentions to implement" (ITI)) to target increasing providers' responsiveness to training and consultation. This triple-blind parallel randomized controlled trial preliminarily examined the efficacy of a pragmatic PIES based on social cognitive theories (SC-PIES) to improve implementation and youth behavioral outcomes from universal preventive EBPs in the education sector. Teachers from a diverse urban district were recruited and randomly assigned to the treatment (SC-PIES; ntreatment = 22) or active control condition (administrative meeting; ncontrol = 21). Based on the condition assigned, teachers received the SC-PIES or met with administrators before their EBP training. We assessed teachers' ITI, intervention fidelity, and youth behavioral outcome (academic engagement as an incompatible behavior to externalizing disorders) at baseline, immediately after training, and 6 weeks afterward. A series of ANCOVAs detected sizeable effects of SC-PIES, where teachers who received SC-PIES demonstrated significantly larger improvement in their ITI, intervention fidelity, and youth behaviors as compared to the control. Conditional analyses indicated that teachers' ITI partially mediated the effect of SC-PIES on intervention fidelity, which in turn led to improved youth behaviors. Findings suggest that theory-informed pragmatic PIES targeting providers' ITI can boost their responsiveness to implementation strategies, as reflected in improved implementation behaviors and youth behavioral outcomes. The results have implications for targeting motivational mechanisms of behavior change and situating preventive implementation strategies at the intersection between the preparation and active implementation stages of an implementation process. Limitations and implications for research and practice are discussed. Clinicaltrials.gov: NCT05240222. Registered on: 2/14/2022. Retrospectively registered.  https://clinicaltrials.gov/show/NCT05240222.

Exploring Intervention Sustainment and Intervention Spread Following a Randomized Clinical Trial of the MOSAIC Program

The randomized trial of the Making Socially Accepting Inclusive Classrooms (MOSAIC) program included intensive coaching from research staff to support teachers' implementation of MOSAIC strategies and resulted in positive student outcomes (Mikami et al., J. Clin. Child Adolesc. Psychol. 51(6):1039-1052, 2022). However, these intensive procedures are costly (in time, money, and resources) and serve as barriers to intervention adoption under typical school conditions. In this study, we explored the extent to which MOSAIC-trained teachers could sustain practices under typical practice conditions (sustainment), the extent to which teachers who did not participate in the trial could adopt the practices under typical practice conditions (spread), and the extent to which strategy use in the follow-up year was associated with participation in MOSAIC-focused professional learning communities (PLCs). Participants were 30 elementary school teachers, including (a) 13 teachers who received intensive coaching on MOSAIC practices during the previous year (MOSAIC group), and (b) seven teachers who participated in the trial in the control condition, plus 10 new teachers interested in MOSAIC (new-to-MOSAIC group). We assessed MOSAIC strategy use over the school year via monthly observations and biweekly teacher self-report surveys. Observation data revealed high sustainment in the MOSAIC group, with teachers showing less than 20% decline in the use of most strategies between the two years of participation. New-to-MOSAIC teachers implemented some core MOSAIC strategies, although not to the extent as those in the MOSAIC group. Higher strategy use was modestly associated with PLC attendance. We discuss implications for encouraging sustainment and intervention spread after initial, intensive supports are withdrawn.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12310-022-09555-w.

Mixed methods implementation evaluation of virtual interview training for transition-age autistic youth in pre-employment transition services

BACKGROUND

Autistic transition-age youth are employed at rates far lower than their non-disabled peers as well as youth with other disabilities. Meanwhile, very few studies have evaluated the implementation of job interviewing practices within pre-employment transition services.

OBJECTIVE

We conducted an initial implementation evaluation as part of a Type I hybrid randomized controlled effectiveness-implementation trial where we trained teachers to deliver Virtual Interview Training for Transition-Age Youth (VIT-TAY) within five pre-employment transition services programs.

METHODS

We used mixed methods to evaluate leader (n=5), teacher (n=15) and autistic transition age youth (n=48) perceptions of VIT-TAY. We used descriptive statistics and thematic network analysis to evaluate survey data. Mixed methods integration was then performed to make comparisons between quantitative and qualitative results.

RESULTS

Quantitative survey data revealed that leaders and teachers found VIT-TAY to be highly acceptable and appropriate for pre-employment transition services; findings which were confirmed via thematic network analysis of qualitative interview data. Autistic students reported via quantitative surveys that VIT-TAY was acceptable and usable, which was confirmed via thematic network analysis of open-ended survey data.

CONCLUSIONS

This initial implementation evaluation can be used to inform a larger scale implementation evaluation of VIT-TAY in schools.

Partnerships at the Interface of Education and Mental Health Services: The Utilisation and Acceptability of the Provision of Specialist Liaison and Teacher Skills Training

Partnerships between school staff and mental health professionals have the potential to improve access to mental health support for students, but uncertainty remains regarding whether and how they work in practice. We report on two pilot projects aimed at understanding the implementation drivers of tailored strategies for supporting and engaging front-line school staff in student mental health. The first project provided regular, accessible mental health professionals with whom school staff could meet and discuss individual or systemic mental health concerns (a school 'InReach' service), and the other offered a short skills training programme on commonly used psychotherapeutic techniques (the School Mental Health Toolbox; SMHT). The findings from the activity of 15 InReach workers over 3 years and 105 individuals who attended the SMHT training demonstrate that school staff made good use of these services. The InReach workers reported more than 1200 activities in schools (notably in providing specialist advice and support, especially for anxiety and emotional difficulties), whilst most SMHT training attendees reported the utilisation of the tools (in particular, supporting better sleep and relaxation techniques). The measures of acceptability and the possible impacts of the two services were also positive. These pilot studies suggest that investment into partnerships at the interface of education and mental health services can improve the availability of mental health support to students.

Strategies for Enhancing the Implementation of Universal Mental Health Prevention Programs in Schools: A Systematic Review

A number of school-based mental health prevention programs have been found to be effective in research trials, but little is known about how to support implementation in real-life settings. To address this translational problem, this systematic review aims to identify effective strategies for enhancing the implementation of mental health prevention programs for children in schools. Four electronic databases were searched for empirical, peer-reviewed articles in English from January 2000 to October 2021 reporting the effects of implementation strategies for school-based universal mental health programs. Twenty-one articles were included in the narrative synthesis and assessed for quality using the Mixed Methods Appraisal Tool. Twenty-two strategies were found to be effective at improving program fidelity or adoption. The strategies with the strongest positive evidence base were those that involved monitoring and provision of feedback, engaging principals as program leaders, improving teachers' buy-in and organising school personnel implementation meetings. We recommend school-based practitioners trial strategies with positive findings from this review as part of their continuous quality improvement. This review highlights the pressing need for large-scale, randomised controlled trials to develop and trial more robust strategies to enhance adoption, as the five implementation studies found to measure adoption used qualitative methods limited by small samples sizes and case study designs.

Implementation strategies and outcomes of school-based programs for adolescent suicide prevention: A scoping review protocol

OBJECTIVE

This scoping review aims to identify and map the empirical literature on the implementation strategies and outcomes of school-based programs for adolescent suicide prevention (SBASP).

INTRODUCTION

School-based programs are preferred interventions for preventing suicide in adolescents, and their effectiveness has been well-systematized in several reviews. Implementation research is a growing field for prevention programs, making it possible to understand the nature of success or failure outcomes and maximize intervention benefits. However, there is a knowledge gap in the implementation research applied to adolescent suicide prevention in the educational context. We conduct a scoping review to provide the first overview of the scope of implementation research applied to adolescent suicide prevention programs in the school setting to know what implementation strategies and outcomes are reported by these programs and how they are evaluated.

METHODS

The proposed scoping review will be conducted following six stages, including the definition of objectives. Studies must be empirical and address implementation strategies or implementation outcomes of school-based programs for adolescent suicide prevention. Studies that focused exclusively on clinical efficacy or effectiveness evaluation will be excluded. A preliminary search of PubMed was conducted to refine the initial search strings, followed by a final search of several other electronic databases. Finally, a gray literature search will identify unpublished literature and reduce location bias. There will be no limits to a specific date. Two independent reviewers will screen, select, and extract the retrieved records. The results will be presented using tabular forms and a narrative summary with attention to the review objectives and research questions and their implications for research and practice of school-based programs for adolescent suicide prevention.

Louisiana trucking companies' implementation of workplace health promotion programs: Explanatory sequential mixed method case study

BACKGROUND

Trucking companies may be key partners for workplace health promotion programs to improve heavy and tractor-trailer drivers' health and wellbeing.

OBJECTIVE

To identify barriers and facilitators to implementing workplace health promotion programs among Louisiana trucking companies' leadership and staff.

METHODS

A case study approach following an explanatory sequential mixed method research design was used. A quantitative survey, based on the Theoretical Domains Framework (TDF), was adapted and distributed online to a convenience sample. Survey respondents were recruited for an interview to gain additional insight on multi-level barriers to implementing workplace health promotion programs. Quantitative data was analyzed using descriptive statistics to describe barriers and facilitators following TDF constructs. Qualitative data were independently coded among two researchers following the TDF and the Consolidated Framework for Implementation Research (CFIR) to determine themes.

RESULTS

Eleven workplace leaders or staff took the survey. Two engaged in a follow-up interview. Regarding the quantitative results, most (82%) believed workplace health promotion programs would save their company money, although were not offering them. No TDF constructs were indicated as barriers given mixed results; however, several were facilitators: Social/Professional Role and Identity; Emotion; Action Planning; Knowledge; Motivation and Goals; and Beliefs about Consequences. Qualitative results captured several Inner (e.g., time, money) and Outer Setting contextual (e.g., truckers' needs and resources) factors considered important to trucking companies' implementation of health promotion programs.

CONCLUSION

Results suggest leadership and staff of Louisiana trucking companies value workplace health promotion programs, although are challenged by limited resources and the broader trucking environment.

Do the Expert Recommendations for Implementing Change (ERIC) strategies adequately address sustainment?

Background

Sustainability science is an emerging area within implementation science. There is limited evidence regarding strategies to best support the continued delivery and sustained impact of evidence-based interventions (EBIs). To build such evidence, clear definitions, and ways to operationalize strategies specific and/or relevant to sustainment are required. Taxonomies and compilations such as the Expert Recommendations for Implementing Change (ERIC) were developed to describe and organize implementation strategies. This study aimed to adapt, refine, and extend the ERIC compilation to incorporate an explicit focus on sustainment. We also sought to classify the specific phase(s) of implementation when the ERIC strategies could be considered and applied.

Methods

We used a two-phase iterative approach to adapt the ERIC. This involved: (1) adapting through consensus (ERIC strategies were mapped against barriers to sustainment as identified via the literature to identify if existing implementation strategies were sufficient to address sustainment, needed wording changes, or if new strategies were required) and; (2) preliminary application of this sustainment-explicit ERIC glossary (strategies described in published sustainment interventions were coded against the glossary to identify if any further amendments were needed). All team members independently reviewed changes and provided feedback for subsequent iterations until consensus was reached. Following this, and utilizing the same consensus process, the Exploration, Preparation, Implementation and Sustainment (EPIS) Framework was applied to identify when each strategy may be best employed across phases.

Results

Surface level changes were made to the definitions of 41 of the 73 ERIC strategies to explicitly address sustainment. Four additional strategies received deeper changes in their definitions. One new strategy was identified: Communicate with stakeholders the continued impact of the evidence-based practice. Application of the EPIS identified that at least three-quarters of strategies should be considered during preparation and implementation phases as they are likely to impact sustainment.

Conclusion

A sustainment-explicit ERIC glossary is provided to help researchers and practitioners develop, test, or apply strategies to improve the sustainment of EBIs in real-world settings. Whilst most ERIC strategies only needed minor changes, their impact on sustainment needs to be tested empirically which may require significant refinement or additions in the future.

The Healthy Native Youth Implementation Toolbox: Using Implementation Mapping to adapt an online decision support system to promote culturally-relevant sexual health education for American Indian and Alaska Native youth

Background

American Indian and Alaska Native (AI/AN) youth experience serious disparities in sexual and reproductive health, including the highest teen birth rate among racial/ethnic groups, and disproportionate rates of sexually transmitted infections (STI), including HIV. A growing number of evidence-based programs (EBPs) that integrate the strengths and cultural teachings of Native communities exist. Yet, multiple factors, including lack of trained personnel, limited resources, and geographic isolation, may hinder their adoption and implementation. Innovative implementation strategies that facilitate the adoption and implementation of sexual health EBPs in Native communities may help reduce these disparities.

Methods

We applied Implementation Mapping, a systematic planning framework that utilizes theory, empirical evidence, and community input, to adapt a theory-based, online decision support system, iCHAMPSS (CHoosing And Maintaining Effective Programs for Sex Education in Schools), to support underlying dissemination and implementation processes unique to Native communities. We used an iterative design process, incorporating input from Native practitioners and academicians, to ensure that the adapted decision support system reflects cultural identification, community values, and experiences.

Results

Grounded in diffusion of innovations, organizational stage theory, and social cognitive theory, the Healthy Native Youth Implementation Toolbox supports Native practitioners through five phases (Gather, Choose, Prepare, Implement, and Grow) to adopt, implement, and maintain a culturally-relevant, age-appropriate sexual health EBP. The Toolbox provides tools, ready-to-use templates, and guidance to plan, implement, and grow a culturally-relevant adolescent health program with their Tribe or community. Hosted within the Healthy Native Youth website (www.healthynativeyouth.org), the Toolbox comprises: (1) a curriculum portal with access to 15 culturally-relevant, age-appropriate evidence-based health promotion programs for AI/AN youth; (2) a "resource library" comprising 20+ support tools, templates, and links to external resources, and (3) "stories from the field" comprising testimonials from experienced Native educators, who have implemented sexual health programs.

Conclusion

There is a continued need to design, test, and evaluate D&I strategies that are relevant to Native communities. The Healthy Native Youth Implementation Toolbox contributes to the dissemination and implementation of evidence-based, culturally-relevant sexual health education programs in diverse Native communities. Implementation Mapping provided a systematic approach to guide the adaptation process and integrate community voice with the ultimate goal of enhancing sexual health equity among AI/AN youth.

What implementation strategies are relational? Using Relational Theory to explore the ERIC implementation strategies

The identification and use of implementation strategies in implementation research and practice have strengthened our understanding of the implementation process as well as the causal pathways between mechanisms, strategies, and implementation outcomes. Although these contributions have advanced the application of strategies, there is still a need to learn more about how strategies might integrate relational exchanges and interactions. The inclusion of critical perspectives has been limited in implementation science, and theories such as Relational Theory can expand our understanding of the relational nature of implementation and enhance rigor through alternative theoretical applications. This study applied Relational Theory through a qualitative directed content analysis of the 73 Expert Recommendations for Implementation Change (ERIC) implementation strategies and examine relational components in strategy descriptions. Three reviewers used the structured approach to review and categorize the implementation strategies based on the Relational and Transactional Strategy Continuum measure, which operationalizes types of interactions, exchanges and alliances. Relational alliance strategies are those in which there is mutual growth and accountability, frequent interaction, shared power, and potential vulnerability. Operational alliances include forms of working exchanges between parties with balanced transactional and relational features. Operational alliances can be somewhat interactive in nature, with minor exchanges and limited accountability. Transactional alliance strategies are mostly uni-directional, influenced by power differentials, and do not require mutual growth, commitment, or exchange; thus, the power of growth is inherently one-sided. Results from the review suggest more implementation strategies with relational alliance features (highly relational, n = 17, semi-relational, n = 19) compared to transactional (highly transactional, n = 9, semi-transactional, n = 10) and 18 strategies coded as operational alliances. The qualitative review revealed opportunities to further expand how relational exchanges are considered within the implementation strategies descriptions, as well as the role of actors and power dynamics within strategy exchanges. The Relational and Transactional Strategy Continuum measure can help practitioners and researchers consider the sequencing, pairing, and impact on outcomes of different types and combinations of strategies in implementation practice and research. Additionally, the measure can support reflection on strategies that promote positive alliances, frequent connections, bi-directional communication, and power sharing.

Implementing a school-based HIV prevention program during public health emergencies: lessons learned in The Bahamas

Background

Natural disasters and public health crises can disrupt communities’ capacities to implement important public health programs. A nationwide implementation of an evidence-based HIV prevention program, Focus on Youth in The Caribbean (FOYC) and Caribbean Informed Parents and Children Together (CImPACT), in The Bahamas was disrupted by Hurricane Dorian and the COVID-19 pandemic, especially in its more remote, Family Islands. We explored the teacher- and school-level factors that affected implementation of the program in these islands during those disruptions.

Methods

Data were collected from 47 Grade 6 teachers and 984 students in 34 government elementary schools during the 2020–2021 school year. Teachers completed a pre-implementation questionnaire to record their characteristics and perceptions that might affect their implementation fidelity and an annual program training workshop. School coordinators and high-performing teachers acting as mentors received additional training to provide teachers with monitoring, feedback, and additional support. Teachers submitted data on their completion of the 9 sessions and 35 core activities of FOYC + CImPACT. The fidelity outcomes were the number of sessions and core activities taught by teachers.

Results

On average, teachers taught 60% of sessions and 53% of core activities. Teachers with “very good” school coordinators (34% of teachers) taught more activities than those with “satisfactory” (43%) or no (34%) school coordinator (27.5 vs. 16.8 vs. 14.8, F = 12.86, P < 0.001). Teachers who had attended online training or both online and in-person training taught more sessions (6.1 vs. 6.2 vs. 3.6, F = 4.76, P < 0.01) and more core activities (21.1 vs. 20.8 vs. 12.6, F = 3.35, P < 0.05) than those who received no training. Teachers’ implementation was associated with improved student outcomes (preventive reproductive health skills, self-efficacy, and intention).

Conclusions

The Hurricane Dorian and the COVID-19 pandemic greatly disrupted education in The Bahamas Family Islands and affected implementation of FOYC + CImPACT. However, we identified several strategies that supported teachers’ implementation following these events. Teacher training and implementation monitoring increased implementation fidelity despite external challenges, and students achieved the desired learning outcomes. These strategies can better support teachers’ implementation of school-based interventions during future crises.

Determinants of School Diabetes Care Services and Quality Through the Lens of the School Nurse: A Qualitative Analysis

Schools play a critical role in students' diabetes management and ensure their safety and well-being. We conducted key informant interviews with 11 school nurses in Missouri to assess determinants for diabetes care implementation. Five themes and 29 subthemes were identified concerning school nurses, schools, external stakeholders, government, and the COVID-19 pandemic. A social-ecological model was developed to elucidate each level's barriers, facilitators, and resources, and their interplay. School nurses should lead diabetes management, emergency planning, and health education for students/staff. Multiple gray areas existed regarding school nurses' specific roles/responsibilities. Lacking funding, insurance, and communication with parents/physicians further challenged diabetes care.

Dosage in Implementation of an Effective School-Based Health Program Impacts Youth Health Risk Behaviors and Experiences

PURPOSE

This study is part of a larger evaluation of a multilevel, multistrategy federal program to reduce high school students' risk for HIV/sexually transmitted infection and unintended pregnancy. Local education agencies supported schools in implementing three strategies: delivering exemplary sexual health education, increasing student access to quality sexual health services, and enhancing safe and supportive school environments (SSE). We examined how levels of school implementation of these strategies moderated program effects on targeted student outcomes.

METHODS

The Youth Risk Behavior Survey was implemented in participating local education agencies in 2015 and 2017 to assess student behaviors and experiences, whereas the School Health Profiles surveys assessed school policies and practices in 2014 and 2016. We used these surveys to measure student-level outcomes and school-level program delivery, respectively, which were analyzed using multilevel modeling in a difference-in-differences framework.

RESULTS

Levels of SSE implementation significantly moderated program effects on multiple student outcomes, including ever having sex, having four or more lifetime sexual partners, being sexually active, using hormonal birth control, dual use of a condom and hormonal birth control, ever being forced to have sex, missing school because of safety concerns, and lifetime and current marijuana use. However, we found few moderating effects of exemplary sexual health education and sexual health services dosage.

DISCUSSION

We found a significant relationship between incremental increases in implementation of activities to increase the safety and supportiveness of school environments and enhanced program effects in improving multiple student health outcomes. These findings suggest that school implementation of SSE activities contributed to intended program effects.