Protocol: Adaptive Implementation of Effective Programs Trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program

Student mental health outcomes of a clustered SMART for developing an adaptive implementation strategy to support school-based CBT delivery

PURPOSE

Most youth experiencing anxiety/depression lack access to evidence-based mental health practices (EBPs). School-delivered care improves access, and various support can help school professionals (SPs; school social workers, counselors) deliver EBPs, like Cognitive Behavioral Therapy (CBT). Understanding implementation strategies' impact on downstream mental health outcomes is crucial to scaling up EBPs to address the treatment gap, but it has rarely been assessed.

METHODS

This paper compares implementation strategies' impact on change in student outcomes, collected as exploratory outcomes from a type III hybrid implementation-effectiveness trial. A clustered, sequential, multiple-assignment randomized trial design was used, which embedded four implementation supports that differentially sequence three implementation strategies, Replicating Effective Programs (REP), Coaching, and Facilitation. Prior to the first randomization, N = 169 SPs from 94 Michigan high schools each identified up to 10 students whom they believed could benefit from CBT and facilitated student survey completion. Changes in students' depression (Patient Health Questionnaire-9, modified for teens) and anxiety symptoms (Generalized Anxiety Disorder-7) over 10 months were compared across the four sequences of implementation support using a generalization of a marginal, weighted least squares approach developed for a clustered SMARTs.

RESULTS

Small, non-clinically significant reductions in symptoms over the study period were found. Pairwise comparisons found no significant differences in symptom change across the four implementation strategies. The difference in the estimated mean PHQ-9T/GAD-7 scores between the least and the most intensive strategies (REP vs. REP+Coaching+Facilitation) was 1.04 (95%CI = -0.95, 3.04) for depression and 0.82 (95%CI = -0.89, 2.52) for anxiety.

DISCUSSION

No difference in symptom change was found across the four implementation strategies. Multiple forms of implementation support may be useful for improving student mental health outcomes.

TRIAL REGISTRATION

NCT03541317-Registered on 29 May 2018 on ClinicalTrials.gov PRS.

A research agenda to advance the study of implementation mechanisms

BACKGROUND

Implementation science scholars have made significant progress identifying factors that enable or obstruct the implementation of evidence-based interventions, and testing strategies that may modify those factors. However, little research sheds light on how or why strategies work, in what contexts, and for whom. Studying implementation mechanisms-the processes responsible for change-is crucial for advancing the field of implementation science and enhancing its value in facilitating equitable policy and practice change. The Agency for Healthcare Research and Quality funded a conference series to achieve two aims: (1) develop a research agenda on implementation mechanisms, and (2) actively disseminate the research agenda to research, policy, and practice audiences. This article presents the resulting research agenda, including priorities and actions to encourage its execution.

METHOD

Building on prior concept mapping work, in a semi-structured, 3-day, in-person working meeting, 23 US-based researchers used a modified nominal group process to generate priorities and actions for addressing challenges to studying implementation mechanisms. During each of the three 120-min sessions, small groups responded to the prompt: "What actions need to be taken to move this research forward?" The groups brainstormed actions, which were then shared with the full group and discussed with the support of facilitators trained in structured group processes. Facilitators grouped critical and novel ideas into themes. Attendees voted on six themes they prioritized to discuss in a fourth, 120-min session, during which small groups operationalized prioritized actions. Subsequently, all ideas were collated, combined, and revised for clarity by a subset of the authorship team.

RESULTS

From this multistep process, 150 actions emerged across 10 priority areas, which together constitute the research agenda. Actions included discrete activities, projects, or products, and ways to shift how research is conducted to strengthen the study of implementation mechanisms.

CONCLUSIONS

This research agenda elevates actions to guide the selection, design, and evaluation of implementation mechanisms. By delineating recommended actions to address the challenges of studying implementation mechanisms, this research agenda facilitates expanding the field of implementation science, beyond studying what works to how and why strategies work, in what contexts, for whom, and with which interventions.

Reducing asthma attacks in disadvantaged school children with asthma: study protocol for a type 2 hybrid implementation-effectiveness trial (Better Asthma Control for Kids, BACK)

BACKGROUND

Asthma is a leading cause of children's hospitalizations, emergency department visits, and missed school days. Our school-based asthma intervention has reduced asthma exacerbations for children experiencing health disparities in the Denver Metropolitan Area, due partly to addressing care coordination for asthma and social determinants of health (SDOH), such as access to healthcare and medications. Limited dissemination of school-based asthma programs has occurred in other metropolitan and rural areas of Colorado. We formed and engaged community advisory boards in socioeconomically diverse regions of Colorado to develop two implementation strategy packages for delivering our school-based asthma intervention - now termed "Better Asthma Control for Kids (BACK)" - with tailoring to regional priorities, needs and resources.

METHODS

In this proposed type 2 hybrid implementation-effectiveness trial, where the primary goal is equitable reach to families to reduce asthma disparities, we will compare two different packages of implementation strategies to deliver BACK across four Colorado regions. The two implementation packages to be compared are: 1) standard set of implementation strategies including Tailor and Adapt to context, Facilitation and Training termed, BACK-Standard (BACK-S); 2) BACK-S plus an enhanced implementation strategy, that incorporates network weaving with community partners and consumer engagement with school families, termed BACK-Enhanced (BACK-E). Our evaluation will be guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework, including its Pragmatic Robust Implementation Sustainability Model (PRISM) determinants of implementation outcomes. Our central hypothesis is that our BACK-E implementation strategy will have significantly greater reach to eligible children/families than BACK-S (primary outcome) and that both BACK-E and BACK-S groups will have significantly reduced asthma exacerbation rates ("attacks") and improved asthma control as compared to usual care.

DISCUSSION

We expect both the BACK-S and BACK-E strategy packages will accelerate dissemination of our BACK program across the state - the comparative impact of BACK-S vs. BACK-E on reach and other RE-AIM outcomes may inform strategy selection for scaling BACK and other effective school-based programs to address chronic illness disparities.

TRIAL REGISTRATION

Clinicaltrials.gov identifier: NCT06003569, registered on August 22, 2023, https://classic.

CLINICALTRIALS

gov/ct2/show/NCT06003569 .

Barriers to and facilitators of implementing colorectal cancer screening evidence-based interventions in federally qualified health centers: a qualitative study

BACKGROUND

There is an urgent need to increase colorectal cancer screening (CRCS) uptake in Texas federally qualified health centers (FQHCs), which serve a predominantly vulnerable population with high demands. Empirical support exists for evidence-based interventions (EBIs) that are proven to increase CRCS; however, as with screening, their use remains low in FQHCs. This study aimed to identify barriers to and facilitators of implementing colorectal cancer screening (CRCS) evidence-based interventions (EBIs) in federally qualified health centers (FQHCs), guided by the Consolidated Framework for Implementation Research (CFIR).

METHODS

We recruited employees involved in implementing CRCS EBIs (e.g., physicians) using data from a CDC-funded program to increase the CRCS in Texas FQHCs. Through 23 group interviews, we explored experiences with practice change, CRCS promotion and quality improvement initiatives, organizational readiness, the impact of COVID-19, and the use of CRCS EBIs (e.g., provider reminders). We used directed content analysis with CFIR constructs to identify the critical facilitators and barriers.

RESULTS

The analysis revealed six primary CFIR constructs that influence implementation: information technology infrastructure, innovation design, work infrastructure, performance measurement pressure, assessing needs, and available resources. Based on experiences with four recommended EBIs, participants described barriers, including data limitations of electronic health records and the design of reminder alerts targeted at deliverers and recipients of patient or provider reminders. Implementation facilitators include incentivized processes to increase provider assessment and feedback, existing clinic processes (e.g., screening referrals), and available resources to address patient needs (e.g., transportation). Staff buy-in emerged as an implementation facilitator, fostering a conducive environment for change within clinics.

CONCLUSIONS

Using CFIR, we identified barriers, such as the burden of technology infrastructure, and facilitators, such as staff buy-in. The results, which enhance our understanding of CRCS EBI implementation in FQHCs, provide insights into designing nuanced, practical implementation strategies to improve cancer control in a critical setting.

Intervention Optimization: A Paradigm Shift and Its Potential Implications for Clinical Psychology

To build a coherent knowledge base about what psychological intervention strategies work, develop interventions that have positive societal impact, and maintain and increase this impact over time, it is necessary to replace the classical treatment package research paradigm. The multiphase optimization strategy (MOST) is an alternative paradigm that integrates ideas from behavioral science, engineering, implementation science, economics, and decision science. MOST enables optimization of interventions to strategically balance effectiveness, affordability, scalability, and efficiency. In this review we provide an overview of MOST, discuss several experimental designs that can be used in intervention optimization, consider how the investigator can use experimental results to select components for inclusion in the optimized intervention, discuss the application of MOST in implementation science, and list future issues in this rapidly evolving field. We highlight the feasibility of adopting this new research paradigm as well as its potential to hasten the progress of psychological intervention science.

Adaptive Designs in Implementation Science and Practice: Their Promise and the Need for Greater Understanding and Improved Communication

The promise of adaptation and adaptive designs in implementation science has been hindered by the lack of clarity and precision in defining what it means to adapt, especially regarding the distinction between adaptive study designs and adaptive implementation strategies. To ensure a common language for science and practice, authors reviewed the implementation science literature and found that the term adaptive was used to describe interventions, implementation strategies, and trial designs. To provide clarity and offer recommendations for reporting and strengthening study design, we propose a taxonomy that describes fixed versus adaptive implementation strategies and implementation trial designs. To improve impact, (a) futureimplementation studies should prespecify implementation strategy core functions that in turn can be taught to and replicated by health system/community partners, (b) funders should support exploratory studies that refine and specify implementation strategies, and (c) investigators should systematically address design requirements and ethical considerations (e.g., randomization, blinding/masking) with health system/community partners.

Getting to Fidelity: Consensus Development Process to Identify Core Activities of Implementation Facilitation

Transferring successful implementation strategies from research to practice requires approaches for assessing fidelity to the strategy's core components. Implementation facilitation (IF) is a strategy involving an interactive process of problem-solving, enabling, and supporting individuals in efforts to implement clinical innovations that occurs in the context of a recognized need for improvement and supportive interpersonal relationships. Because IF is a dynamic strategy involving numerous activities, our objective was to conduct a rigorous consensus development process to identify core activities for monitoring fidelity to IF when applied in clinical settings. We first conducted a scoping literature review to identify the range of activities used when IF has been applied in clinical settings, searching multiple citation databases for English-language articles including "facilitation" or other commonly-used terms for the strategy published from 1996-2015. Through multi-stage screening, 135 articles (from 94 studies) were identified for data extraction on IF activities, frequency with which IF activities were identified as 'core' by study authors, and study outcomes. From the literature review, we identified 32 distinct IF activities and developed definitions/examples for each. Next, we conducted a 3-stage, modified-Delphi expert panel consensus development process to identify core IF activities across three implementation phases (i.e., Pre-Implementation, Implementation, Sustainment). The expert panel identified 8 core activities for the Pre-Implementation Phase, 8 core activities for the Implementation Phase, and 4 core activities for the Sustainment Phase. This work provides an important foundation for developing measures/tools to assess use of core IF activities to ensure the strategy is delivered with fidelity.

Matters of Fidelity: School Provider Adherence and Competence in a Clustered Study of Adaptive Implementation Strategies

Schools are a promising access point for youth with mental health concerns, but school-based mental health professionals (SPs) often need ongoing support to provide high-fidelity cognitive behavioral therapy (CBT). Adherence and competence, two critical elements of fidelity, were examined in a cluster-randomized implementation trial. We evaluated CBT adherence and then triangulated CBT adherence with end-of-study competence. We then evaluated the effects of two implementation supports, Coaching and (for slower-responding schools) Facilitation, on adherence and competence. By the end of the 43-week study period, 27.8% of SPs met adherence criteria. Adherent SPs scored higher on the competence measure, the CBT Competence Scale (t (116.2) = 3.71, p < .001). No significant difference in adherence was found among SPs at schools assigned to Coaching vs. not (Δ = 6.0%, p = .385), however SPs at schools randomized to Coaching scored significantly higher on two of the four competence subscales (Non-Behavioral and Behavioral skills). Among slower-responder schools, SPs at schools assigned to Facilitation were more likely to demonstrate adherence (Δ = 16.3%, p = .022), but there was no effect of Facilitation on competence. Approximately one quarter of SPs met adherence criteria in the trial; adequate delivery of exposure was a primary obstacle to reaching adherence. Facilitation may be especially suited to help SPs overcome barriers to delivery, whereas Coaching may be especially suited to help SPs improve CBT competence. Both are likely needed to build a mental health work force with the competence and ability to deliver EBPs in schools.

Factors influencing support for the implementation of community-based out-of-hospital cardiac arrest interventions in high- and low-performing counties

Aim of the study

Survival to hospital discharge from out-of-hospital cardiac arrest (OHCA) after receiving treatment from emergency medical services (EMS) is less than 10% in the United States. Community-focused interventions improve survival rates, but there is limited information on how to gain support for new interventions or program activities within these populations. Using data from the RAndomized Cluster Evaluation of Cardiac ARrest Systems (RACE-CARS) trial, we aimed to identify the factors influencing emergency response agencies' support in implementing an OHCA intervention.

Methods

North Carolina counties were stratified into high-performing or low-performing counties based on the county's cardiac arrest volume, percent of bystander-cardiopulmonary resuscitation (CPR) performed, patient survival to hospital discharge, cerebral performance in patients after cardiac arrest, and perceived engagement in the RACE-CARS project. We randomly selected 4 high-performing and 3 low-performing counties and conducted semi-structured qualitative interviews with emergency response stakeholders in each county.

Results

From 10/2021 to 02/2022, we completed 29 interviews across the 7 counties (EMS (n = 9), telecommunications (n = 7), fire/first responders (n = 7), and hospital representatives (n = 6)). We identified three themes salient to community support for OHCA intervention: (1) initiating support at emergency response agencies; (2) obtaining support from emergency response agency staff (senior leadership and emergency response teams); and (3) and maintaining support. For each theme, we described similarities and differences by high- and low-performing county.

Conclusions

We identified techniques for supporting effective engagement of emergency response agencies in community-based interventions for OHCA improving survival rates. This work may inform future programs and initiatives around implementation of community-based interventions for OHCA.

Organize and mobilize for implementation effectiveness to improve overdose education and naloxone distribution from syringe services programs: a randomized controlled trial

BACKGROUND

The United States (US) continues to face decades-long increases in opioid overdose fatalities. As an opioid overdose reversal medication, naloxone can dramatically reduce opioid overdose mortality rates when distributed to people likely to experience or witness an opioid overdose and packaged with education on its use, known as overdose education and naloxone distribution (OEND). Syringe services programs (SSPs) are ideal venues for OEND with staff who are culturally competent in providing services for people who are at risk of experiencing or observing an opioid overdose. We carried out a randomized controlled trial of SSPs to understand the effectiveness of the organize and mobilize for implementation effectiveness (OMIE) approach at improving OEND implementation effectiveness within SSPs.

METHODS

Using simple randomization, 105 SSPs were enrolled into the trial and assigned to one of two study arms - (1) dissemination of OEND best practice recommendations (Control SSPs) or the OMIE approach along with dissemination of the OEND best practice recommendations (i.e., OMIE SSPs). OMIE SSPs could participate in 60-min OMIE sessions once a month for up to 12 months. At 12-month post-baseline, 102 of 105 SSPs (97%) responded to the follow-up survey.

RESULTS

The median number of sessions completed by OMIE SSPs was 10. Comparing OMIE SSPs to control SSPs, we observed significant increases in the number of participants receiving naloxone (incidence rate ratio: 2.15; 95% CI: 1.42, 3.25; p < 0.01) and the rate of naloxone doses distributed per SSP participant (adjusted incidence rate ratio: 1.97; 95% CI: 1.18, 3.30; p = 0.01). We observed no statistically significant difference in the number of adopted best practices between conditions (difference in means 0.2, 95% CI: - 0.7, 1.0; p = 0.68). We also observed a threshold effect where SSPs receiving a higher OMIE dose had greater effect sizes with regard to the number of people given naloxone and the number of naloxone doses distributed.

CONCLUSIONS

In conclusion, the multifaceted OMIE approach was effective at increasing naloxone distribution from SSPs, despite substantial external shocks during the trial. These findings have major implications for addressing the overdose crisis, which has continued unabated for decades.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03924505 . Registered 19 April 2019.

Cluster-randomized implementation trial of two facilitation strategies to implement a novel information and communications technology at the Veterans Health Administration

BACKGROUND

Information and communication technologies (ICTs) improve quality and efficiency of healthcare, but effective practices for implementing new ICTs are unknown. From 2019 to 2021, the Veterans Health Administration (VHA) implemented FLOW3, an ICT that facilitates prosthetic limb care. The goal of this study was to compare the impact of two facilitation strategies on FLOW3 adoption, implementation, and sustainment.

METHODS

FLOW3 is a computerized workflow management system comprised of three applications that facilitate the three steps for prosthesis authorization. During VHA's implementation of FLOW3, we randomized 60 VHA sites to basic or enhanced facilitation groups. Basic facilitation included a manualized training toolkit and office hours. Enhanced facilitation included basic facilitation plus monthly learning collaboratives and site-specific performance reports. Outcomes included time to adoption of FLOW3 and complete FLOW3 utilization rates during implementation and sustainment periods. We compared outcomes between sites assigned to basic versus enhanced facilitation groups. Results were calculated using both intent-to-treat (ITT) and dose-response analyses. The dose-response analysis used a per-protocol approach and required sites in the enhanced facilitation group to join two of six learning collaboratives; sites that attended fewer were reassigned to the basic group.

RESULTS

Randomization assigned 30 sites to enhanced facilitation and 30 to basic. Eighteen of 30 randomized sites were included in the enhanced facilitation group for dose-response analysis. During the implementation period, enhanced facilitation sites were significantly more likely to completely utilize FLOW3 than basic facilitation sites (HR: 0.17; 95% CI: 1.18, 4.53, p = 0.02) based on ITT analysis. In the dose-response analysis, the enhanced group was 2.32 (95% CI: 1.18, 4.53) times more likely to adopt FLOW3 than basic group (p = 0.014).

CONCLUSIONS

Enhanced facilitation including a learning collaborative and customized feedback demonstrated greater likelihood for sites to complete a prosthetics consult using FLOW3 throughout our study. We identified statistically significant differences in likelihood of adoption using the dose-response analysis and complete utilization rate using ITT analysis during the implementation period. All sites that implemented FLOW3 demonstrated improvement in completion rate during the sustainment period, but the difference between facilitation groups was not statistically significant. Further study to understand sustainability is warranted.

A mixed methods partner-focused cost and budget impact analysis to deploy implementation strategies for school-based prevention

BACKGROUND

Obtaining information on implementation strategy costs and local budget impacts from multiple perspectives is essential to data-driven decision-making about resource allocation for successful evidence-based intervention delivery. This mixed methods study determines the costs and priorities of deploying Enhanced Replicating Effective Programs (REP) to implement the Michigan Model for Health™, a universal school-based prevention intervention, from key shareholder perspectives.

METHODS

Our study included teachers in 8 high schools across 3 Michigan counties as part of a pilot cluster randomized trial. We used activity-based costing, mapping key Enhanced REP activities across implementation phases. We included multiple perspectives, including state agencies, regional education service agencies, lead organization, and implementers. We also conducted a budget impact analysis (BIA, assessing the potential financial impact of adopting Enhanced REP) and a scenario analysis to estimate replication and account for cost variability. We used an experimental embedded mixed methods approach, conducting semi-structured interviews and collecting field notes during the trial to expand and explain the cost data and the implications of costs across relevant perspectives.

RESULTS

Based on trial results, we estimate costs for deploying Enhanced REP are $11,903/school, with an estimated range between $8263/school and $15,201/school. We estimate that adding four additional schools, consistent with the pilot, would cost $8659/school. Qualitative results indicated misalignment in school and teacher priorities in some cases. Implementation activities, including training and implementation facilitation with the health coordinator, were sometimes in addition to regular teaching responsibilities. The extent to which this occurred was partly due to leadership priorities (e.g., sticking to the district PD schedule) and organizational priorities (e.g., budget).

CONCLUSIONS

Previous research findings indicate that, from a societal perspective, universal prevention is an excellent return on investment. However, notable misalignment in cost burden and priorities exists across shareholder groups. Our results indicate significant personal time costs by teachers when engaging in implementation strategy activities that impose an opportunity cost. Additional strategies are needed to improve the alignment of costs and benefits to enhance the success and sustainability of implementation. We focus on those perspectives informed by the analysis and discuss opportunities to expand a multi-level focus and create greater alignment across perspectives.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04752189. Registered on 12 February 2021.

Developing and testing a COVID-19 vaccination acceptance intervention: A pragmatic trial comparing vaccine acceptance intervention vs usual care - Rationale, methods, and implementation

BACKGROUND

COVID-19 has resulted in significant disability and loss of life. COVID-19 vaccines effectively prevent severe illness, hospitalization, and death. Nevertheless, many people remain hesitant to accept vaccination. Veterans perceive healthcare providers (HCP) and staff as trusted vaccine information sources and thereby are well suited to initiate vaccine discussions. The overall objective of this study is to implement and test a virtual COVID-19 Vaccine Acceptance Intervention (VAI) training that is informed by motivational interviewing (MI) techniques.

METHODS

The VAI training is being delivered to VA HCPs and staff within a Hybrid Type 2 pragmatic implementation-effectiveness trial using Implementation Facilitation as the implementation strategy. The implementation team includes external facilitators paired with VA Healthcare System (VAHCS)-level internal facilitators. The trial has three aims: 1) Examine the effectiveness of the VAI versus usual care on unvaccinated veterans' vaccination rates in a one-year cluster randomized controlled trial, with randomization at the level of VAHCS. 2) Determine factors associated with veterans' decisions to accept or decline primary COVID-19 vaccination, and better understand how these factors influence vaccination decisions, through survey and qualitative data; and 3) Use qualitative interviews with HCPs and staff from clinics with high and low vaccination rates to learn what was helpful and not helpful about the VAI and implementation strategies.

CONCLUSION

This is the first multisite randomized controlled trial to test an MI-informed vaccine acceptance intervention to improve COVID-19 vaccine acceptance. Information gained can be used to inform healthcare systems' approaches to improve future vaccination and other public health campaigns.

CLINICALTRIALS

gov Identifier: NCT05027464.

The engagement problem: A review of engagement with digital mental health interventions and recommendations for a path forward

Purpose of the review

Digital mental health interventions (DMHIs) are an effective and accessible means of addressing the unprecedented levels of mental illness worldwide. Currently, however, patient engagement with DMHIs in real-world settings is often insufficient to see clinical benefit. In order to realize the potential of DMHIs, there is a need to better understand what drives patient engagement.

Recent findings

We discuss takeaways from the existing literature related to patient engagement with DMHIs and highlight gaps to be addressed through further research. Findings suggest that engagement is influenced by patient-, intervention- and systems-level factors. At the patient-level, variables such as sex, education, personality traits, race, ethnicity, age and symptom severity appear to be associated with engagement. At the intervention-level, integrating human support, gamification, financial incentives and persuasive technology features may improve engagement. Finally, although systems-level factors have not been widely explored, the existing evidence suggests that achieving engagement will require addressing organizational and social barriers and drawing on the field of implementation science.

Summary

Future research clarifying the patient-, intervention- and systems-level factors that drive engagement will be essential. Additionally, to facilitate improved understanding of DMHI engagement, we propose the following: (a) widespread adoption of a minimum necessary 5-element engagement reporting framework; (b) broader application of alternative clinical trial designs; and (c) directed efforts to build upon an initial parsimonious conceptual model of DMHI engagement.

Team-focused implementation strategies to improve implementation of mental health screening and referral in rural Children's Advocacy Centers: study protocol for a pilot cluster randomized hybrid type 2 trial

BACKGROUND

Children's Advocacy Centers (CACs) use multidisciplinary teams to investigate and respond to maltreatment allegations. CACs play a critical role in connecting children with mental health needs to evidence-based mental health treatment, especially in low-resourced rural areas. Standardized mental health screening and referral protocols can improve CACs' capacity to identify children with mental health needs and encourage treatment engagement. In the team-based context of CACs, teamwork quality is likely to influence implementation processes and outcomes. Implementation strategies that target teams and apply the science of team effectiveness may enhance implementation outcomes in team-based settings.

METHODS

We will use Implementation Mapping to develop team-focused implementation strategies to support the implementation of the Care Process Model for Pediatric Traumatic Stress (CPM-PTS), a standardized screening and referral protocol. Team-focused strategies will integrate activities from effective team development interventions. We will pilot team-focused implementation in a cluster-randomized hybrid type 2 effectiveness-implementation trial. Four rural CACs will implement the CPM-PTS after being randomized to either team-focused implementation (n = 2 CACs) or standard implementation (n = 2 CACs). We will assess the feasibility of team-focused implementation and explore between-group differences in hypothesized team-level mechanisms of change and implementation outcomes (implementation aim). We will use a within-group pre-post design to test the effectiveness of the CPM-PTS in increasing caregivers' understanding of their child's mental health needs and caregivers' intentions to initiate mental health services (effectiveness aim).

CONCLUSIONS

Targeting multidisciplinary teams is an innovative approach to improving implementation outcomes. This study will be one of the first to test team-focused implementation strategies that integrate effective team development interventions. Results will inform efforts to implement evidence-based practices in team-based service settings.

TRIAL REGISTRATION

Clinicaltrials.gov, NCT05679154 . Registered on January 10, 2023.

How does facilitation in healthcare work? Using mechanism mapping to illuminate the black box of a meta-implementation strategy

BACKGROUND

Healthcare facilitation, an implementation strategy designed to improve the uptake of effective clinical innovations in routine practice, has produced promising yet mixed results in randomized implementation trials and has not been fully researched across different contexts.

OBJECTIVE

Using mechanism mapping, which applies directed acyclic graphs that decompose an effect of interest into hypothesized causal steps and mechanisms, we propose a more concrete description of how healthcare facilitation works to inform its further study as a meta-implementation strategy.

METHODS

Using a modified Delphi consensus process, co-authors developed the mechanistic map based on a three-step process. First, they developed an initial logic model by collectively reviewing the literature and identifying the most relevant studies of healthcare facilitation components and mechanisms to date. Second, they applied the logic model to write vignettes describing how facilitation worked (or did not) based on recent empirical trials that were selected via consensus for inclusion and diversity in contextual settings (US, international sites). Finally, the mechanistic map was created based on the collective findings from the vignettes.

FINDINGS

Theory-based healthcare facilitation components informing the mechanistic map included staff engagement, role clarification, coalition-building through peer experiences and identifying champions, capacity-building through problem solving barriers, and organizational ownership of the implementation process. Across the vignettes, engagement of leaders and practitioners led to increased socialization of the facilitator's role in the organization. This in turn led to clarifying of roles and responsibilities among practitioners and identifying peer experiences led to increased coherence and sense-making of the value of adopting effective innovations. Increased trust develops across leadership and practitioners through expanded capacity in adoption of the effective innovation by identifying opportunities that mitigated barriers to practice change. Finally, these mechanisms led to eventual normalization and ownership of the effective innovation and healthcare facilitation process.

IMPACT

Mapping methodology provides a novel perspective of mechanisms of healthcare facilitation, notably how sensemaking, trust, and normalization contribute to quality improvement. This method may also enable more efficient and impactful hypothesis-testing and application of complex implementation strategies, with high relevance for lower-resourced settings, to inform effective innovation uptake.

Adapting adaptive design methods to accelerate adoption of a digital asthma management intervention

Investigators conducting translational research in real-world settings may experience changes that create challenges to the successful completion of the trial as well as post-trial adoption and implementation. Adaptive designs support translational research by systematically adapting content and methods to meet the needs of target populations, settings and contexts. This manuscript describes an adaptive implementation research model that provides strategies for changing content, delivery processes, and research methods to correct course when anticipated and unanticipated circumstances occur during a pragmatic trial. The Breathewell Program included two large pragmatic trials of the effectiveness of a digital communication technology intervention to improve symptom management and medication adherence in asthma care. The first trial targeted parents of children with asthma; the second targeted adults with asthma. Adaptations were made iteratively to adjust to dynamic conditions within the healthcare setting, informed by prospectively collected stakeholder input, and were categorized retrospectively by the authors as proactive or reactive. Study outcomes demonstrated improved treatment adherence and clinical efficiency. Kaiser Permanente Colorado, the setting for both studies, adopted the speech recognition intervention into routine care, however, both interventions required numerous adaptations, including changes to target population, intervention content, and internal workflows. Proactive and reactive adaptations assured that both trials were successfully completed. Adaptive research designs will continue to provide an important pathway to move healthcare delivery research into practice while conducting ongoing effectiveness evaluation.

How Can Implementation Science Improve the Care of Familial Hypercholesterolaemia?

PURPOSE OF REVIEW

Describe the application of implementation science to improve the detection and management of familial hypercholesterolaemia.

RECENT FINDINGS

Gaps between evidence and practice, such as underutilization of genetic testing, family cascade testing, failure to achieve LDL-cholesterol goals and low levels of knowledge and awareness, have been identified through clinical registry analyses and clinician surveys. Implementation science theories, models and frameworks have been applied to assess barriers and enablers in the literature specific to local contextual factors (e.g. stages of life). The effect of implementation strategies to overcome these factors has been evaluated; for example, automated identification of individuals with FH or training and education to improve statin adherence. Clinical registries were identified as a key infrastructure to monitor, evaluate and sustain improvements in care. The expansion in evidence supporting the care of familial hypercholesterolaemia requires a similar expansion of efforts to translate new knowledge into clinical practice.

Intensification of Implementation Strategies: Developing a Model of Foundational and Enhanced Implementation Approaches to Support National Adoption and Scale-up

Implementation strategies are activities to support integration of evidence-based programs (EBPs) into routine care. Comprised of 170+ facilities, the Veterans Affairs Healthcare System is conducive to evaluating feasibility and scalability of implementation strategies on a national level. In previous work evaluating implementation of three EBPs for older Veterans (hospital-based walking, caregiver skills training, group physical therapy), we found facilities varied in their need for implementation support, with some needing minimal guidance and others requiring intensive support. Committed to national scalability, our team developed an implementation intensification model consisting of foundational (low-touch) and enhanced (high-touch) implementation support. This Forum article describes our multilevel and multistep process to develop and evaluate implementation intensification. Steps included (a) review completed trial data; (b) conduct listening sessions; (c) review literature; (d) draft foundational and enhanced implementation support packages; (e) iteratively refine packages; and (7) devise an evaluation plan. Our model of implementation intensification may be relevant to other health care systems seeking strategies that can adapt to diverse delivery settings, optimize resources, help build capacity, and ultimately enhance implementation outcomes. As more health care systems focus on spread of EBPs into routine care, identifying scalable and effective implementation strategies will be critical.

Interim monitoring in sequential multiple assignment randomized trials

A sequential multiple assignment randomized trial (SMART) facilitates the comparison of multiple adaptive treatment strategies (ATSs) simultaneously. Previous studies have established a framework to test the homogeneity of multiple ATSs by a global Wald test through inverse probability weighting. SMARTs are generally lengthier than classical clinical trials due to the sequential nature of treatment randomization in multiple stages. Thus, it would be beneficial to add interim analyses allowing for an early stop if overwhelming efficacy is observed. We introduce group sequential methods to SMARTs to facilitate interim monitoring based on the multivariate chi-square distribution. Simulation studies demonstrate that the proposed interim monitoring in SMART (IM-SMART) maintains the desired type I error and power with reduced expected sample size compared to the classical SMART. Finally, we illustrate our method by reanalyzing a SMART assessing the effects of cognitive behavioral and physical therapies in patients with knee osteoarthritis and comorbid subsyndromal depressive symptoms.

Barriers and Enablers to Implementing Peer Specialists in Veterans Health Administration Primary Care: a Qualitative Study

BACKGROUND

Peer specialists (PSs) are increasingly deployed in a variety of settings to provide patient-centered care. In the Veterans Health Administration (VHA), efforts are underway to integrate PSs into primary care settings. Little is known about the barriers and enablers to implementing PS services in primary care.

OBJECTIVE

To characterize barriers and enablers to implementing PSs in primary care.

DESIGN

Qualitative study using semi-structured interviews.

PARTICIPANTS

PSs and their supervisors from 25 VHA primary care settings.

APPROACH

PSs and supervisors were interviewed about their experiences integrating PSs in primary care. Rapid analysis was conducted to identify barriers and enablers to PS integration, as well as to examine the role of external facilitation in implementation experiences.

KEY RESULTS

Fifty-two interviews were completed (25 PSs from 19 sites and 27 supervisors from 24 sites). Barriers and enablers to PS integration in VHA primary care settings included PS role clarity and constraints, provider buy-in, supervision, leadership support, and primary care culture. The barriers and enablers were consistent across both external facilitation and control sites.

CONCLUSIONS

Results describe how the characteristics of the innovation, the recipients, and the context impact successful implementation of PSs in primary care settings. The identification of barriers and enablers holds promise for improving future efforts to embed PSs in primary care.

TRIAL REGISTRATION

This project is registered at ClinicalTrials.gov with number NCT02732600 (URL: https://clinicaltrials.gov/ct2/show/NCT02732600 ).

Applying concepts from "rapid" and "agile" implementation to advance implementation research

Background

The translation of research findings into practice can be improved to maximize benefits more quickly and with greater flexibility. To expedite translation, researchers have developed innovative approaches to implementation branded as “rapid” and “agile” implementation. Rapid implementation has roots in precision medicine and agile implementation has roots in systems engineering and software design. Research has shown that innovation often derives from learning and applying ideas that have impacted other fields.

Implications for implementation researchers

This commentary examines “rapid” and “agile” approaches to implementation and provides recommendations to implementation researchers stemming from these approaches. Four key ideas are synthesized that may be broadly applicable to implementation research, including (1) adopting a problem orientation, (2) applying lessons from behavioral economics, (3) using adaptive study designs and adaptive interventions, and (4) using multi-level models to guide implementation. Examples are highlighted from the field where researchers are applying these key ideas to illustrate their potential impact.

Conclusions

“Rapid” and “agile” implementation approaches to implementation stem from diverse fields. Elements of these approaches show potential for advancing implementation research, although adopting them may entail shifting scientific norms in the field.

Baseline data of a sequential multiple assignment randomized trial (STEP study)

AIM

Research has shown that preventative intervention in individuals at ultra-high risk of psychosis (UHR) improves symptomatic and functional outcomes. The staged treatment in early psychosis (STEP) trial aims to determine the most effective type, timing and sequence of interventions in the UHR population by sequentially studying the effectiveness of (1) support and problem solving, (2) cognitive-behavioural case management and (3) antidepressant medication with an embedded fast-fail option of (4) omega-3 fatty acids or low-dose antipsychotic medication. This paper presents the recruitment flow and baseline clinical characteristics of the sample.

METHODS

STEP is a sequential multiple assignment randomized trial. We present the baseline demographics, clinical characteristics and acceptability and feasibility of this treatment approach as indicated by the flow of participants from first contact up until enrolment into the trial. Recruitment took place between April 2016 and January 2019.

RESULTS

Of 1343, help-seeking young people who were considered for participation, 402 participants were not eligible and 599 declined/disengaged, resulting in a total of 342 participants enrolled in the study. The most common reason for exclusion was an active prescription of antidepressant medication. Eighty-five percent of the enrolled sample had a non-psychotic DSM-5 diagnosis and symptomatic/functional measures showed a moderate level of clinical severity and functional impairment.

DISCUSSION

The present study demonstrates the acceptability and participant's general positive appraisal of sequential treatment. It also shows, in line with other trials in UHR individuals, a significant level of psychiatric morbidity and impairment, demonstrating the clear need for care in this group and that treatment is appropriate.

Stepped implementation-to-target: a study protocol of an adaptive trial to expand access to addiction medications

BACKGROUND

In response to the US opioid epidemic, significant national campaigns have been launched to expand access to `opioid use disorder (MOUD). While adoption has increased in general medical care settings, specialty addiction programs have lagged in both reach and adoption. Elevating the quality of implementation strategy, research requires more precise methods in tailoring strategies rather than a one-size-fits-all-approach, documenting participant engagement and fidelity to the delivery of the strategy, and conducting an economic analysis to inform decision making and policy. Research has yet to incorporate all three of these recommendations to address the challenges of implementing and sustaining MOUD in specialty addiction programs.

METHODS

This project seeks to recruit 72 specialty addiction programs in partnership with the Washington State Health Care Authority and employs a measurement-based stepped implementation-to-target approach within an adaptive trial design. Programs will be exposed to a sequence of implementation strategies of increasing intensity and cost: (1) enhanced monitoring and feedback (EMF), (2) 2-day workshop, and then, if outcome targets are not achieved, randomization to either internal facilitation or external facilitation. The study has three aims: (1) evaluate the sequential impact of implementation strategies on target outcomes, (2) examine contextual moderators and mediators of outcomes in response to the strategies, and (3) document and model costs per implementation strategy. Target outcomes are organized by the RE-AIM framework and the Addiction Care Cascade.

DISCUSSION

This implementation project includes elements of a sequential multiple assignment randomized trial (SMART) design and a criterion-based design. An innovative and efficient approach, participating programs only receive the implementation strategies they need to achieve target outcomes. Findings have the potential to inform implementation research and provide key decision-makers with evidence on how to address the opioid epidemic at a systems level.

TRIAL REGISTRATION

This trial was registered at ClinicalTrials.gov (NCT05343793) on April 25, 2022.

The economics of adaptations to evidence-based practices

Background

Evidence-based practices (EBPs) are frequently adapted in response to the dynamic contexts in which they are implemented. Adaptation is defined as the degree to which an EBP is altered to fit the setting or to improve fit to local context and can be planned or unplanned. Although adaptations are common and necessary to maximizing the marginal impact of EBPs, little attention has been given to the economic consequences and how adaptations affect marginal costs.

Discussion

In assessing the economic consequences of adaptation, one should consider its impact on core components, the planned adaptive periphery, and the unplanned adaptive periphery. Guided by implementation science frameworks, we examine how various economic evaluation approaches accommodate the influence of adaptations and discuss the pros and cons of these approaches. Using the Framework for Reporting Adaptations and Modifications to Evidence-based interventions (FRAME), mixed methods can elucidate the economic reasons driving the adaptations. Micro-costing approaches are applied in research that integrates the adaptation of EBPs at the planning stage using innovative, adaptive study designs. In contrast, evaluation of unplanned adaptation is subject to confounding and requires sensitivity analysis to address unobservable measures and other uncertainties. A case study is presented using the RE-AIM framework to illustrate the costing of adaptations. In addition to empirical approaches to evaluating adaptation, simulation modeling approaches can be used to overcome limited follow-up in implementation studies.

Conclusions

As implementation science evolves to improve our understanding of the mechanisms and implications of adaptations, it is increasingly important to understand the economic implications of such adaptations, in addition to their impact on clinical effectiveness. Therefore, explicit consideration is warranted of how costs can be evaluated as outcomes of adaptations to the delivery of EBPs.

Michigan Model for HealthTM Learning to Enhance and Adapt for Prevention (Mi-LEAP): protocol of a pilot randomized trial comparing Enhanced Replicating Effective Programs versus standard implementation to deliver an evidence-based drug use prevention curriculum

BACKGROUND

School-based drug use prevention programs have demonstrated notable potential to reduce the onset and escalation of drug use, including among youth at risk of poor outcomes such as those exposed to trauma. Researchers have found a robust relationship between intervention fidelity and participant (i.e., student) outcomes. Effective implementation of evidence-based interventions, such as the Michigan Model for HealthTM (MMH), is critical to achieving desired public health objectives. Yet, a persistent gap remains in what we know works and how to effectively translate these findings into routine practice. The objective of this study is to design and test a multi-component implementation strategy to tailor MMH to meet population needs (i.e., students exposed to trauma), and improve the population-context fit to enhance fidelity and effectiveness.

METHODS

Using a 2-group, mixed-method randomized controlled trial design, this study will compare standard implementation versus Enhanced Replicating Effective Programs (REP) to deliver MMH. REP is a theoretically based implementation strategy that promotes evidence-based intervention (EBI) fidelity through a combination of EBI curriculum packaging, training, and as-needed technical assistance and is consistent with standard MMH implementation. Enhanced REP will tailor the intervention and training to integrate trauma-informed approaches and deploy customized implementation support (i.e., facilitation). The research will address the following specific aims: (1) design and test an implementation strategy (Enhanced REP) to deliver the MMH versus standard implementation and evaluate feasibility, acceptability, and appropriateness using mixed methods, (2) estimate the costs and cost-effectiveness of Enhanced REP to deliver MMH versus standard implementation.

DISCUSSION

This research will design and test a multi-component implementation strategy focused on enhancing the fit between the intervention and population needs while maintaining fidelity to MMH core functions. We focus on the feasibility of deploying the implementation strategy bundle and costing methods and preliminary information on cost input distributions. The substantive focus on youth at heightened risk of drug use and its consequences due to trauma exposure is significant because of the public health impact of prevention. Pilot studies of implementation strategies are underutilized and can provide vital information on designing and testing effective strategies by addressing potential design and methods uncertainties and the effects of the implementation strategy on implementation and student outcomes.

TRIAL REGISTRATION

NCT04752189-registered on 8 February 2021 on ClinicalTrials.gov PRS.

Advancing rapid adaptation for urgent public health crises: Using implementation science to facilitate effective and efficient responses

Responding rapidly to emerging public health crises is vital to reducing their escalation, spread, and impact on population health. These responses, however, are challenging and disparate processes for researchers and practitioners. Researchers often develop new interventions that take significant time and resources, with little exportability. In contrast, community-serving systems are often poorly equipped to properly adopt new interventions or adapt existing ones in a data-driven way during crises' onset and escalation. This results in significant delays in deploying evidence-based interventions (EBIs) with notable public health consequences. This prolonged timeline for EBI development and implementation results in significant morbidity and mortality that is costly and preventable. As public health emergencies have demonstrated (e.g., COVID-19 pandemic), the negative consequences often exacerbate existing health disparities. Implementation science has the potential to bridge the extant gap between research and practice, and enhance equity in rapid public health responses, but is underutilized. For the field to have a greater "real-world" impact, it needs to be more rapid, iterative, participatory, and work within the timeframes of community-serving systems. This paper focuses on rapid adaptation as a developing implementation science area to facilitate system responses during public health crises. We highlight frameworks to guide rapid adaptation for optimizing existing EBIs when responding to urgent public health issues. We also explore the economic implications of rapid adaptation. Resource limitations are frequently a central reason for implementation failure; thus, we consider the economic impacts of rapid adaptation. Finally, we provide examples and propose directions for future research and application.

Implementing a text-messaging intervention for unhealthy alcohol use in emergency departments: protocol for implementation strategy development and a pilot cluster randomized implementation trial

BACKGROUND

Unhealthy alcohol use (UAU) is a leading cause of premature mortality among adults in the USA. Emergency departments (EDs) are key intervention settings for UAU but often have limited time and resources. One low-burden, scalable approach to address UAU is text-messaging interventions. Despite strong research support and promise for scalability, there is little research on how to implement such interventions in healthcare settings. The process of providers making them available to patients in an efficient way within already busy and overburdened ED workflows and patients adopting them remains a new area of research. The purpose of this three-phase study is to develop and test an implementation strategy for UAU text-messaging interventions in EDs.

METHOD

Our first aim is to examine barriers and facilitators to staff offering and patients accepting a text-messaging intervention in the ED using an explanatory, sequential mixed methods approach. We will examine alcohol screening data in the electronic health records of 17 EDs within a large integrated health system in the Northeast and conduct surveys among chairpersons in each. This data will be used to purposively sample 4 EDs for semi-structured interviews among 20 clinical staff, 20 patients, and 4 chairpersons. Our second aim is to conduct a stakeholder-engaged intervention mapping process to develop a multi-component implementation strategy for EDs. Our third aim is to conduct a mixed method 2-arm cluster randomized pilot study in 4 EDs that serve ~11,000 UAU patients per year to assess the feasibility, acceptability, and preliminary effectiveness of the implementation strategy. The Integrated Promoting Action on Research Implementation in Health Services framework will guide study activities.

DISCUSSION

Low-burden technology, like text messaging, along with targeted implementation support and strategies driven by identified barriers and facilitators could sustain large-scale ED-based alcohol screening programs and provide much needed support to patients who screen positive while reducing burden on EDs. The proposed study would be the first to develop and test this targeted implementation strategy and will prepare for a larger, fully powered hybrid effectiveness-implementation trial. Findings may also be broadly applicable to implementation of patient-facing mobile health technologies.

TRIAL REGISTRATION

This study was registered at ClinicalTrials.gov (NCT05350878) on 4/28/2022.

Longitudinal Effects of a Motivationally Focused Strategy to Increase the Yield of Training and Consultation on Teachers' Adoption and Fidelity of a Universal Program

Group-based didactic training is a cornerstone implementation strategy used to support the adoption and delivery of evidence-based prevention programs (EBPP) by teachers in schools, but it is often insufficient to drive successful implementation. Beliefs and Attitudes for Successful Implementation in Schools for Teachers (BASIS-T) is a theory-based, motivational implementation strategy designed to increase the yield of EBPP training and consultation. The purpose of this study was to examine the longitudinal effects of BASIS-T on hypothesized mechanisms of behavior change (e.g., attitudes toward EBPP, self-efficacy, intentions to implement) and implementation and student outcomes associated with a well-established universal prevention program-the good behavior game (GBG). This pilot trial included 82 elementary school teachers from nine public elementary schools who were randomly assigned at the school-level to the BASIS-T (n = 43) or active comparison (n = 39) condition, with both conditions receiving training and consultation of the good behavior game by a third-party purveyor. Analyses included mixed-effects and multilevel growth modeling of adoption, mechanisms of behavior change, and student behavior outcomes. Meaningful effects were found favoring BASIS-T on immediate adoption of the GBG within the first month of school (74% vs. 40%) and self-efficacy (p < 0.05). These findings advance our understanding of the type of implementation strategies that complement pre-implementation training and post-training consultation in schools by identifying the importance of task self-efficacy as a mechanism of behavior change related to adoption for prevention programming.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12310-022-09536-z.

Primary aim results of a clustered SMART for developing a school-level, adaptive implementation strategy to support CBT delivery at high schools in Michigan

BACKGROUND

Schools increasingly provide mental health services to students, but often lack access to implementation strategies to support school-based (and school professional [SP]) delivery of evidence-based practices. Given substantial heterogeneity in implementation barriers across schools, development of adaptive implementation strategies that guide which implementation strategies to provide to which schools and when may be necessary to support scale-up.

METHODS

A clustered, sequential, multiple-assignment randomized trial (SMART) of high schools across Michigan was used to inform the development of a school-level adaptive implementation strategy for supporting SP-delivered cognitive behavioral therapy (CBT). All schools were first provided with implementation support informed by Replicating Effective Programs (REP) and then were randomized to add in-person Coaching or not (phase 1). After 8 weeks, schools were assessed for response based on SP-reported frequency of CBT delivered to students and/or barriers reported. Responder schools continued with phase 1 implementation strategies. Slower-responder schools (not providing ≥ 3 CBT components to ≥10 students or >2 organizational barriers identified) were re-randomized to add Facilitation to current support or not (phase 2). The primary aim hypothesis was that SPs at schools receiving the REP + Coaching + Facilitation adaptive implementation strategy would deliver more CBT sessions than SPs at schools receiving REP alone. Secondary aims compared four implementation strategies (Coaching vs no Coaching × Facilitation vs no Facilitation) on CBT sessions delivered, including by type (group, brief and full individual). Analyses used a marginal, weighted least squares approach developed for clustered SMARTs.

RESULTS

SPs (n = 169) at 94 high schools entered the study. N = 83 schools (88%) were slower-responders after phase 1. Contrary to the primary aim hypothesis, there was no evidence of a significant difference in CBT sessions delivered between REP + Coaching + Facilitation and REP alone (111.4 vs. 121.1 average total CBT sessions; p = 0.63). In secondary analyses, the adaptive strategy that offered REP + Facilitation resulted in the highest average CBT delivery (154.1 sessions) and the non-adaptive strategy offering REP + Coaching the lowest (94.5 sessions).

CONCLUSIONS

The most effective strategy in terms of average SP-reported CBT delivery is the adaptive implementation strategy that (i) begins with REP, (ii) augments with Facilitation for slower-responder schools (schools where SPs identified organizational barriers or struggled to deliver CBT), and (iii) stays the course with REP for responder schools.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03541317 , May 30, 2018.

Linking organizational climate for evidence-based practice implementation to observed clinician behavior in patient encounters: a lagged analysis

Background

Theory and empirical research suggest organizational climate for evidence-based practice (EBP) implementation may be an important and malleable target to improve clinician use of EBPs in healthcare; however, this work has been criticized for overreliance on self-report measures of implementation outcomes and cross-sectional designs. This study combines data from two studies spanning 7 years to test the hypothesis that higher levels of organizational EBP implementation climate prospectively predicts improved clinician adherence to an EBP, cognitive behavioral therapy (CBT), as rated by expert observers.

Methods

Biennial assessments of EBP implementation climate collected in 10 community mental health agencies in Philadelphia as part of a systemwide evaluation (time 1) were linked to subsequent observer ratings of clinician adherence to CBT in clinical encounters with 108 youth (time 2). Experts rated clinician adherence to CBT using the Therapy Process Observation Coding System which generated two primary outcomes (a) maximum CBT adherence per session (i.e., highest rated CBT intervention per session; depth of delivery) and (b) average CBT adherence per session (i.e., mean rating across all CBT interventions used; depth and breadth of delivery).

Results

On average, time 2 clinician adherence observations occurred 19.8 months (SD = 10.15) after time 1 organizational climate assessments. Adjusting for organization, clinician, and client covariates, a one standard deviation increase in organizational EBP implementation climate at time 1 predicted a 0.63-point increase in clinicians’ maximum CBT adherence per session at time 2 (p = 0.000), representing a large effect size (d = 0.93; 95% CI = 0.63–1.24) when comparing organizations in the upper (k = 3) versus lower tertiles (k = 3) of EBP implementation climate. Higher levels of time 1 organizational EBP implementation climate also predicted higher time 2 average CBT adherence per session (b = 0.23, p < 0.001, d = 0.72). Length of time between assessments of climate and adherence did not moderate these relationships.

Conclusions

Organizational EBP implementation climate is a promising predictor of clinicians’ subsequent observed adherence to CBT. Implementation strategies that target this antecedent may improve the delivery of EBPs in healthcare settings.

Operationalizing Leadership and Clinician Buy-In to Implement Evidence-Based Tobacco Treatment Programs in Routine Oncology Care: A Mixed-Method Study of the U.S. Cancer Center Cessation Initiative

BACKGROUND

Delivering evidence-based tobacco dependence treatment in oncology settings improves smoking abstinence and cancer outcomes. Leadership engagement/buy-in is critical for implementation success, but few studies have defined buy-in or described how to secure buy-in for tobacco treatment programs (TTPs) in cancer care. This study examines buy-in during the establishment of tobacco treatment programs at National Cancer Institute (NCI)-designated cancer centers.

METHODS

We utilized a sequential, explanatory mixed-methods approach to analyze quantitative data and qualitative interviews with program leads in the U.S.-based NCI Moonshot-supported Cancer Center Cessation Initiative (n = 20 Centers). We calculated descriptive statistics and applied structural coding and content analysis to qualitative data.

RESULTS

At least 75% of participating centers secured health care system administrative, clinical, and IT leadership buy-in and support. Six themes emerged from interviews: engaging leadership, access to resources, leveraging federal funding support to build leadership interest, designating champions, identifying training needs, and ensuring staff roles and IT systems support workflows.

CONCLUSIONS

Buy-in among staff and clinicians is defined by the belief that the TTP is necessary, valuable, and evidence based. Recognizing and securing these dimensions of buy-in can facilitate implementation success, leading to improved cancer outcomes.

Development of the ASSESS tool: a comprehenSive tool to Support rEporting and critical appraiSal of qualitative, quantitative, and mixed methods implementation reSearch outcomes

Background

Several tools to improve reporting of implementation studies for evidence-based decision making have been created; however, no tool for critical appraisal of implementation outcomes exists. Researchers, practitioners, and policy makers lack tools to support the concurrent synthesis and critical assessment of outcomes for implementation research. Our objectives were to develop a comprehensive tool to (1) describe studies focused on implementation that use qualitative, quantitative, and/or mixed methodologies and (2) assess risk of bias of implementation outcomes.

Methods

A hybrid consensus-building approach combining Delphi Group and Nominal Group techniques (NGT) was modeled after comparative methodologies for developing health research reporting guidelines and critical appraisal tools. First, an online modified NGT occurred among a small expert panel (n = 5), consisting of literature review, item generation, round robin with clarification, application of the tool to various study types, voting, and discussion. This was followed by a larger e-consensus meeting and modified Delphi process with implementers and implementation scientists (n = 32). New elements and elements of various existing tools, frameworks, and taxonomies were combined to produce the ASSESS tool.

Results

The 24-item tool is applicable to a broad range of study designs employed in implementation science, including qualitative studies, randomized-control trials, non-randomized quantitative studies, and mixed methods studies. Two key features are a section for assessing bias of the implementation outcomes and sections for describing the implementation strategy and intervention implemented. An accompanying explanation and elaboration document that identifies and describes each of the items, explains the rationale, and provides examples of reporting and appraising practice, as well as templates to allow synthesis of extracted data across studies and an instructional video, has been prepared.

Conclusions

The comprehensive, adaptable tool to support both reporting and critical appraisal of implementation science studies including quantitative, qualitative, and mixed methods assessment of intervention and implementation outcomes has been developed. This tool can be applied to a methodologically diverse and growing body of implementation science literature to support reviews or meta-analyses that inform evidence-based decision-making regarding processes and strategies for implementation.

Care team and practice-level implementation strategies to optimize pediatric collaborative care: study protocol for a cluster-randomized hybrid type III trial

BACKGROUND

Implementation facilitation is an effective strategy to support the implementation of evidence-based practices (EBPs), but our understanding of multilevel strategies and the mechanisms of change within the "black box" of implementation facilitation is limited. This implementation trial seeks to disentangle and evaluate the effects of facilitation strategies that separately target the care team and leadership levels on implementation of a collaborative care model in pediatric primary care. Strategies targeting the provider care team (TEAM) should engage team-level mechanisms, and strategies targeting leaders (LEAD) should engage organizational mechanisms.

METHODS

We will conduct a hybrid type 3 effectiveness-implementation trial in a 2 × 2 factorial design to evaluate the main and interactive effects of TEAM and LEAD and test for mediation and moderation of effects. Twenty-four pediatric primary care practices will receive standard REP training to implement Doctor-Office Collaborative Care (DOCC) and then be randomized to (1) Standard REP only, (2) TEAM, (3) LEAD, or (4) TEAM + LEAD. Implementation outcomes are DOCC service delivery and change in practice-level care management competencies. Clinical outcomes are child symptom severity and quality of life.

DISCUSSION

This statewide trial is one of the first to test the unique and synergistic effects of implementation strategies targeting care teams and practice leadership. It will advance our knowledge of effective care team and practice-level implementation strategies and mechanisms of change. Findings will support efforts to improve common child behavioral health conditions by optimizing scale-up and sustainment of CCMs in a pediatric patient-centered medical home.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT04946253 . Registered June 30, 2021.

Development and Preliminary Evaluation of an Implementation Facilitation Training Program

Background: Implementation scientists are identifying evidence-based implementation strategies that support the uptake of evidence-based practices and other clinical innovations. However, there is limited information regarding the development of training methods to educate implementation practitioners on the use of implementation strategies and help them sustain these competencies. Methods: To address this need, we developed, implemented, and evaluated a training program for one strategy, implementation facilitation (IF), that was designed to maximize applicability in diverse clinical settings. Trainees included implementation practitioners, clinical managers, and researchers. From May 2017 to July 2019, we sent trainees an electronic survey via email and asked them to complete the survey at three-time points: approximately 2 weeks before and 2 weeks and 6 months after each training. Participants ranked their knowledge of and confidence in applying IF skills using a 4-point Likert scale. We compared scores at baseline to post-training and at 6 months, as well as post-training to 6 months post-training (nonparametric Wilcoxon signed-rank tests). Results: Of the 102 participants (76 in-person, 26 virtual), there was an increase in perceived knowledge and confidence in applying IF skills across all learning objectives from pre- to post-training (95% response rate) and pre- to 6-month (35% response rate) follow-up. There was no significant difference in results between virtual and in-person trainees. When comparing post-training to 6 months (30% response rate), perceptions of knowledge increase remained unchanged, although participants reported reduced perceived confidence in applying IF skills for half of the learning objectives at 6 months. Conclusions: Findings indicated that we have developed a promising IF training program. Lack of differences in results between virtual and in-person participants indicated the training can be provided to a remote site without loss of knowledge/skills transfer but ongoing support may be needed to help sustain perceived confidence in applying these skills.

Plain Language Summary

While implementation scientists are documenting an increasing number of implementation strategies that support the uptake of evidence-based practices and other clinical innovations, little is known about how to transfer this knowledge to those who conduct implementation efforts in the frontline clinical practice settings. We developed, implemented, and conducted a preliminary evaluation of a training program for one strategy, implementation facilitation (IF). The training program targets facilitation practitioners, clinical managers, and researchers. This paper describes the development of the training program, the program components, and the results from an evaluation of IF knowledge and skills reported by a subset of people who participated in the training. Findings from the evaluation indicate that this training program significantly increased trainees' perceived knowledge of and confidence in applying IF skills. Further research is needed to examine whether ongoing mentoring helps trainees retain confidence in applying some IF skills over the longer term.

Economic costs of implementing evidence-based telemedicine outreach for posttraumatic stress disorder in VA

Background

Telemedicine outreach for posttraumatic stress disorder (TOP) is a virtual evidence-based practice (EBP) involving telephone care management and telepsychology that engages rural patients in trauma-focused psychotherapy. This evaluation examined implementation and intervention costs attributable to deploying TOP from a health system perspective.

Methods

Costs were ascertained as part of a stepped wedge cluster randomized trial at five sites within the Veterans Affairs (VA) Healthcare System. All sites initially received a standard implementation strategy, which included internal facilitation, dissemination of an internal facilitators operational guide, funded care manager, care managing training, and technical support. A subset of clinics that failed to meet performance metrics were subsequently randomized to enhanced implementation, which added external facilitation that focused on incorporating TOP clinical processes into existing clinic workflow. We measured site-level implementation activities using project records and structured activity logs tracking personnel-level time devoted to all implementation activities. We monetized time devoted to implementation activities by applying an opportunity cost approach. Intervention costs were measured as accounting-based costs for telepsychiatry/telepsychology and care manager visits, ascertained using VA administrative data. We conducted descriptive analyses of strategy-specific implementation costs across five sites. Descriptive analyses were conducted instead of population-level cost-effectiveness analysis because previous research found enhanced implementation was not more successful than the standard implementation in improving uptake of TOP.

Results

Over the 40-month study period, four of five sites received enhanced implementation. Mean site-level implementation cost per month was $919 (SD = $238) during standard implementation and increased to $1,651 (SD = $460) during enhanced implementation. Mean site-level intervention cost per patient-month was $46 (SD = $28) during standard implementation and $31 (SD = $21) during enhanced implementation.

Conclusions

Project findings inform the expected cost of implementing TOP, which represents one factor health systems should consider in the decision to broadly adopt this EBP. Plain Language Summary: What is already known about the topic: Trauma-focused psychotherapy delivered through telemedicine has been demonstrated as an effective approach for the treatment of post-traumatic stress disorder (PTSD). However, uptake of this evidence-based approach by integrated health systems such as the Veterans Affairs (VA) Health Care System is low. What does this paper add: This paper presents new findings on the costs of two implementation approaches designed to increase adoption telemedicine outreach for PTSD from a health system perspective. What are the implications for practice, research, and policy: Cost estimates from this paper can be used by health systems to inform the relative value of candidate implementation strategies to increase adoption of evidence-based treatments for PTSD or other mental health conditions.

Translating Violence Prevention Programs from Research to Practice: SafERteens Implementation in an Urban Emergency Department

BACKGROUND

Youth violence is a leading cause of adolescent mortality, underscoring the need to integrate evidence-based violence prevention programs into routine emergency department (ED) care.

OBJECTIVES

To examine the translation of the SafERteens program into clinical care.

METHODS

Hospital staff provided input on implementation facilitators/barriers to inform toolkit development. Implementation was piloted in a four-arm effectiveness-implementation trial, with youth (ages 14-18 years) screening positive for past 3-month aggression randomized to either SafERteens (delivered remotely or in-person) or enhanced usual care (EUC; remote or in-person), with follow-up at post-test and 3 months. During maintenance, ED staff continued in-person SafERteens delivery and external facilitation was provided. Outcomes were measured using the RE-AIM implementation framework.

RESULTS

SafERteens completion rates were 77.6% (52/67) for remote and 49.1% (27/55) for in-person delivery. In addition to high acceptability ratings (e.g., helpfulness), post-test data demonstrated increased self-efficacy to avoid fighting among patients receiving remote (incidence rate ratio [IRR] 1.22, 95% confidence interval [CI] 1.09-1.36) and in-person (IRR 1.23, 95% CI 1.12-1.36) SafERteens, as well as decreased pro-violence attitudes among patients receiving remote (IRR 0.83, 95% CI 0.75-0.91) and in-person (IRR 0.87, 95% CI 0.77-0.99) SafERteens when compared with their respective EUC groups. At 3 months, youth receiving remote SafERteens reported less non-partner aggression (IRR 0.52, 95% CI 0.31-0.87, Cohen's d -0.39) and violence consequences (IRR 0.47, 95% CI 0.22-1.00, Cohen's d -0.49) compared with remote EUC; no differences were noted for in-person SafERteens delivery. Barriers to implementation maintenance included limited staff availability and a lack of reimbursement codes.

CONCLUSIONS

Implementing behavioral interventions such as SafERteens into routine ED care is feasible using remote delivery. Policymakers should consider reimbursement for violence prevention services to sustain long-term implementation.

The Life Goals Self-Management Mobile App for Bipolar Disorder: Consumer Feasibility, Usability, and Acceptability Study

Background

Life Goals is an evidence-based self-management intervention that assists individuals with bipolar disorder (BD) by aligning BD symptom coping strategies with their personal goals. The intervention can be availed via in-person and telephonic sessions, and it has been recently developed as an individualized, customizable mobile app.

Objective

We examined the feasibility, usability, and acceptability of the Life Goals self-management app among individuals diagnosed with BD who used the app for up to 6 months.

Methods

A total of 28 individuals with BD used the Life Goals app on their personal smartphone for 6 months. They completed key clinical outcome measurements of functioning, disability, and psychiatric symptoms at baseline, 3 months, and 6 months, in addition to a poststudy survey about usability and satisfaction.

Results

Participants used the app for a median of 25 times (IQR 13-65.75), and for a longer time during the first 3 months of the study. The modules on depression and anxiety were the most frequently used, accounting for 35% and 22% of total usage, respectively. Overall, the study participants found the app useful (15/25, 60%) and easy to use (18/25, 72%), and they reported that the screen displayed the material adequately (22/25, 88%). However, less than half of the participants found the app helpful in managing their health (10/25, 40%) or in making progress on their wellness goals (9/25, 36%). Clinical outcomes showed a trend for improvements in mental and physical health and mania-related well-being.

Conclusions

The Life Goals app showed feasibility of use among individuals with BD. Higher user engagement was observed in the initial 3 months with users interested more frequently in the mood modules than other wellness modules. Participants reported acceptability with the ease of app use and satisfaction with the app user interface, but the app showed low success in encouraging self-management within this small sample. The Life Goals app is a mobile health technology that can provide individuals with serious mental illness with more flexible access to evidence-based treatments.

Optimization Methods and Implementation Science: An Opportunity for Behavioral and Biobehavioral Interventions

This editorial introduces the multiphase optimization strategy (MOST), a principled framework for the development, optimization and evaluation of multicomponent interventions, to the field of implementation science. We suggest that MOST may be integrated with implementation science to advance the field, moving closer towards the ultimate goal of disseminating effective interventions to those in need. We offer three potential ways MOST may advance implementation science: (1) development of an effective and immediately scalable intervention; (2) adaptation of interventions to local contexts; and (3) optimization of the implementation of an intervention itself. Our goal is to inspire the integration of MOST with implementation science across a number of public health contexts.

Optimal allocation to treatments in a sequential multiple assignment randomized trial

One of the main questions in the design of a trial is how many subjects should be assigned to each treatment condition. Previous research has shown that equal randomization is not necessarily the best choice. We study the optimal allocation for a novel trial design, the sequential multiple assignment randomized trial, where subjects receive a sequence of treatments across various stages. A subject's randomization probabilities to treatments in the next stage depend on whether he or she responded to treatment in the current stage. We consider a prototypical sequential multiple assignment randomized trial design with two stages. Within such a design, many pairwise comparisons of treatment sequences can be made, and a multiple-objective optimal design strategy is proposed to consider all such comparisons simultaneously. The optimal design is sought under either a fixed total sample size or a fixed budget. A Shiny App is made available to find the optimal allocations and to evaluate the efficiency of competing designs. As the optimal design depends on the response rates to first-stage treatments, maximin optimal design methodology is used to find robust optimal designs. The proposed methodology is illustrated using a sequential multiple assignment randomized trial example on weight loss management.

Strengthening methods for tracking adaptations and modifications to implementation strategies

BACKGROUND

Developing effective implementation strategies requires adequate tracking and reporting on their application. Guidelines exist for defining and reporting on implementation strategy characteristics, but not for describing how strategies are adapted and modified in practice. We built on existing implementation science methods to provide novel methods for tracking strategy modifications.

METHODS

These methods were developed within a stepped-wedge trial of an implementation strategy package designed to help community clinics adopt social determinants of health-related activities: in brief, an 'Implementation Support Team' supports clinics through a multi-step process. These methods involve five components: 1) describe planned strategy; 2) track its use; 3) monitor barriers; 4) describe modifications; and 5) identify / describe new strategies. We used the Expert Recommendations for Implementing Change taxonomy to categorize strategies, Proctor et al.'s reporting framework to describe them, the Consolidated Framework for Implementation Research to code barriers / contextual factors necessitating modifications, and elements of the Framework for Reporting Adaptations and Modifications-Enhanced to describe strategy modifications.

RESULTS

We present three examples of the use of these methods: 1) modifications made to a facilitation-focused strategy (clinics reported that certain meetings were too frequent, so their frequency was reduced in subsequent wedges); 2) a clinic-level strategy addition which involved connecting one study clinic seeking help with community health worker-related workflows to another that already had such a workflow in place; 3) a study-level strategy addition which involved providing assistance in overcoming previously encountered (rather than de novo) challenges.

CONCLUSIONS

These methods for tracking modifications made to implementation strategies build on existing methods, frameworks, and guidelines; however, as none of these were a perfect fit, we made additions to several frameworks as indicated, and used certain frameworks' components selectively. While these methods are time-intensive, and more work is needed to streamline them, they are among the first such methods presented to implementation science. As such, they may be used in research on assessing effective strategy modifications and for replication and scale-up of effective strategies. We present these methods to guide others seeking to document implementation strategies and modifications to their studies.

TRIAL REGISTRATION

clinicaltrials.gov ID: NCT03607617 (first posted 31/07/2018).

Strengthening methods for tracking adaptations and modifications to implementation strategies

BACKGROUND

Developing effective implementation strategies requires adequate tracking and reporting on their application. Guidelines exist for defining and reporting on implementation strategy characteristics, but not for describing how strategies are adapted and modified in practice. We built on existing implementation science methods to provide novel methods for tracking strategy modifications.

METHODS

These methods were developed within a stepped-wedge trial of an implementation strategy package designed to help community clinics adopt social determinants of health-related activities: in brief, an 'Implementation Support Team' supports clinics through a multi-step process. These methods involve five components: 1) describe planned strategy; 2) track its use; 3) monitor barriers; 4) describe modifications; and 5) identify / describe new strategies. We used the Expert Recommendations for Implementing Change taxonomy to categorize strategies, Proctor et al.'s reporting framework to describe them, the Consolidated Framework for Implementation Research to code barriers / contextual factors necessitating modifications, and elements of the Framework for Reporting Adaptations and Modifications-Enhanced to describe strategy modifications.

RESULTS

We present three examples of the use of these methods: 1) modifications made to a facilitation-focused strategy (clinics reported that certain meetings were too frequent, so their frequency was reduced in subsequent wedges); 2) a clinic-level strategy addition which involved connecting one study clinic seeking help with community health worker-related workflows to another that already had such a workflow in place; 3) a study-level strategy addition which involved providing assistance in overcoming previously encountered (rather than de novo) challenges.

CONCLUSIONS

These methods for tracking modifications made to implementation strategies build on existing methods, frameworks, and guidelines; however, as none of these were a perfect fit, we made additions to several frameworks as indicated, and used certain frameworks' components selectively. While these methods are time-intensive, and more work is needed to streamline them, they are among the first such methods presented to implementation science. As such, they may be used in research on assessing effective strategy modifications and for replication and scale-up of effective strategies. We present these methods to guide others seeking to document implementation strategies and modifications to their studies.

TRIAL REGISTRATION

clinicaltrials.gov ID: NCT03607617 (first posted 31/07/2018).

Implementation of peer specialist services in VA primary care: a cluster randomized trial on the impact of external facilitation

Background

Over 1100 veterans work in the Veterans Health Administration (VHA) as peer specialists (PSs)—those with formal training who support other veterans with similar diagnoses. A White House Executive Action mandated the pilot reassignment of VHA PSs from their usual placement in mental health to 25 primary care Patient Aligned Care Teams (PACTs) in order to broaden the provision of wellness services that can address many chronic illnesses. An evaluation of this initiative was undertaken to assess the impact of outside assistance on the deployment of PSs in PACTs, as implementation support is often needed to prevent challenges commonly experienced when first deploying PSs in new settings.

Methods

This study was a cluster-randomized hybrid II effectiveness-implementation trial to test the impact of minimal implementation support vs. facilitated implementation on the deployment of VHA PSs in PACT over 2 years. Twenty-five Veterans Affairs Medical Centers (VAMCs) were recruited to reassign mental health PSs to provide wellness-oriented care in PACT. Sites in three successive cohorts (n = 7, 10, 8) over 6-month blocks were matched and randomized to each study condition. In facilitated implementation, an outside expert worked with site stakeholders through a site visit and regular calls, and provided performance data to guide the planning and address challenges. Minimal implementation sites received a webinar and access to the VHA Office of Mental Health Services work group. The two conditions were compared on PS workload data and veteran measures of activation, satisfaction, and functioning. Qualitative interviews collected information on perceived usefulness of the PS services.

Results

In the first year, sites that received facilitation had higher numbers of unique veterans served and a higher number of PS visits, although the groups did not differ after the second year. Also, sites receiving external facilitation started delivering PS services more quickly than minimal support sites. All sites in the external facilitation condition continued in the pilot into the second year, whereas two of the sites in the minimal assistance condition dropped out after the first year. There were no differences between groups on veterans’ outcomes—activation, satisfaction, and functioning. Most veterans were very positive about the help they received as evidenced in the qualitative interviews.

Discussion

These findings demonstrate that external facilitation can be effective in supporting the implementation of PSs in primary care settings. The lack of significant differences across conditions after the second year highlights the positive outcomes associated with active facilitation, while also raising the important question of whether longer-term success may require some level of ongoing facilitation and implementation support.

Trial registration

This project is registered at ClinicalTrials.gov with number NCT02732600 (URL: https://clinicaltrials.gov/ct2/show/NCT02732600).

Implementation science should give higher priority to health equity

Background

There is growing urgency to tackle issues of equity and justice in the USA and worldwide. Health equity, a framing that moves away from a deficit mindset of what society is doing poorly (disparities) to one that is positive about what society can achieve, is becoming more prominent in health research that uses implementation science approaches. Equity begins with justice—health differences often reflect societal injustices. Applying the perspectives and tools of implementation science has potential for immediate impact to improve health equity.

Main text

We propose a vision and set of action steps for making health equity a more prominent and central aim of implementation science, thus committing to conduct implementation science through equity-focused principles to achieve this vision in U.S. research and practice. We identify and discuss challenges in current health disparities approaches that do not fully consider social determinants. Implementation research challenges are outlined in three areas: limitations of the evidence base, underdeveloped measures and methods, and inadequate attention to context. To address these challenges, we offer recommendations that seek to (1) link social determinants with health outcomes, (2) build equity into all policies, (3) use equity-relevant metrics, (4) study what is already happening, (5) integrate equity into implementation models, (6) design and tailor implementation strategies, (7) connect to systems and sectors outside of health, (8) engage organizations in internal and external equity efforts, (9) build capacity for equity in implementation science, and (10) focus on equity in dissemination efforts.

Conclusions

Every project in implementation science should include an equity focus. For some studies, equity is the main goal of the project and a central feature of all aspects of the project. In other studies, equity is part of a project but not the singular focus. In these studies, we should, at a minimum, ensure that we “leave no one behind” and that existing disparities are not widened. With a stronger commitment to health equity from funders, researchers, practitioners, advocates, evaluators, and policy makers, we can harvest the rewards of the resources being invested in health-related research to eliminate disparities, resulting in health equity.

Adapting the InPACT Intervention to Enhance Implementation Fidelity and Flexibility

Adapting classroom-based physical activity interventions are critical for program feasibility and fidelity in low-resource elementary schools. The purpose of this pilot study was to use Replicating Effective Programs (REP) framework to tailor the Interrupting Prolonged sitting with ACTivity (InPACT) intervention and evaluate its effectiveness on program fidelity in classrooms within a low-resource school. REP was applied to adapt program packaging, teacher training, and technical assistance to disseminate Tailored InPACT, a 20-week intervention where teachers implemented 5 × 4-min activity breaks per day. Tailored InPACT was implemented in nine, 3rd-6th grade classrooms in one low-resource school in Detroit Michigan (80% qualified for free/reduced lunch). Intervention fidelity was measured via daily, weekly, and end-of-study self-report questionnaires and direct observation. Throughout the 20-week intervention period, 3rd-5th grade teachers achieved intervention dose (5 activity breaks per day at an average duration of 4 min 8 s). Sixth grade teachers did not achieve intervention dose as they were only able to implement 2 activity breaks per day at an average duration of 4 min 12 s. These findings suggest 5 × 4-min classroom activity breaks per day is a feasible dose of classroom activity that 3rd-5th grade teachers can implement in low-resource classroom settings. Additional adaptations are needed to maximize fidelity in 6th grade classrooms.

Cost-effectiveness of the Adaptive Implementation of Effective Programs Trial (ADEPT): approaches to adopting implementation strategies

BACKGROUND

Theory-based methods to support the uptake of evidence-based practices (EBPs) are critical to improving mental health outcomes. Implementation strategy costs can be substantial, and few have been rigorously evaluated. The purpose of this study is to conduct a cost-effectiveness analysis to identify the most cost-effective approach to deploying implementation strategies to enhance the uptake of Life Goals, a mental health EBP.

METHODS

We used data from a previously conducted randomized trial to compare the cost-effectiveness of Replicating Effective Programs (REP) combined with external and/or internal facilitation among sites non-responsive to REP. REP is a low-level strategy that includes EBP packaging, training, and technical assistance. External facilitation (EF) involves external expert support, and internal facilitation (IF) augments EF with protected time for internal staff to support EBP implementation. We developed a decision tree to assess 1-year costs and outcomes for four implementation strategies: (1) REP only, (2) REP+EF, (3) REP+EF add IF if needed, (4) REP+EF/IF. The analysis used a 1-year time horizon and assumed a health payer perspective. Our outcome was quality-adjusted life years (QALYs). The economic outcome was the incremental cost-effectiveness ratio (ICER). We conducted deterministic and probabilistic sensitivity analysis (PSA).

RESULTS

Our results indicate that REP+EF add IF is the most cost-effective option with an ICER of $593/QALY. The REP+EF/IF and REP+EF only conditions are dominated (i.e., more expensive and less effective than comparators). One-way sensitivity analyses indicate that results are sensitive to utilities for REP+EF and REP+EF add IF. The PSA results indicate that REP+EF, add IF is the optimal strategy in 30% of iterations at the threshold of $100,000/QALY.

CONCLUSIONS

Our results suggest that the most cost-effective implementation support begins with a less intensive, less costly strategy initially and increases as needed to enhance EBP uptake. Using this approach, implementation support resources can be judiciously allocated to those clinics that would most benefit. Our results were not robust to changes in the utility measure. Research is needed that incorporates robust and relevant utilities in implementation studies to determine the most cost-effective strategies. This study advances economic evaluation of implementation by assessing costs and utilities across multiple implementation strategy combinations.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02151331 , 05/30/2014.

Comparative effectiveness of external vs blended facilitation on collaborative care model implementation in slow-implementer community practices

OBJECTIVE

To evaluate the comparative effectiveness of external facilitation (EF) vs external + internal facilitation (EF/IF), on uptake of a collaborative chronic care model (CCM) in community practices that were slower to implement under low-level implementation support.

STUDY SETTING

Primary data were collected from 43 community practices in Michigan and Colorado at baseline and for 12 months following randomization.

STUDY DESIGN

Sites that failed to meet a pre-established implementation benchmark after six months of low-level implementation support were randomized to add either EF or EF/IF support for up to 12 months. Key outcomes were change in number of patients receiving the CCM and number of patients receiving a clinically significant dose of the CCM. Moderators' analyses further examined whether comparative effectiveness was dependent on prerandomization adoption, number of providers trained or practice size. Facilitation log data were used for exploratory follow-up analyses.

DATA COLLECTION

Sites reported monthly on number of patients that had received the CCM. Facilitation logs were completed by study EF and site IFs and shared with the study team.

PRINCIPAL FINDINGS

N = 21 sites were randomized to EF and 22 to EF/IF. Overall, EF/IF practices saw more uptake than EF sites after 12 months (Δ_EF/IF-EF  = 4.4 patients, 95% CI = 1.87-6.87). Moderators' analyses, however, revealed that it was only sites with no prerandomization uptake of the CCM (nonadopter sites) that saw significantly more benefit from EF/IF (Δ_EF/IF-EF  = 9.2 patients, 95% CI: 5.72, 12.63). For sites with prerandomization uptake (adopter sites), EF/IF offered no additional benefit (Δ_EF/IF-EF  = -0.9; 95% CI: -4.40, 2.60). Number of providers trained and practice size were not significant moderators.

CONCLUSIONS

Although stepping up to the more intensive EF/IF did outperform EF overall, its benefit was limited to sites that failed to deliver any CCM under the low-level strategy. Once one or more providers were delivering the CCM, additional on-site personnel did not appear to add value to the implementation effort.

The use of the PARIHS framework in implementation research and practice-a citation analysis of the literature

BACKGROUND

The Promoting Action on Research Implementation in Health Services (PARIHS) framework was developed two decades ago and conceptualizes successful implementation (SI) as a function (f) of the evidence (E) nature and type, context (C) quality, and the facilitation (F), [SI = f (E,C,F)]. Despite a growing number of citations of theoretical frameworks including PARIHS, details of how theoretical frameworks are used remains largely unknown. This review aimed to enhance the understanding of the breadth and depth of the use of the PARIHS framework.

METHODS

This citation analysis commenced from four core articles representing the key stages of the framework's development. The citation search was performed in Web of Science and Scopus. After exclusion, we undertook an initial assessment aimed to identify articles using PARIHS and not only referencing any of the core articles. To assess this, all articles were read in full. Further data extraction included capturing information about where (country/countries and setting/s) PARIHS had been used, as well as categorizing how the framework was applied. Also, strengths and weaknesses, as well as efforts to validate the framework, were explored in detail.

RESULTS

The citation search yielded 1613 articles. After applying exclusion criteria, 1475 articles were read in full, and the initial assessment yielded a total of 367 articles reported to have used the PARIHS framework. These articles were included for data extraction. The framework had been used in a variety of settings and in both high-, middle-, and low-income countries. With regard to types of use, 32% used PARIHS in planning and delivering an intervention, 50% in data analysis, 55% in the evaluation of study findings, and/or 37% in any other way. Further analysis showed that its actual application was frequently partial and generally not well elaborated.

CONCLUSIONS

In line with previous citation analysis of the use of theoretical frameworks in implementation science, we also found a rather superficial description of the use of PARIHS. Thus, we propose the development and adoption of reporting guidelines on how framework(s) are used in implementation studies, with the expectation that this will enhance the maturity of implementation science.

Protocol: examining the effectiveness of an adaptive implementation intervention to improve uptake of the VA suicide risk identification strategy: a sequential multiple assignment randomized trial

Background

In 2018, the Veterans Health Administration (VHA) mandated implementation of a national suicide risk identification strategy (Risk ID). The goal of Risk ID is to improve the detection and management of suicide risk by standardizing suicide risk screening and evaluation enterprise-wide. In order to ensure continuous quality improvement (QI), ongoing evaluation and targeted interventions to improve implementation of Risk ID are needed. Moreover, given that facilities will vary with respect to implementation needs and barriers, the dose and type of intervention needed may vary across facilities. Thus, the objective of this study is to examine the effectiveness of an adaptive implementation strategy to improve the uptake of suicide risk screening and evaluation in VHA ambulatory care settings. In addition, this study will examine specific factors that may impact the uptake of suicide risk screening and evaluation and the adoption of different implementation strategies. This protocol describes the stepped implementation approach and proposed evaluation plan.

Methods

Using a sequential multiple assignment randomized trial (SMART) design, two evidence-based implementation strategies will be evaluated: (1) audit and feedback (A&F); (2) A&F plus external facilitation (A&F + EF). Implementation outcomes of interest include uptake of secondary suicide risk screening and uptake of comprehensive suicide risk evaluation (stages 2 and 3 of Risk ID). Secondary outcomes include rates of other clinical outcomes (i.e., safety planning) and organizational factors that may impact Risk ID implementation (i.e., leadership climate and leadership support).

Discussion

This national QI study will use a SMART design to evaluate whether an adaptive implementation strategy is effective in improving uptake of a mandated VHA-wide suicide risk screening and evaluation initiative. If this study finds that the proposed stepped implementation strategy is effective at increasing uptake and maintaining performance improvements, this approach may be used as an overarching QI strategy for other national suicide prevention programs.

Trial registration

ClinicalTrials.gov NCT04243330. Registered 28 January 2020