A systematic review of empirical studies examining mechanisms of implementation in health

Supporting implementation of universal prevention initiatives in K-12 schools: impacts on fidelity through organizational readiness and team functioning in a cluster-randomized trial

BACKGROUND

Rural School Support Strategies (RS3) is a bundle of implementation supports (including training, technical assistance, and a virtual learning collaborative) designed for the scale-up of universal prevention initiatives. This study addresses mechanisms of action, exploring whether positive effects of RS3 on implementation fidelity are attributable to improvements in functioning of school implementation teams, and increases in organizational readiness.

METHODS

Data are from a cluster-randomized hybrid Type 3 implementation-effectiveness trial of RS3 among rural Idaho schools implementing Positive Behavioral Interventions and Supports (PBIS). Forty public K-12 schools in Idaho, located in rural areas or townships, were recruited for the trial and were equally randomized to either the basic supports condition, including standard trainings, or to the RS3 condition. Condition was not masked. The mechanistic aims were hypothesized prior to the trial and tested with survey data from individuals on each school's implementation team (n = 205). Surveys were collected in spring 2019 and 2020 regarding organizational readiness and team functioning. The outcome measure was PBIS implementation fidelity, measured by school teams during the summers of 2019 and 2020 using the Tiered Fidelity Inventory. School-level path models tested the effect of RS3 on implementation fidelity, controlling for baseline, school grade level, and school location. Multilevel (2-1-2) mediation models tested the degree to which individual team members' perceptions of organizational readiness and team functioning mediated the relationship between school-level experimental condition and fidelity, controlling for the team members' role.

RESULTS

Schools receiving RS3 reported significantly greater implementation fidelity, although effects were slightly reduced (b = 8.40, p = .056, 95% CI [-0.22,17.01], β = 0.54) after inclusion of baseline and demographic controls. Models indicated a significant indirect effect of RS3 on fidelity through increased team productivity (b = 6.30, SE = 2.63, p = .017, MC 95% CI [0.83,13.86], β = 0.21), and effects through organizational readiness, change commitment, team culture, and team goal setting.

CONCLUSIONS

External supports may improve implementation of universal prevention initiatives in rural schools through improvements in readiness and fostering teaming in organizations.

TRIAL REGISTRATION

This research was prospectively registered on ClinicalTrials.gov ( NCT03736395 ), on November 9, 2018.

Development of an implementation intervention to promote adoption of the COMFORT clinical practice guideline for peripartum pain management: a qualitative study

BACKGROUND

Pain management after childbirth is widely variable, increasing risk of untreated pain, opioid harms, and inequitable experiences of care. The Creating Optimal Pain Management FOR Tailoring Care (COMFORT) clinical practice guideline (CPG) seeks to promote evidence-based, equitable acute peripartum pain management in the United States. We aimed to identify contextual conditions (i.e., barriers and facilitators) and discrete implementation strategies (i.e., theory-based actions taken to routinize a clinical practice) likely to influence COMFORT CPG uptake and specify corresponding multi-component implementation interventions at the perinatal quality collaborative- and unit-level.

METHODS

We conducted a qualitative study involving virtual individual interviews and focus groups. Interviews included individuals undergoing childbirth from 2018-2023, (recruited through two online registries), and actively practicing maternity clinicians and surgeons, (recruited via snowball sampling with the eDelphi panel creating the COMFORT CPG), caring for pregnant people in the United States. Focus groups included physicians, midwives, nurses, and unit-based quality improvement (QI) staff working at Michigan hospitals within the Obstetrics Initiative, a statewide perinatal quality collaborative funded by Blue Cross Blue Shield of Michigan and Blue Care Network. The Consolidated Framework for Implementation Research, Expert Recommendations for Implementing Change taxonomy, and Replicating Effective Programs framework informed data collection and analysis. Qualitative content analysis characterized influential contextual conditions, which were linked to implementation strategies and tools using principles of implementation mapping. We then specified multi-component implementation interventions for use by quality collaboratives and unit-based teams.

RESULTS

From May-September 2023, we completed 57 semi-structured individual interviews (31 patients, 26 clinicians) and six focus groups (44 QI champions). Participants identified 10 key conditions influential for COMFORT CPG adoption. Findings enabled identification of five collaborative-level implementation strategies, 27 unit-level implementation strategies, and 12 associated tools to promote COMFORT CPG adoption including the specification of each strategy's hypothesized mechanism of action and each tool's goal and potential uses.

CONCLUSIONS

This work identifies contextual conditions and implementation strategies and tools at the perinatal quality collaborative and unit levels to promote COMFORT CPG adoption on maternity units. These findings may foster more rapid CPG implementation and thereby promote more equitable and evidence-based perinatal pain management care.

Generalizing from qualitative data: a case example using critical realist thematic analysis and mechanism mapping to evaluate a community health worker-led screening program in India

BACKGROUND

A central goal of implementation science is to generate insights that allow evidence-based practices to be successfully applied across diverse settings. However, challenges often arise in preserving programs' effectiveness outside the context of their intervention development. We propose that qualitative data can inform generalizability via elucidating mechanisms of an intervention. Critical realist thematic analysis provides a framework for applying qualitative data to identify causal relationships. This approach can be used to develop mechanism maps, a tool rooted in policy that has been used in health systems interventions, to explain how and why interventions work. We illustrate use of these approaches through a case example of a community health worker (CHW)-delivered gestational diabetes (GDM) screening intervention in Pune, India. CHWs successfully improved uptake of oral glucose tolerance tests (OGTT) among pregnant women, however clinical management of GDM was suboptimal.

METHODS

Qualitative interviews were conducted with 53 purposively sampled participants (pregnant women, CHWs, maternal health clinicians). Interview transcripts were reviewed using a critical realist thematic analysis approach to develop a coding scheme pertinent to our research questions: "What caused high uptake of GDM screening?" and "Why did most women with GDM referred to clinics did not receive evidence-based management?". Mechanism maps were retrospectively generated using short- and long-term outcomes as fenceposts to illustrate causal pathways of the CHW-delivered program and subsequent clinical GDM management.

RESULTS

Critical realist thematic analysis generated mechanism maps showed that CHWs facilitated GDM screening uptake through affective, cognitive and logistic pathways of influence. Lack of evidence-based treatment of GDM at clinics was caused by 1) clinicians lacking time or initiative to provide GDM counseling and 2) low perceived pre-test probability of GDM in this population of women without traditional risk factors. Mechanism mapping identified areas for adaptation to improve the intervention for future iterations.

CONCLUSIONS

Mechanism maps created by repeated engagement following the critical realist thematic analysis method can provide a retrospective framework to understand causal relationships between factors driving intervention successes or failures. This process, in turn, can inform the generalizability of health programs by identifying constituent factors and their interrelationships that are central to implementation.

Tailored Multifaceted Strategy for Implementing Fundamental Evidence-Based Nursing Care: An Evaluation Study

Background/Objectives: Extensive research has emphasised the persistent challenges and failures in providing hospitalised patients with fundamental evidence-based nursing care, often resulting in grave consequences for patient safety. Recommendations from implementation research indicate that a tailored theory- and research-based implementation strategy targeting contextual determinants can optimise the implementation of evidence-based clinical practice for the benefit of patients. This study evaluated the feasibility of an implementation strategy designed to improve the quality of nursing care by targeting behavioural and environmental barriers in a hospital setting. Methods: Proctor's conceptual model for implementation was applied to evaluate the strategy based on eight outcomes: adoption, acceptability, appropriateness, fidelity, feasibility, penetration, sustainability, and costs. Data collection methods included field observations, informal and focus group interviews, registrations, and audits of electronic patient records. Results: The strategy was adoptive, acceptable, appropriate, and feasible in targeting complex environmental and behavioural determinants (at the individual, team, and management level), enabling successful implementation of fundamental evidence-based nursing care. However, fidelity, feasibility, and sustainability were challenged by competing organisational demands and time constraints. Conclusions: The tailored, multifaceted strategy proved effective in addressing complex environmental and behavioural determinants across multiple levels, facilitating the implementation of fundamental evidence-based nursing care in a clinical practice. Further testing and larger-scale studies is needed to assess the strategy's transferability and its impact on nursing-sensitive patient outcomes in different clinical settings.

Context, mechanisms and community engagement in implementation strategies: a systematic review protocol of implementation strategies designed using implementation mapping

INTRODUCTION

Implementation Mapping (IM) has increasingly been used for the design and/or selection and tailoring implementation strategies to promote the effective translation of evidence-based interventions into practice. IM is a tool for guiding the systematic, iterative, step-by-step design process that helps guide the co-creation of theory-based, evidence-informed implementation strategies. This protocol aims to describe the methods of the planned systematic review that will (1) examine the overall landscape of IM in developing implementation strategies, (2) explore the mechanisms of implementation strategies developed using IM and (3) investigate the use of IM and community engagement in the development of implementation strategies. This review will produce synthesised evidence that will describe how IM has been used in selecting, designing and tailoring implementation strategies, with a focus on implementation mechanism and community engagement.

METHODS AND ANALYSIS

We will include peer-reviewed original English articles that describe the use of Implementation Mapping or Intervention Mapping to design implementation strategies. We will search Medline Ovid, PsycInfo, Embase, CINAHL and Web of Science using the following terms: 'Implementation Mapping' and 'Intervention Mapping'. At least two reviewers will independently screen the records to examine the eligibility. After several rounds of consistency testing of data extraction among at least two reviewers and confirming the consistent coding among the reviewer team, one coder will proceed with the data extraction. In case of uncertainty, we will reach a consensus through interactive reviewers' meetings and in consultation with a senior author. We examined the risk of bias in individual studies using the quality assessment tool developed by Hawker et al in 2002. We will synthesise and present the data by each research aim.

ETHICS AND DISSEMINATION

Ethics approval was not required. Findings will be shared via peer-reviewed journals, and data from the included studies will be made openly accessible.

A Partner-Engaged Approach to Developing an Implementation Research Logic Model for a Traumatic Brain Injury-Intensive Evaluation and Treatment Program

BACKGROUND

A partnered evaluation project with Veterans Health Administration Physical Medicine and Rehabilitation program office uses a partner-engaged approach to characterize and evaluate the national implementation of traumatic brain injury (TBI)Intensive Evaluation and Treatment Program (IETP).

OBJECTIVE

This paper illustrates a partner-engaged approach to contextualizing the IETP within an implementation research logic model (IRLM) to inform program sustainment and spread.

SETTING

The project was conducted at five IETP sites: Tampa, Richmond, San Antonio, Palo Alto, and Minneapolis.

PARTICIPANTS

Partners included national and site program leaders, clinicians, Department of Defense Referral Representatives, and researchers. Participants included program staff ( n  = 46) and Service Members/Veterans ( n  = 48).

DESIGN

This paper represents a component of a larger participatory-based concurrent mixed methods quality improvement project.

MAIN MEASURES

Participant scripts and demographic surveys.

METHODS

Datasets were analyzed using rapid iterative content analysis; IETP model was iteratively revised with partner feedback. Each site had an IETP clinical team member participate. The IRLM was contextualized within the Consolidated Framework for Implementation Research (CFIR); systematic consensus building expert reviewed implementation strategies; RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance); and Implementation Outcomes Framework (IOF).

RESULTS

Analyses and partner feedback identified key characteristics, determinants, implementation strategies, mechanisms, and outcomes.

CONCLUSIONS

This partner-engaged IRLM informs implementation and sustainment of a rehabilitation program for individuals with TBI. Findings will be leveraged to examine implementation, standardize core outcome measurements, and inform knowledge translation.

Identifying Barriers and Implementation Strategies to Inform TBI Screening Adoption in Behavioral Healthcare Settings

OBJECTIVE

Identify barriers to the adoption of the Ohio State University Traumatic Brain Injury Identification Method (OSU TBI-ID) in behavioral healthcare organizations and match these barriers to implementation strategies to inform future implementation efforts.

PARTICIPANTS

Twenty behavioral health providers.

DESIGN

Qualitative study involving individual, semi-structured interviews regarding barriers to adopting the OSU TBI-ID. Data were thematically analyzed around constructs from the Consolidated Framework for Implementation Research (CFIR). Results were matched to strategies using the Expert Recommendations for Implementing Change (CFIR/ERIC) matching tool.

RESULTS

Ten barriers were identified across 4 CFIR domains. Inner-Setting barriers were inadequate leadership engagement, priorities, resources, and organizational incentives. Individual-Characteristics barriers were insufficient knowledge about the connection between TBI and behavioral health and how to conduct the OSU TBI-ID with fidelity, low self-efficacy to conduct screening, and inadequate motivation and buy-in to conduct screening. Outer-Setting barriers were lack of state-level mandates and inadequate incentives to conduct screenings. The Process domain barrier was an insufficient engagement of key personnel. Strategy recommendations include: identify and prepare champions; alter incentive and allowance structures; inform local opinion leaders; build a coalition; access new funding; conduct local consensus discussions; involve executive boards; capture/share local knowledge; conduct educational meetings; assess for readiness and identify determinants; identify early adopters; fund and contract for the clinical innovation; create a learning collaborative; and conduct a local needs assessment.

CONCLUSIONS

This is the first study to examine barriers to adopting the OSU TBI-ID in real-world practice settings. Our results suggest that multilevel implementation strategies addressing mechanisms of change are necessary at the provider, organizational, and systems levels to overcome the complex barriers affecting TBI screening adoption and implementation. Future research is needed to test these strategies and their mechanisms of action on the adoption, implementation, and sustainment of TBI screening, as well as their effect on client-level outcomes.

Implementing learning into practice from continuous professional development activities: a scoping review of health professionals' views and experiences

PURPOSE

Continuing professional development (CPD) is an approach for health professionals to preserve and expand their knowledge, skills, and performance, and can contribute to improving delivery of care. However, evidence indicates that simply delivering CPD activities to health professionals does not lead to a change in practice. This review aimed to collate, summarize, and categorize the literature that reported the views and experiences of health professionals on implementing into practice their learning from CPD activities.

METHODS

This review was guided by the Joanna Briggs Institute Reviewers' Manual methodology for scoping reviews. Three databases, PubMed, Embase and Cumulative Index to Nursing and Allied Health Literature (CINAHL), were systematically searched in February 2023 for articles published since inception. Two independent reviewers screened the articles against the inclusion criteria, and completed the data extraction. Data were summarized quantitatively, and the findings relating to views and experiences were categorized into challenges and facilitators.

RESULTS

Thirteen articles were included. Implementation of learning was not the primary focus in the majority of studies. Studies were published between 2008-2022; the majority were conducted in North America and nurses were the most common stakeholder group among Healthcare Professionals (HCPs). Five studies adopted qualitative methods, four quantitative studies, and four mixed-methods studies. The reported barriers of implementation included lack of time and human resource; the facilitators included the nature of the training, course content and opportunity for communal learning.

CONCLUSION

This review highlights a gap in the literature. Available studies indicate some barriers for health professionals to implement their learning from CPD activities into their practice. Further studies, underpinned with appropriate theory and including all relevent stakeholders are required to investigate strategies that may facilitate the integration of learning from CPD into routine practice.

A research agenda to advance the study of implementation mechanisms

BACKGROUND

Implementation science scholars have made significant progress identifying factors that enable or obstruct the implementation of evidence-based interventions, and testing strategies that may modify those factors. However, little research sheds light on how or why strategies work, in what contexts, and for whom. Studying implementation mechanisms-the processes responsible for change-is crucial for advancing the field of implementation science and enhancing its value in facilitating equitable policy and practice change. The Agency for Healthcare Research and Quality funded a conference series to achieve two aims: (1) develop a research agenda on implementation mechanisms, and (2) actively disseminate the research agenda to research, policy, and practice audiences. This article presents the resulting research agenda, including priorities and actions to encourage its execution.

METHOD

Building on prior concept mapping work, in a semi-structured, 3-day, in-person working meeting, 23 US-based researchers used a modified nominal group process to generate priorities and actions for addressing challenges to studying implementation mechanisms. During each of the three 120-min sessions, small groups responded to the prompt: "What actions need to be taken to move this research forward?" The groups brainstormed actions, which were then shared with the full group and discussed with the support of facilitators trained in structured group processes. Facilitators grouped critical and novel ideas into themes. Attendees voted on six themes they prioritized to discuss in a fourth, 120-min session, during which small groups operationalized prioritized actions. Subsequently, all ideas were collated, combined, and revised for clarity by a subset of the authorship team.

RESULTS

From this multistep process, 150 actions emerged across 10 priority areas, which together constitute the research agenda. Actions included discrete activities, projects, or products, and ways to shift how research is conducted to strengthen the study of implementation mechanisms.

CONCLUSIONS

This research agenda elevates actions to guide the selection, design, and evaluation of implementation mechanisms. By delineating recommended actions to address the challenges of studying implementation mechanisms, this research agenda facilitates expanding the field of implementation science, beyond studying what works to how and why strategies work, in what contexts, for whom, and with which interventions.

Universal substance use care for adolescents with chronic medical conditions: a protocol to examine equitable implementation determinants and strategies for SBIRT at a pediatric hospital

BACKGROUND

Adolescents with chronic medical conditions (CMC) use alcohol and marijuana at levels equal to or even greater than their peers without CMC and are more likely to initiate substance use at 14 years or younger. Approximately 33% of adolescents with CMC binge drink alcohol and 20% use marijuana. When using substances, adolescents with CMC are at elevated risk for problem use and adverse consequences given their medical conditions. Although there has recently been progress integrating substance use services into adult hospitals, there has been almost no implementation of standardized substance use services into pediatric hospitals for adolescents with CMC. Screening, Brief Intervention, and Referral to Treatment (SBIRT) for adolescents is an evidence-based, public health approach to promote the early detection and intervention of risky alcohol use in high-risk youth. This paper describes a study protocol combining two leading implementation science frameworks, the Consolidated Framework for Implementation Research (CFIR) and the Health Equity Implementation framework (HEIF), to engage pediatric hospital partners (hospital staff and clinicians, patients with CMC, and caregivers) to identify and specify contextual determinants of SBIRT implementation, which can be used to derive implementation strategies to optimize SBIRT adoption, reach, and fidelity.

METHOD

This study will use semi-structured interviews and focus groups with pediatric hospital partners (e.g., hospital staff and clinicians, adolescent patients, and caregivers) to identify SBIRT implementation determinants, using semi-structured interview and focus group guides that integrate CFIR and HEIF dimensions.

DISCUSSION

Understanding implementation determinants is one of the first steps in the implementation science process. The use of two determinant frameworks highlighting a comprehensive set of determinants including health equity and justice will enable identification of barriers and facilitators that will then map on to strategies that address these factors. This study will serve as an essential precursor to further work evaluating the feasibility of and the degree of engagement with SBIRT among this vulnerable pediatric population.

Barriers and facilitators of implementation of evidence-based interventions in children and young people's mental health care - a systematic review

BACKGROUND

Effective evidence-based practices (EBP) for children and young people's (CYP) mental health exist, however, there is low uptake in clinical practice and interventions do not always reach those in need. This review aimed to comprehensively identify and synthesise the barriers and facilitators to implementing EBP in CYP mental health care, mapped according to an implementation framework in order to make pragmatic recommendations for practitioners, commissioners and researchers.

METHODS

Following the PRISMA guidelines, an electronic search of PsycINFO, MEDLINE, CINAHL and Embase in 2021 yielded 1830 results. In total, 107 abstracts were screened, 57 of which were included for full-text review and 26 were included for data extraction and analysis.

RESULTS

We identified a number of organisational and clinician-level barriers which impede the implementation of evidence in 'real world' practice. Barriers included lack of access to funding; poor access to resources; clinician attitudes and flexibility of EBP. Facilitators included targeted funding and access to resources; supportive staff and leadership committed to innovation and skills in the EBP.

CONCLUSIONS

Although the process of translating research into practice is challenging and the solutions are not straightforward, we have identified a set of practical recommendations for ways in which implementation practices can improve. Researchers, commissioners, funders and practitioners can work together to improve the implementation of EBP in CYP mental health settings by ensuring funding is available, prioritise implementation beyond the end of trials, upskill staff on integrating EBP into care, ensure EBPs are flexible and are co-produced with service users.

REGISTRATION

This review was registered on PROSPERO international prospective register of systematic reviews (CRD42021252995).

Community-Engaged Implementation Strategies in Occupational Therapy: A Scoping Review

IMPORTANCE

 Despite the potential of community-engaged implementation research (CEIR) in developing strategies to accelerate the translation of evidence-based interventions (EBIs), there is a noticeable knowledge gap in the current state of CEIR in occupational therapy. A synthesis of the concept, purpose, and operationalization of CEIR is necessary.

OBJECTIVE

 To identify the contexts, purposes, and operationalization of CEIR, focusing on implementation strategies in occupational therapy.

DATA SOURCES

 PubMed/MEDLINE, Embase, CINAHL, Scopus, and Web of Science.

STUDY SELECTION AND DATA COLLECTION

 We included studies that were explicit and intentional about CEIR and that focused on implementation strategies to support the translation of occupational therapy interventions, clinical guidelines, practice models, theories, or assessments. We extracted the research context (e.g., partners, recruitment), purpose (e.g., why community-engaged research was used), and operationalization (e.g., community engagement [CE] activities, how their findings inform the research) using thematic analysis.

FINDINGS

 Of 3,219 records, 6 studies were included. Involved partners were mainly occupational therapy practitioners from existing networks. CEIR that focuses on implementation strategies informs various aspects of research design, ranging from study design to sustainability, by developing community-academia partnerships, building implementation capacity, and creating implementation strategies across diverse research areas. Current research has used various but mostly traditional CE activities (e.g., focus groups).

CONCLUSIONS AND RELEVANCE

 We synthesized evidence on CEIR focused on implementation strategies in occupational therapy. Intentional efforts are needed to collaborate with diverse partners, explore innovative CE activities, produce equitable outputs, and develop multilevel implementation strategies to accelerate the translation of EBIs into practice. Plain-Language Summary: In this review, we synthesize evidence on the contexts, purposes, and operationalization of community-engaged implementation research (CEIR), focusing on implementation strategies in occupational therapy research. We found that current implementation efforts mainly rely on occupational therapy practitioners as community partners and use traditional recruitment methods and community engagement activities. In turn, they develop implementation strategies that mainly target practitioners without comprehensive, multilevel implementation support. We suggest more equitable collaboration with diverse partners to effectively promote the implementation and dissemination of evidence-based interventions in occupational therapy practice.

Specifying behavioural and strategy components of de-implementation efforts targeting low-value prescribing practices in secondary health care

BACKGROUND

/Aims De-implementation, including the removal or reduction of unnecessary or inappropriate prescribing, is crucial to ensure patients receive appropriate evidence-based health care. The utilization of de-implementation efforts is contingent on the quality of strategy reporting. To further understand effective ways to de-implement medical practices, specification of behavioural targets and components of de-implementation strategies are required. This paper aims to critically analyse how well the behavioural targets and strategy components, in studies that focused on de-implementing unnecessary or inappropriate prescribing in secondary healthcare settings, were reported.

METHODS

A supplementary analysis of studies included in a recently published review of de-implementation studies was conducted. Article text was coded verbatim to two established specification frameworks. Behavioural components were coded deductively to the five elements of the Action, Actor, Context, Target, Time (AACTT) framework. Strategy components were mapped to the nine elements of the Proctor's 'measuring implementation strategies' framework.

RESULTS

The behavioural components of low-value prescribing, as coded to the AACTT framework, were generally specified well. However, the Actor and Time components were often vague or not well reported. Specification of strategy components, as coded to the Proctor framework, were less well reported. Proctor's Actor, Action target: specifying targets, Dose and Justification elements were not well reported or varied in the amount of detail offered. We also offer suggestions of additional specifications to make, such as the 'interactions' participants have with a strategy.

CONCLUSION

Specification of behavioural targets and components of de-implementation strategies for prescribing practices can be accommodated by the AACTT and Proctor frameworks when used in conjunction. These essential details are required to understand, replicate and successfully de-implement unnecessary or inappropriate prescribing. In general, standardisation in the reporting quality of these components is required to replicate any de-implementation efforts.

TRIAL REGISTRATION

Not registered.

What works in implementing shared medical appointments for patients with diabetes in primary care to enhance reach: a qualitative comparative analysis from the Invested in Diabetes study

BACKGROUND

Diabetes is a serious public health problem affecting 37.3 million Americans. Diabetes shared medical appointments (SMAs) are an effective strategy for providing diabetes self-management support and education in primary care. However, practices delivering SMAs experience implementation challenges. This analysis examined conditions associated with successful practice implementation of diabetes SMAs in the context of participation in a pragmatic trial.

METHODS

Mixed methods study using qualitative and quantitative data collected from interviews, observations, surveys, and practice-reported data, guided by the practical, robust implementation and sustainability model (PRISM). Data were analyzed using qualitative comparative analysis (QCA). Successful implementation was defined as meeting patient recruitment targets (Reach) during the study period. Participants were clinicians and staff members from 22 primary care practices in Colorado and Missouri, USA.

RESULTS

The first necessary condition identified from the QCA was the presence of additional resources for patients with diabetes in the practice. Within practices that had these additional resources, we found that a sufficiency condition was the presence of an effective key person to make things happen with the SMAs. A second QCA was conducted to determine conditions underlying the presence of the effective key person (often performing functions of an implementation champion), which revealed factors including low or managed employee turnover, a strong baseline practice culture, and previous experience delivering SMAs.

CONCLUSIONS

Identification of key factors necessary and sufficient for implementation of new care processes is important to enhance patient access to evidence-based interventions. This study suggests that practice features and resources have important implications for implementation of diabetes SMAs. There may be opportunities to support practices with SMA implementation by enabling the presence of skilled implementation champions.

TRIAL REGISTRATION

Registered at clinicaltrials.gov under trial ID NCT03590041, registered on July 18, 2018.

Using Causative Methods to Determine System-Level Factors Driving the Uptake and Use of Evidence-Based Practices in a Public Early Intervention System

Part C Early Intervention (EI) systems are an entry point to services for autistic toddlers and can be leveraged to facilitate access to autism evidence-based practices (EBPs). However, EI systems are complex and limited research has examined how an EI system's infrastructure (i.e. system-level factors) impacts the adoption and implementation of EBPs. To address this gap, 36 EI providers and 9 EI administrators completed a semi-structured interview or focus group about factors impacting the implementation of autism EBPs. Qualitative analysis included a combination of grounded theory and causative coding. Analyses were refined by input from providers, administrators, and family stakeholders in the form of round tables and presentations at the state's interagency coordinating council. Primary themes centered on: (1) the costs associated with independent contracting structures; (2) operational demands; (3) workforce stability; (4) communication consistency; and (5) implementation supports for EBP implementation. Causative coding helped to demonstrate the perceived relationships between these factors and underscored the important role of incentivization structures, collaboration opportunities, and championing in supporting the use of EBPs within a system that primarily uses independent contracting structures. The current study extends previous research by demonstrating how several system-level factors are perceived to play a role in the adoption and implementation of EBPs by independently contracted EI providers. These findings underscore the need for implementation strategies, such as incentivization strategies and social network building, to increase providers' implementation of autism EBPs within EI systems.

Mapping implementation strategies to reach community-dwelling older adults in Northwest Switzerland

BACKGROUND

In Northwestern Switzerland, recent legislation tackles the needs of community-dwelling older adults by creating Information and Advice Centers (IACs). IACs are a new service in the community that aims to assess the needs and provide information on age-related issues to community-dwelling older adults and their families. Previous studies reported difficulties in reaching community-dwelling older adults for community-based programs. We aimed to: 1) systematically identify implementation strategies to promote the IAC among community care providers, older adults and informal caregivers; 2) monitor the delivery of these strategies by the IAC management; and 3) describe the impact of those strategies on reach of community-dwelling older adults. This study was conducted as part of the TRANS-SENIOR project.

METHODS

As part of the INSPIRE feasibility assessment, we conducted a pre-test post-test study between March and September 2022. The sample included 8,840 older adults aged 65 + visiting/calling or being referred to the IAC for the first time. Implementation strategies were selected using implementation mapping and organized in bundles for each group of community care providers and older adults/caregivers. Our evaluation included: estimation of fidelity to the delivery of implementation strategies and bundles by the IAC management and their coverage; referral source of older adults to the IAC; and impact of the strategies on reach of the IAC on the 65 + population living in the care region. Adaptations to the strategies were documented using the FRAME-IS. Descriptive statistics were calculated and reported.

RESULTS

Seven implementation strategies were selected and organized in bundles for each community care provider and older adults and their caregivers. The lowest fidelity score was found in implementation strategies selected for nursing homes whereas the highest score corresponded to strategies targeting older adults and caregivers. "Informational visits" was the strategy with the lowest coverage (2.5% for nursing homes and 10.5% for hospitals and specialized clinics). The main referral sources were self-referrals and referrals by caregivers, followed by nursing homes. The IAC reach among the 65 + population was 5.4%.

CONCLUSION

We demonstrated the use of implementation mapping to select implementation strategies to reach community-dwelling older adults. The reach was low suggesting that higher fidelity to the delivery of the strategies, and reflection on the causal pathway of the implementation strategies might be needed.

Health-care organization characteristics in cancer care delivery: an integrated conceptual framework with content validation

Context can influence cancer-related outcomes. For example, health-care organization characteristics, including ownership, leadership, and culture, can affect care access, communication, and patient outcomes. Health-care organization characteristics and other contextual factors can also influence whether and how clinical discoveries reduce cancer incidence, morbidity, and mortality. Importantly, policy, market, and technology changes are transforming health-care organization design, culture, and operations across the cancer continuum. Consequently, research is essential to examine when, for whom, and how organizational characteristics influence person-level, organization-level, and population-level cancer outcomes. Understanding organizational characteristics-the structures, processes, and other features of entities involved in health care delivery-and their dynamics is an important yet understudied area of care delivery research across the cancer continuum. Research incorporating organizational characteristics is critical to address health inequities, test care delivery models, adapt interventions, and strengthen implementation. The field lacks conceptual grounding, however, to help researchers identify germane organizational characteristics. We propose a framework identifying organizational characteristics relevant for cancer care delivery research based on conceptual work in health services, organizational behavior, and management science and refined using a systematic review and key informant input. The proposed framework is a tool for organizing existing research and enhancing future cancer care delivery research. Following a 2012 Journal of the National Cancer Institute monograph, this work complements National Cancer Institute efforts to stimulate research addressing the relationship between cancer outcomes and contextual factors at the patient, provider, team, delivery organization, community, and health policy levels.

Strengthening the Science and Practice of Implementation Support: Evaluating the Effectiveness of Training and Technical Assistance Centers

Hundreds of millions of dollars are spent each year by U.S. federal agencies for training and technical assistance (TTA) to be delivered by training and technical assistance centers (TTACs) to "delivery system organizations" (e.g., federally qualified health centers, state departments of health, substance abuse treatment centers, schools, and healthcare organizations). TTACs are often requested to help delivery system organizations implement evidence-based interventions. Yet, counterintuitively, TTACs are rarely required to use evidence-based approaches when supporting delivery systems (in the use of evidence-based programs). In fact, evaluations of TTAC activities tend to be minimal; evaluation of technical assistance (if conducted at all) often emphasizes outputs (number of encounters), satisfaction, and self-reports of knowledge gained-more substantive outcomes are not evaluated. The gap between (a) the volume of TTA services being funded and provided and (b) the evaluation of those services is immense and has the potential to be costly. The basic question to be answered is: how effective are TTA services? This article introduces the special issue on Strengthening the Science and Practice of Implementation Support: Evaluating the Effectiveness of Training and Technical Assistance Centers. The special issue promotes 1) knowledge of the state of the art of evaluation of TTACs and 2) advances in what to evaluate in TTA. A major goal of the issue is to improve the science and practice of implementation support, particularly in the areas of TTA.

Sustainability of an outcome measure in outpatient stroke rehabilitation: A realist evaluation

RATIONALE

Only half of newly implemented evidence-based practices are sustained. Though poor sustainment can lead to negative consequences for clinical teams, organizations and patients, the causal explanations of sustainment are largely unknown.

AIMS AND OBJECTIVES

We aimed to ascertain how (mechanisms) and in what circumstances (context) a newly implemented outcome measure in rehabilitation was sustained or not (outcome).

METHODOLOGY

Informed by an integrated knowledge translation approach, we conducted a realist evaluation using a mixed method, embedded single case study design with data collection up to 18 months following the implementation of the Mayo-Portland Adaptability Inventory - version 4 (MPAI-4), a rehabilitation outcome measure. Quantitative data (survey and patient charts) was analysed using descriptive statistics, then integrated with qualitative data (interviews with 10 key informants) and analysed using inductive and deductive retroduction. We integrated the data to develop a case description and ultimately, to refine the programme theory to better understand the sustainability of the MPAI-4.

RESULTS

We linked context, mechanisms and outcomes, and also emphasized sustainability strategies in 18 explanations of how sustainability works. These explanations provide evidence for four overarching patterns: (1) implementation and sustainability phases are interconnected, (2) outcomes build on each other recursively, with patient benefits as the keystone outcome, (3) sustainment is achieved to varying levels across different sustainability outcomes [e.g., high level (e.g., rate of MPAI-4 scoring: 77.7%) to low level (e.g., rate of MPAI-4 application to clinical decision-making: 3.7%)] and (4) the work of sustaining the MPAI-4 is shared amongst different stakeholders.

CONCLUSION

Implementation teams can draw from this programme theory to improve the sustainment of outcomes measures while researchers could continue to refine the theory. Continued investigation of sustainability, including diverse and continuous sustainability outcomes, is needed to understand how to maintain improvements in quality of care and patient outcomes.

How do oilcloth sessions work? A realist evaluation approach to exploring ripple effects in an implementation strategy

PURPOSE

To explore the mechanisms of the implementation strategy, "oilcloth sessions" and understand and explain the ripple effects of oilcloth sessions as a strategy to implement a new emergency department.

DESIGN/METHODOLOGY/APPROACH

A qualitative design was used whereby data were collected using field notes from an ethnographic study of the oilcloth sessions and follow-up semi-structured interviews with staff, managers and key employees who participated in the oilcloth sessions. The data analysis was inspired by the realist evaluation approach of generative causality proposed by Pawson and Tilley.

FINDINGS

The primary ripple effect was that the oilcloth sessions were used for different purposes than the proposed program theory, including being used as: (1) a stage, (2) a battlefield, (3) a space for imagination and (4) a strategic management tool influencing the implementation outcomes. The results bring essential knowledge that may help to explain why and how a well-defined implementation strategy has unplanned outcomes.

ORIGINALITY/VALUE

Unintended outcomes of implementation strategies are an underexplored issue. This study may help implementation researchers rethink the activities required to reduce unintended negative outcomes or explore potential unplanned outcomes and, in this way, hinder or enhance outcomes, effectiveness and sustainability. Future studies within implementation research should incorporate attention to unintended outcomes to fully understand the impact of implementation strategies.

Allied Health Clinicians' Uptake of Evidence in Stroke Rehabilitation: A Systematic Review of Determinants Targeted in Implementation Studies

OBJECTIVE

This study aimed to determine whether targeting specific implementation determinants is associated with allied health clinicians' uptake of evidence when implemented within stroke rehabilitation settings.

DATA SOURCES

7 key databases were searched to identify articles published between 1990 and 2022 for inclusion. Reference lists of relevant articles were hand searched.

STUDY SELECTION

Studies were independently screened by 2 authors and included if the implementation intervention targeted allied health clinical practice in any stroke rehabilitation context and reported at least 1 quantitative measure of evidence uptake. Thirty studies met inclusion criteria.

DATA EXTRACTION

Data were independently extracted by the first and second author. Implementation outcomes for each study were categorized as either mostly successful, partially successful, or not successful based on the degree of evidence uptake achieved. Determinants targeted were categorized using the Consolidated Framework for Implementation Research (CFIR).

DATA SYNTHESIS

Patterns between the degree of evidence uptake and determinants targeted across studies were analyzed by the first and second authors in 1 of 3 groups: A (pre-post statistical analysis), B (pre-post descriptive analysis), or C (post-only descriptive analysis). Patterns between evidence uptake and determinants targeted were first identified within groups A and B, with group C consulted to support findings. All studies categorized as "mostly successful" targeted facilitation in combination with establishing face-to-face networks and communication strategies. Conversely, no studies rated "not successful" targeted either of these determinants. Studies rated "partially successful" targeted either 1, but seldom both, of these determinants.

CONCLUSIONS

This review has provided descriptive evidence of determinants which may be important to target for allied health clinicians' uptake of evidence within stroke rehabilitation settings.

Bridges and Mechanisms: Integrating Systems Science Thinking into Implementation Research

We present a detailed argument for how to integrate, or bridge, systems science thinking and methods with implementation science. We start by showing how fundamental systems science principles of structure, dynamics, information, and utility are relevant for implementation science. Then we examine the need for implementation science to develop and apply richer theories of complex systems. This can be accomplished by emphasizing a causal mechanisms approach. Identifying causal mechanisms focuses on the "cogs and gears" of public health, clinical, and organizational interventions. A mechanisms approach focuses on how a specific strategy will produce the implementation outcome. We show how connecting systems science to implementation science opens new opportunities for examining and addressing social determinants of health and conducting equitable and ethical implementation research. Finally, we present case studies illustrating successful applications of systems science within implementation science in community health policy, tobacco control, health care access, and breast cancer screening.

Implementation Science and Practice-Oriented Research: Convergence and Complementarity

Implementation science is the scientific study of methods to promote the uptake of research findings and other evidence-based practices in routine care, with the goal of improving the quality and effectiveness of health services (Bauer et al., 2015). In addition to this common goal, practice-oriented psychotherapy research (and researchers) and implementation science (and scientists) share a common focus on the people and the places where treatment happens. Thus, there exists strong potential for combining these two approaches. In this article, we provide a primer on implementation science for psychotherapy researchers and highlight important areas and examples of convergence and complementarity between implementation science and practice-oriented psychotherapy research. Specifically, we (a) define and describe the core features of implementation science; (b) discuss similarities and areas of complementarity between implementation science and practice-oriented psychotherapy research; (c) discuss a case example that exemplifies the integration of implementation science and practice-oriented research; and (d) propose directions for future research and collaborations that leverage both implementation science and practice-oriented research.

Implementation of Health IT for Cancer Screening in US Primary Care: Scoping Review

BACKGROUND

A substantial percentage of the US population is not up to date on guideline-recommended cancer screenings. Identifying interventions that effectively improve screening rates would enhance the delivery of such screening. Interventions involving health IT (HIT) show promise, but much remains unknown about how HIT is optimized to support cancer screening in primary care.

OBJECTIVE

This scoping review aims to identify (1) HIT-based interventions that effectively support guideline concordance in breast, cervical, and colorectal cancer screening provision and follow-up in the primary care setting and (2) barriers or facilitators to the implementation of effective HIT in this setting.

METHODS

Following scoping review guidelines, we searched MEDLINE, CINAHL Plus, Web of Science, and IEEE Xplore databases for US-based studies from 2015 to 2021 that featured HIT targeting breast, colorectal, and cervical cancer screening in primary care. Studies were dual screened using a review criteria checklist. Data extraction was guided by the following implementation science frameworks: the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework; the Expert Recommendations for Implementing Change taxonomy; and implementation strategy reporting domains. It was also guided by the Integrated Technology Implementation Model that incorporates theories of both implementation science and technology adoption. Reporting was guided by PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews).

RESULTS

A total of 101 studies met the inclusion criteria. Most studies (85/101, 84.2%) involved electronic health record-based HIT interventions. The most common HIT function was clinical decision support, primarily used for panel management or at the point of care. Most studies related to HIT targeting colorectal cancer screening (83/101, 82.2%), followed by studies related to breast cancer screening (28/101, 27.7%), and cervical cancer screening (19/101, 18.8%). Improvements in cancer screening were associated with HIT-based interventions in most studies (36/54, 67% of colorectal cancer-relevant studies; 9/14, 64% of breast cancer-relevant studies; and 7/10, 70% of cervical cancer-relevant studies). Most studies (79/101, 78.2%) reported on the reach of certain interventions, while 17.8% (18/101) of the included studies reported on the adoption or maintenance. Reported barriers and facilitators to HIT adoption primarily related to inner context factors of primary care settings (eg, staffing and organizational policies that support or hinder HIT adoption). Implementation strategies for HIT adoption were reported in 23.8% (24/101) of the included studies.

CONCLUSIONS

There are substantial evidence gaps regarding the effectiveness of HIT-based interventions, especially those targeting guideline-concordant breast and colorectal cancer screening in primary care. Even less is known about how to enhance the adoption of technologies that have been proven effective in supporting breast, colorectal, or cervical cancer screening. Research is needed to ensure that the potential benefits of effective HIT-based interventions equitably reach diverse primary care populations.

Family cascade screening for equitable identification of familial hypercholesterolemia: study protocol for a hybrid effectiveness-implementation type III randomized controlled trial

BACKGROUND

Familial hypercholesterolemia (FH) is a heritable disorder affecting 1.3 million individuals in the USA. Eighty percent of people with FH are undiagnosed, particularly minoritized populations including Black or African American people, Asian or Asian American people, and women across racial groups. Family cascade screening is an evidence-based practice that can increase diagnosis and improve health outcomes but is rarely implemented in routine practice, representing an important care gap. In pilot work, we leveraged best practices from behavioral economics and implementation science-including mixed-methods contextual inquiry with clinicians, patients, and health system constituents-to co-design two patient-facing implementation strategies to address this care gap: (a) an automated health system-mediated strategy and (b) a nonprofit foundation-mediated strategy with contact from a foundation-employed care navigator. This trial will test the comparative effectiveness of these strategies on completion of cascade screening for relatives of individuals with FH, centering equitable reach.

METHODS

We will conduct a hybrid effectiveness-implementation type III randomized controlled trial testing the comparative effectiveness of two strategies for implementing cascade screening with 220 individuals with FH (i.e., probands) per arm identified from a large northeastern health system. The primary implementation outcome is reach, or the proportion of probands with at least one first-degree biological relative (parent, sibling, child) in the USA who is screened for FH through the study. Our secondary implementation outcomes include the number of relatives screened and the number of relatives meeting the American Heart Association criteria for FH. Our secondary clinical effectiveness outcome is post-trial proband cholesterol level. We will also use mixed methods to identify implementation strategy mechanisms for implementation strategy effectiveness while centering equity.

DISCUSSION

We will test two patient-facing implementation strategies harnessing insights from behavioral economics that were developed collaboratively with constituents. This trial will improve our understanding of how to implement evidence-based cascade screening for FH, which implementation strategies work, for whom, and why. Learnings from this trial can be used to equitably scale cascade screening programs for FH nationally and inform cascade screening implementation efforts for other genetic disorders.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT05750667. Registered 15 February 2023-retrospectively registered, https://clinicaltrials.gov/study/NCT05750667 .

Protocol for the process evaluation of an intervention to improve antenatal smoking cessation support (MOHMQuit) in maternity services in New South Wales, Australia

INTRODUCTION

Smoking cessation in pregnancy remains a public health priority. Our team used the Behaviour Change Wheel to develop the Midwives and Obstetricians Helping Mothers to Quit smoking (MOHMQuit) intervention with health system, leader (including managers and educators) and clinician components. MOHMQuit addresses a critical evidence to practice gap in the provision of smoking cessation support in antenatal care. It involves nine maternity services in New South Wales in a cluster randomised stepped-wedge controlled trial of effectiveness. This paper describes the design and rationale for the process evaluation of MOHMQuit. The process evaluation aims to assess to what extent and how MOHMQuit is being implemented (acceptability; adoption/uptake; appropriateness; feasibility; fidelity; penetration and sustainability), and the context in which it is implemented, in order to support further refinement of MOHMQuit throughout the trial, and aid understanding and interpretation of the results of the trial.

METHODS AND ANALYSIS

The process evaluation is an integral part of the stepped-wedge trial. Its design is underpinned by implementation science frameworks and adopts a mixed methods approach. Quantitative evidence from participating leaders and clinicians in our study will be used to produce individual and site-level descriptive statistics. Qualitative evidence of leaders' perceptions about the implementation will be collected using semistructured interviews and will be analysed descriptively within-site and thematically across the dataset. The process evaluation will also use publicly available data and observations from the research team implementing MOHMQuit, for example, training logs. These data will be synthesised to provide site-level as well as individual-level implementation outcomes.

ETHICS AND DISSEMINATION

The study received ethical approval from the Population Health Services Research Ethics Committee for NSW, Australia (Reference 2021/ETH00887). Results will be communicated via the study's steering committee and will also be published in peer-reviewed journals and presented at conferences.

TRIAL REGISTRATION NUMBER

Australian New Zealand Trials Registry ACTRN12622000167763. https://www.australianclinicaltrials.gov.au/anzctr/trial/ACTRN12622000167763.

Protocol for a hybrid type 3 effectiveness-implementation trial of a pragmatic individual-level implementation strategy for supporting school-based prevention programming

BACKGROUND

For approximately one in five children who have social, emotional, and behavioral (SEB) challenges, accessible evidence-based prevention practices (EBPPs) are critical. In the USA, schools are the primary setting for children's SEB service delivery. Still, EBPPs are rarely adopted and implemented by front-line educators (e.g., teachers) with sufficient fidelity to see effects. Given that individual behavior change is ultimately required for successful implementation, focusing on individual-level processes holds promise as a parsimonious approach to enhance impact. Beliefs and Attitudes for Successful Implementation in Schools for Teachers (BASIS-T) is a pragmatic, multifaceted pre-implementation strategy targeting volitional and motivational mechanisms of educators' behavior change to enhance implementation and student SEB outcomes. This study protocol describes a hybrid type 3 effectiveness-implementation trial designed to evaluate the main effects, mediators, and moderators of the BASIS-T implementation strategy as applied to Positive Greetings at the Door, a universal school-based EBPP previously demonstrated to reduce student disruptive behavior and increase academic engagement.

METHODS

This project uses a blocked randomized cohort design with an active comparison control (ACC) condition. We will recruit and include approximately 276 teachers from 46 schools randomly assigned to BASIS-T or ACC conditions. Aim 1 will evaluate the main effects of BASIS-T on proximal implementation mechanisms (attitudes, subjective norms, self-efficacy, intentions to implement, and maintenance self-efficacy), implementation outcomes (adoption, reach, fidelity, and sustainment), and child outcomes (SEB, attendance, discipline, achievement). Aim 2 will examine how, for whom, under what conditions, and how efficiently BASIS-T works, specifically by testing whether the effects of BASIS-T on child outcomes are (a) mediated via its putative mechanisms of behavior change, (b) moderated by teacher factors or school contextual factors, and (c) cost-effective.

DISCUSSION

This study will provide a rigorous test of BASIS-T-a pragmatic, theory-driven, and generalizable implementation strategy designed to target theoretically-derived motivational mechanisms-to increase the yield of standard EBPP training and support strategies.

TRIAL REGISTRATION

ClinicalTrials.gov ID: NCT05989568. Registered on May 30, 2023.

Instrumental variables for implementation science: exploring context-dependent causal pathways between implementation strategies and evidence-based interventions

BACKGROUND

The impact of both implementation strategies (IS) and evidence-based interventions (EBI) can vary across contexts, and a better understanding of how and why this occurs presents fundamental but challenging questions that implementation science as a field will need to grapple with. We use causal epidemiologic methods to explore the mechanisms of why sharp distinctions between implementation strategies (IS) and efficacy of an evidence-based intervention (EBI) may fail to recognize that the effect of an EBI can be deeply intertwined and dependent on the context of the IS leading to its uptake.

METHODS

We explore the use of instrumental variable (IV) analyses as a critical tool for implementation science methods to isolate three relevant quantities within the same intervention context when exposure to an implementation strategy is random: (1) the effect of an IS on implementation outcomes (e.g., uptake), (2) effect of EBI uptake on patient outcomes, and (3) overall effectiveness of the IS (i.e., ~ implementation*efficacy). We discuss the mechanisms by which an implementation strategy can alter the context, and therefore effect, of an EBI using the underlying IV assumptions. We illustrate these concepts using examples of the implementation of new ART initiation guidelines in Zambia and community-based masking programs in Bangladesh.

RESULTS

Causal questions relevant to implementation science are answered at each stage of an IV analysis. The first stage assesses the effect of the IS (e.g., new guidelines) on EBI uptake (e.g., same-day treatment initiation). The second stage leverages the IS as an IV to estimate the complier average causal effect (CACE) of the EBI on patient outcomes (e.g., effect of same-day treatment initiation on viral suppression). The underlying assumptions of CACE formalize that the causal effect of EBI may differ in the context of a different IS because (1) the mechanisms by which individuals uptake an intervention may differ and (2) the subgroup of individuals who take up an EBI may differ. IV methods thus provide a conceptual framework for how IS and EBIs are linked and that the IS itself needs to be considered a critical contextual determinant. Moreover, it also provides rigorous methodologic tools to isolate the effect of an IS, EBI, and combined effect of the IS and EBI.

DISCUSSION

Leveraging IV methods when exposure to an implementation strategy is random helps to conceptualize the context-dependent nature of implementation strategies, EBIs, and patient outcomes. IV methods formalize that the causal effect of an EBI may be specific to the context of the implementation strategy used to promote uptake. This integration of implementation science concepts and theory with rigorous causal epidemiologic methods yields novel insights and provides important tools for exploring the next generation of questions related to mechanisms and context in implementation science.

Audit and feedback to reduce unwarranted clinical variation at scale: a realist study of implementation strategy mechanisms

BACKGROUND

Unwarranted clinical variation in hospital care includes the underuse, overuse, or misuse of services. Audit and feedback is a common strategy to reduce unwarranted variation, but its effectiveness varies widely across contexts. We aimed to identify implementation strategies, mechanisms, and contextual circumstances contributing to the impact of audit and feedback on unwarranted clinical variation.

METHODS

Realist study examining a state-wide value-based healthcare program implemented between 2017 and 2021 in New South Wales, Australia. Three initiatives within the program included audit and feedback to reduce unwarranted variation in inpatient care for different conditions. Multiple data sources were used to formulate the initial audit and feedback program theory: a systematic review, realist review, program document review, and informal discussions with key program stakeholders. Semi-structured interviews were then conducted with 56 participants to refute, refine, or confirm the initial program theories. Data were analysed retroductively using a context-mechanism-outcome framework for 11 transcripts which were coded into the audit and feedback program theory. The program theory was validated with three expert panels: senior health leaders (n = 19), Agency for Clinical Innovation (n = 11), and Ministry of Health (n = 21) staff.

RESULTS

The program's audit and feedback implementation strategy operated through eight mechanistic processes. The strategy worked well when clinicians (1) felt ownership and buy-in, (2) could make sense of the information provided, (3) were motivated by social influence, and (4) accepted responsibility and accountability for proposed changes. The success of the strategy was constrained when the audit process led to (5) rationalising current practice instead of creating a learning opportunity, (6) perceptions of unfairness and concerns about data integrity, 7) development of improvement plans that were not followed, and (8) perceived intrusions on professional autonomy.

CONCLUSIONS

Audit and feedback strategies may help reduce unwarranted clinical variation in care where there is engagement between auditors and local clinicians, meaningful audit indicators, clear improvement plans, and respect for clinical expertise. We contribute theoretical development for audit and feedback by proposing a Model for Audit and Feedback Implementation at Scale. Recommendations include limiting the number of audit indicators, involving clinical staff and local leaders in feedback, and providing opportunities for reflection.

A grounded theory-based qualitative approach for examining local implementation of public health policies during crises

Background

COVID-19's emergence questions the agility of health policy deployment in a context of urgency. This exceptional pandemic offers a unique Implementation Science study opportunity. It reveals how actors adapt, coordinate, and mitigate an unknown global threat to safeguard populations from an initially mysterious virus. Limited research has explored how involved players act and adapt their practices to fulfil health protection missions during a global health crisis. Bridging the gap between public policy expectations and achievements requires a methodology for stakeholder identification and implementation practice description.

Objective

Focusing on COVID-19 management in France's second-largest region, we investigate ministerial recommendation implementation and the emergence of new links, coordination modes, and practices.

Methods

Due to the novel subject, we adopted grounded theory. Initial documentary data collection identifies stakeholders for subsequent interviews. Open-ended coding of collected discourse enables content analysis.

Results

Findings reveal a crisis-driven re-evaluation of stakeholder relationships. This research identifies three levels of implementation of health policies at the local level (administrative, organizational and operational) and reveals different types of coordination specific to each of these levels. Our results provide insights on how to better coordinate and implement healthcare policies in a period of crisis. Recommendations include real-life simulations of large-scale crises.

Conclusion

Our work establishes a methodological foundation for analysing coordination dynamics. Future research could compare these findings with other unpredictable health emergencies, such as episiotomic veterinary health crises.•The first step of the method is to analyse the guidelines of health policy implementation during the Covid-19 crisis and to identify the main stakeholders in charge of the local health policy implementation.•The second step consists of interviewing these stakeholders using a co-constructed sample and structural coding of their speech to reveal the forms of coordination between stakeholders.

Champion and audit and feedback strategy fidelity and their relationship to depression intervention fidelity: A mixed method study

Background

Globally, mental health disorders rank as the greatest cause of disability. Low and middle-income countries (LMICs) hold a disproportionate share of the mental health burden, especially as it pertains to depression. Depression is highly prevalent among those with non-communicable diseases (NCDs), creating a barrier to successful treatment. While some treatments have proven efficacy in LMIC settings, wide dissemination is challenged by multiple factors, leading researchers to call for implementation strategies to overcome barriers to care provision. However, implementation strategies are often not well defined or documented, challenging the interpretation of study results and the uptake and replication of strategies in practice settings. Assessing implementation strategy fidelity (ISF), or the extent to which a strategy was implemented as designed, overcomes these challenges. This study assessed fidelity of two implementation strategies (a 'basic' champion strategy and an 'enhanced' champion + audit and feedback strategy) to improve the integration of a depression intervention, measurement based care (MBC), at 10 NCD clinics in Malawi. The primary goal of this study was to assess the relationship between the implementation strategies and MBC fidelity using a mixed methods approach.

Methods

We developed a theory-informed mixed methods fidelity assessment that first combined an implementation strategy specification technique with a fidelity framework. We then created corresponding fidelity indicators to strategy components. Clinical process data and one-on-one in-depth interviews with 45 staff members at 6 clinics were utilized as data sources. Our final analysis used descriptive statistics, reflexive-thematic analysis (RTA), data merging, and triangulation to examine the relationship between ISF and MBC intervention fidelity.

Results

Our mixed methods analysis revealed how ISF may moderate the relationship between the strategies and MBC fidelity. Leadership engagement and implementation climate were critical for clinics to overcome implementation barriers and preserve implementation strategy and MBC fidelity. Descriptive statistics determined champion strategy fidelity to range from 61 to 93% across the 10 clinics. Fidelity to the audit and feedback strategy ranged from 82 to 91% across the 5 clinics assigned to that condition. MBC fidelity ranged from 54 to 95% across all clinics. Although correlations between ISF and MBC fidelity were not statistically significant due to the sample of 10 clinics, associations were in the expected direction and of moderate effect size. A coefficient for shared depression screening among clinicians had greater face validity compared to depression screening coverage and functioned as a proximal indicator of implementation strategy success.

Conclusion

Fidelity to the basic and enhanced strategies varied by site and were influenced by leadership engagement and implementation climate. Champion strategies may benefit from the addition of leadership strategies to help address implementation barriers outside the purview of champions. ISF may moderate the relationship between strategies and implementation outcomes.

Guideline evaluation and implementation mechanisms in school health services (GuideMe): protocol for a hybrid randomized factorial trial

BACKGROUND

Norwegian school health services received a national best-practice guideline in 2017. To promote healthy life skills and identify adolescents needing support, the guideline includes strong recommendations for individual consultations with all 8th graders and increased collaboration with schools. To help implement the recommendations, a blended implementation strategy (SchoolHealth) was co-created with school nurses, students, and stakeholders. SchoolHealth consists of three implementation elements: Digital dialog and administration tool (audit and feedback +), Dialog support (external consultation), and Collaboration materials (targeted dissemination). This hybrid study will test the main and combined effects of the elements on guideline fidelity and effectiveness.

METHODS

The GuideMe study is a factorial cluster randomized controlled trial examining SchoolHealth's effectiveness on guideline fidelity and guideline effectiveness goals. Forty Norwegian secondary schools will be randomized to eight different combinations of the elements in SchoolHealth. Participants will include school nurses and school personnel from these schools, and 8th grade students (n = 1200). Primary outcomes are school nurses' fidelity to the guidelines and student's ability to cope with their life (i.e., health literacy, positive health behaviors and self-efficacy). Quantitative methods will be used to test effects and mechanisms, while mixed- and qualitative methods will be used to explore mechanisms, experiences, and other phenomena in depth. Participants will complete digital questionnaires at the start and end of the schoolyear, and after the consultation during the schoolyear. The study will run in two waves, each lasting for one school year. The multifactorial design allows testing of interactions and main effects due to equal distribution of all factors within each main effect. Sustainment and scale-up of optimized SchoolHealth elements using national infrastructure are simultaneously prepared.

DISCUSSION

The study will investigate possible effects of the implementation elements in isolation and in combination, and hypothesized implementation mechanisms. In-depth study of user experiences will inform improvements to elements in SchoolHealth. The results will yield causal knowledge about implementation strategies and the mechanisms through which they assert effects. Mixed-methods will provide insights into how and when the elements work. Optimizing guideline implementation elements can support adolescents in a crucial life phase.

TRAIL REGISTRATION

ISRCTN24173836. Registration date 8 August 2022.

Applying the Theoretical Domains Framework to Develop an Intervention to 'Re-implement' Parent-Child Interaction Therapy (PCIT)

Parent-Child Interaction Therapy (PCIT) is an empirically supported treatment for childhood conduct problems, with increasing numbers of clinicians being trained in Aotearoa/New Zealand. However, ensuring sustained delivery of effective treatments by trained clinicians in routine care environments is notoriously challenging. The aims of this qualitative study were to (1) systematically examine and prioritise PCIT implementation barriers and facilitators, and (2) develop a well specified and theory-driven 're-implementation' intervention to support already-trained clinicians to resume or increase their implementation of PCIT. To triangulate and refine existing understanding of PCIT implementation determinants from an earlier cross-sectional survey, we integrated previously unanalysed qualitative survey data (54 respondents; response rate 60%) with qualitative data from six new focus groups with 15 PCIT-trained clinicians and managers in Aotearoa/New Zealand. We deductively coded data, using a directed content analysis process and the Theoretical Domains Framework, resulting in the identification of salient theoretical domains and belief statements within these. We then used the Theory and Techniques Tool to identify behaviour change techniques, possible intervention components, and their hypothesised mechanisms of action. Eight of the 14 theoretical domains were identified as influential on PCIT-trained clinician implementation behaviour (Knowledge; Social/Professional Role and Identity; Beliefs about Capabilities; Beliefs about Consequences; Memory, Attention and Decision Processes; Environmental Context and Resources; Social Influences; Emotion). Two of these appeared to be particularly salient: (1) 'Environmental Context and Resources', specifically lacking suitable PCIT equipment, with (lack of) access to a well-equipped clinic room appearing to influence implementation behaviour in several ways. (2) 'Social/Professional Role and Identity', with beliefs relating to a perception that colleagues view time-out as harmful to children, concerns that internationally-developed PCIT is not suitable for non-Māori clinicians to deliver to Indigenous Māori families, and clinicians feeling obligated yet isolated in their advocacy for PCIT delivery. In conclusion, where initial implementation has stalled or languished, re-implementation may be possible, and makes good sense, both fiscally and practically. This study suggests that re-implementation of PCIT in Aotearoa/New Zealand may be facilitated by intervention components such as ensuring access to a colleague or co-worker who is supportive of PCIT delivery, access to suitable equipment (particularly a time-out room), and targeted additional training for clinicians relating to the safety of time-out for children. The feasibility and acceptability of these intervention components will be tested in a future clinical trial.

Application of the Expert Recommendations for Implementing Change (ERIC) compilation of strategies to health intervention implementation in low- and middle-income countries: a systematic review

BACKGROUND

The Expert Recommendations for Implementing Change (ERIC) project developed a compilation of implementation strategies that are intended to standardize reporting and evaluation. Little is known about the application of ERIC in low- and middle-income countries (LMICs). We systematically reviewed the literature on the use and specification of ERIC strategies for health intervention implementation in LMICs to identify gaps and inform future research.

METHODS

We searched peer-reviewed articles published through March 2023 in any language that (1) were conducted in an LMIC and (2) cited seminal ERIC articles or (3) mentioned ERIC in the title or abstract. Two co-authors independently screened all titles, abstracts, and full-text articles, then abstracted study, intervention, and implementation strategy characteristics of included studies.

RESULTS

The final sample included 60 studies describing research from all world regions, with over 30% published in the final year of our review period. Most studies took place in healthcare settings (n = 52, 86.7%), while 11 (18.2%) took place in community settings and four (6.7%) at the policy level. Across studies, 548 distinct implementation strategies were identified with a median of six strategies (range 1-46 strategies) included in each study. Most studies (n = 32, 53.3%) explicitly matched implementation strategies used for the ERIC compilation. Among those that did, 64 (87.3%) of the 73 ERIC strategies were represented. Many of the strategies not cited included those that target systems- or policy-level barriers. Nearly 85% of strategies included some component of strategy specification, though most only included specification of their action (75.2%), actor (57.3%), and action target (60.8%). A minority of studies employed randomized trials or high-quality quasi-experimental designs; only one study evaluated implementation strategy effectiveness.

CONCLUSIONS

While ERIC use in LMICs is rapidly growing, its application has not been consistent nor commonly used to test strategy effectiveness. Research in LMICs must better specify strategies and evaluate their impact on outcomes. Moreover, strategies that are tested need to be better specified, so they may be compared across contexts. Finally, strategies targeting policy-, systems-, and community-level determinants should be further explored.

TRIAL REGISTRATION

PROSPERO, CRD42021268374.

A structured approach to applying systems analysis methods for examining implementation mechanisms

BACKGROUND

It is challenging to identify and understand the specific mechanisms through which an implementation strategy affects implementation outcomes, as implementation happens in the context of complex, multi-level systems. These systems and the mechanisms within each level have their own dynamic environments that change frequently. For instance, sequencing may matter in that a mechanism may only be activated indirectly by a strategy through another mechanism. The dosage or strength of a mechanism may vary over time or across different health care system levels. To elucidate the mechanisms relevant to successful implementation amidst this complexity, systems analysis methods are needed to model and manage complexity.

METHODS

The fields of systems engineering and systems science offer methods-which we refer to as systems analysis methods-to help explain the interdependent relationships between and within systems, as well as dynamic changes to systems over time. When applied to studying implementation mechanisms, systems analysis methods can help (i) better identify and manage unknown conditions that may or may not activate mechanisms (both expected mechanisms targeted by a strategy and unexpected mechanisms that the methods help detect) and (ii) flexibly guide strategy adaptations to address contextual influences that emerge after the strategy is selected and used.

RESULTS

In this paper, we delineate a structured approach to applying systems analysis methods for examining implementation mechanisms. The approach includes explicit steps for selecting, tailoring, and evaluating an implementation strategy regarding the mechanisms that the strategy is initially hypothesized to activate, as well as additional mechanisms that are identified through the steps. We illustrate the approach using a case example. We then discuss the strengths and limitations of this approach, as well as when these steps might be most appropriate, and suggest work to further the contributions of systems analysis methods to implementation mechanisms research.

CONCLUSIONS

Our approach to applying systems analysis methods can encourage more mechanisms research efforts to consider these methods and in turn fuel both (i) rigorous comparisons of these methods to alternative mechanisms research approaches and (ii) an active discourse across the field to better delineate when these methods are appropriate for advancing mechanisms-related knowledge.

Use of a multi-phased approach to identify and address facilitators and barriers to the implementation of a population-wide genomic screening program

INTRODUCTION

Population-wide genomic screening for CDC Tier-1 conditions offers the ability to identify the 1-2% of the US population at increased risk for Hereditary Breast and Ovarian Cancer, Lynch Syndrome, and Familial Hypercholesterolemia. Implementation of population-wide screening programs is highly complex, requiring engagement of diverse collaborators and implementation teams. Implementation science offers tools to promote integration of these programs through the identification of determinants of success and strategies to address potential barriers.

METHODS

Prior to launching the program, we conducted a pre-implementation survey to assess anticipated barriers and facilitators to reach, effectiveness, adoption, implementation, and maintenance (RE-AIM), among 51 work group members (phase 1). During the first year of program implementation, we completed coding of 40 work group meetings guided by the Consolidated Framework for Implementation Research (CFIR) (phase 2). We matched the top barriers to implementation strategies identified during phase 2 using the CFIR-ERIC (Expert Recommendation for Implementing Change) matching tool.

RESULTS

Staffing and workload concerns were listed as the top barrier in the pre-implementation phase of the program. Top barriers during implementation included adaptability (n = 8, 20%), complexity (n = 14, 35%), patient needs and resources (n = 9, 22.5%), compatibility (n = 11, 27.5%), and self-efficacy (n = 9, 22.5%). We identified 16 potential implementation strategies across six ERIC clusters to address these barriers and operationalized these strategies for our specific setting and program needs.

CONCLUSION

Our findings provide an example of successful use of the CFIR-ERIC tool to guide implementation of a population-wide genomic screening program.

The implementation and effectiveness of multi-tasked, paid community health workers on maternal and child health: A cluster-randomized pragmatic trial and qualitative process evaluation in Tanzania

Community health worker programs have proliferated worldwide based on evidence that they help prevent mortality, particularly among children. However, there is limited evidence from randomized studies on the processes and effectiveness of implementing community health worker programs through public health systems. This paper describes the results of a cluster-randomized pragmatic implementation trial (registration number ISRCTN96819844) and qualitative process evaluation of a community health worker program in Tanzania that was implemented from 2011-2015. Program effects on maternal, newborn and child health service utilization, childhood morbidity and sick childcare seeking were evaluated using difference-in-difference regression analysis with outcomes measured through pre- and post-intervention household surveys in intervention and comparison trial arms. A qualitative process evaluation was conducted between 2012 and 2014 and comprised of in-depth interviews and focus group discussions with community health workers, community members, facility-based health workers and staff of district health management teams. The community health worker program reduced incidence of illness and improved access to timely and appropriate curative care for children under five; however, there was no effect on facility-based maternal and newborn health service utilization. The positive outcomes occurred because of high levels of acceptability of community health workers within communities, as well as the durability of community health workers' motivation and confidence. Implementation factors that generated these effects were the engagement of communities in program startup; the training, remuneration and supervision of the community health workers from the local health system and community. The lack of program effects on maternal and newborn health service utilization at facilities were attributed to lapses in the availability of needed care at facilities. Strategies that strengthen and align communities' and health systems core capacities, and their ability to learn, adapt and integrate evidence-based interventions, are needed to maximize the health impact of community health workers.

A scoping review of implementation science theories, models, and frameworks - an appraisal of purpose, characteristics, usability, applicability, and testability

BACKGROUND

A proliferation of theories, models, and frameworks (TMFs) have been developed in the implementation science field to facilitate the implementation process. The basic features of these TMFs have been identified by several reviews. However, systematic appraisals on the quality of these TMFs are inadequate. To fill this gap, this study aimed to assess the usability, applicability, and testability of the current TMFs in a structured way.

METHODS

A scoping review method was employed. Electronic databases were searched to locate English and Chinese articles published between January 2000 and April 2022. Search terms were specific to implementation science. Additionally, hand searches were administered to identify articles from related reviews. Purpose and characteristics such as the type of TMF, analytical level, and observation unit were extracted. Structured appraisal criteria were adapted from Birken et al.'s Theory Comparison and Selection Tool (T-CaST) to conduct an in-depth analysis of the TMFs' usability, applicability, and testability.

RESULTS

A total of 143 TMFs were included in this analysis. Among them, the most common purpose was to identify barriers and facilitators. Most TMFs applied the descriptive method to summarize the included constructs or the prescriptive method to propose courses of implementation actions. TMFs were mainly mid-range theories built on existing conceptual frameworks or demonstrated grand theories. The usability of the TMFs needs to be improved in terms of the provision of conceptually matched strategies to barriers and facilitators and instructions on the TMFs usage. Regarding the applicability, little attention was paid to the constructs of macro-level context, stages of scale-up and sustainability, and implementation outcomes like feasibility, cost, and penetration. Also, fewer TMFs could propose recommended research and measurement methods to apply the TMFs. Lastly, explicit hypotheses or propositions were lacking in most of the TMFs, and empirical evidence was lacking to support the claimed mechanisms between framework elements in testability.

CONCLUSIONS

Common limitations were found in the usability, application, and testability of the current TMFs. The findings of this review could provide insights for developers of TMFs for future theoretical advancements.

Effectiveness of de-implementation strategies for low-value prescribing in secondary care: a systematic review

BACKGROUND/AIMS

Considerable efforts have been made to improve guideline adherence in healthcare through de-implementation, such as decreasing the prescription of inappropriate medicines. However, we have limited knowledge about the effectiveness, barriers, facilitators and consequences of de-implementation strategies targeting inappropriate medication prescribing in secondary care settings. This review was conducted to understand these factors to contribute to better replication and optimisation of future de-implementation efforts to reduce low-value care.

METHODS

A systematic review of randomised control trials was conducted. Papers were identified through CINAHL, EMBASE, MEDLINE and Cochrane register of controlled trials to February 2021. Eligible studies were randomised control trials evaluating behavioural strategies to de-implement inappropriate prescribing in secondary healthcare. Risk of bias was assessed using the Cochrane Risk of Bias tool. Intervention characteristics, effectiveness, barriers, facilitators and consequences were identified in the study text and tabulated.

RESULTS

Eleven studies were included, of which seven were reported as effectively de-implementing low-value prescribing. Included studies were judged to be mainly at low to moderate risk for selection biases and generally high risk for performance and reporting biases. The majority of these strategies were clinical decision support at the 'point of care'. Clinical decision support tools were the most common and effective. They were found to be a low-cost and simple strategy. However, barriers such as clinician's reluctance to accept recommendations, or the clinical setting were potential barriers to their success. Educational strategies were the second most reported intervention type however the utility of educational strategies for de-implementation remains varied. Multiple barriers and facilitators relating to the environmental context, resources and knowledge were identified across studies as potentially influencing de-implementation. Various consequences were identified; however, few measured the impact of de-implementation on usual appropriate practice.

CONCLUSION

This review offers insight into the intervention strategies, potential barriers, facilitators and consequences that may affect the de-implementation of low-value prescribing in secondary care. Identification of these key features helps understand how and why these strategies are effective and the wider (desirable or undesirable) impact of de-implementation. These findings can contribute to the successful replication or optimisation of strategies used to de-implement low-value prescribing practices in future.

TRIAL REGISTRATION

The review protocol was registered at PROSPERO (ID: CRD42021243944).

Assessment of Adoption and Early Implementation Barriers and Facilitators of the Safe Environment for Every Kid (SEEK) Model

OBJECTIVE

Although the use of interventions for screening for social determinants of health of families in pediatric primary care clinics has increased in the past decade, research on the barriers and facilitators of implementing such interventions has been limited. We explored barriers, facilitators, and the mechanisms clarifying their roles in the adoption and implementation of the Safe Environment for Every Kid (SEEK) model, an approach for strengthening families, promoting children's health and development, and preventing child maltreatment.

METHODS

A total of 28 semistructured interviews were completed with 9 practice champions, 11 primary care professionals, 5 behavioral health professionals, and 3 nursing/administrative staff representing 12 pediatric primary care practices participating in a larger randomized control trial of implementing SEEK.

RESULTS

We identified several barriers and facilitators in the stages of SEEK's adoption and early implementation. Barriers associated with outer and inner setting determinants and poor innovation-organization fit declined in importance over time, while facilitators associated with SEEK characteristics increased in importance based on participants' responses. Barriers and facilitators were linked by mechanisms of comparison and contrast of burdens and benefits, and problem-solving to address limited capacity with available resources.

CONCLUSIONS

Any screening for and addressing social determinants of health demands greater attention to adoption and implementation mechanisms and the processes by which primary care professionals assess and utilize facilitators to address barriers. This occurs in a context defined by perceived burdens and benefits of innovation adoption and implementation, the capacity of the practice, and changes in perception with experiencing the innovation.

Physical activity interventions in older adults with a cognitive impairment: A critical review of reviews

This critical review explores the review material on physical activity combined with cognitive stimulation interventions in older adults with cognitive impairment and/or dementia. A critical, systematic, review of the review method was used, considering four electronic databases: WEB OF SCIENCE, SCOPUS, MEDLINE, and the COCHRANE ELECTRONIC LIBRARY. The search terms "exercise," "physical activity," "cognitive impairment," "dementia," and "systematic review" were used. All available reviews were marked against predetermined inclusion and exclusion criteria. There were 32 reviews that met the inclusion criteria. A combination of various types of training and aerobic exercises were the most frequently reported interventions; meanwhile, dual task training programs (combining physical exercise with cognitive stimulation), functional training programs along with exercises combination, aerobic exercise as well as strength, stretching, or balance workouts were also reported. The evidence is compelling; exercise can improve physical health by ensuring cognitive, psychological, and behavioral benefits. Overall, exercise can improve the physical and mental health of people living with dementia: there is sufficient evidence to recommend multimodal exercise.

Systematic development of a set of implementation strategies for transitional care innovations in long-term care

BACKGROUND

Numerous transitional care innovations (TCIs) are being developed and implemented to optimize care continuity for older persons when transferring between multiple care settings, help meet their care needs, and ultimately improve their quality of life. Although the implementation of TCIs is influenced by contextual factors, the use of effective implementation strategies is largely lacking. Thus, to improve the implementation of TCIs targeting older persons receiving long-term care services, we systematically developed a set of viable strategies selected to address the influencing factors.

METHODS

As part of the TRANS-SENIOR research network, a stepwise approach following Implementation Mapping (steps 1 to 3) was applied to select implementation strategies. Building on the findings of previous studies, existing TCIs and factors influencing their implementation were identified. A combination of four taxonomies and overviews of change methods as well as relevant evidence on their effectiveness were used to select the implementation strategies targeting each of the relevant factors. Subsequently, individual consultations with scientific experts were performed for further validation of the process of mapping strategies to implementation factors and for capturing alternative ideas on relevant implementation strategies.

RESULTS

Twenty TCIs were identified and 12 influencing factors (mapped to the Consolidated Framework for Implementation Research) were designated as priority factors to be addressed with implementation strategies. A total of 40 strategies were selected. The majority of these target factors at the organizational level, e.g., by using structural redesign, public commitment, changing staffing models, conducting local consensus discussions, and organizational diagnosis and feedback. Strategies at the level of individuals included active learning, belief selection, and guided practice. Each strategy was operationalized into practical applications.

CONCLUSIONS

This project developed a set of theory and evidence-based implementation strategies to address the influencing factors, along further tailoring for each context, and enhance the implementation of TCIs in daily practice settings. Such work is critical to advance the use of implementation science methods to implement innovations in long-term care successfully.

Rapid implementation mapping to identify implementation determinants and strategies for cervical cancer control in Nigeria

Background

Cervical cancer constitutes a huge burden among women in Nigeria, particularly HIV-infected women. However, the provision and uptake of cervical cancer screening and treatment is limited in Nigeria. Understanding implementation determinants is essential for the effective translation of such evidence-based interventions into practice, particularly in low-resource settings. COVID-19 pandemic necessitated online collaboration making implementation mapping challenging in some ways, while providing streamlining opportunities. In this study, we describe the use of a virtual online approach for implementation mapping (steps 1-3) to identify implementation determinants, mechanisms, and strategies to implement evidence-based cervical cancer screening and treatment in existing HIV infrastructure in Nigeria.

Methods

This study used a mixed methods study design with a virtual modified nominal group technique (NGT) process aligning with Implementation Mapping steps 1-3. Eleven stakeholders (six program staff and five healthcare providers and administrators) participated in a virtual NGT process which occurred in two phases. The first phase utilized online surveys, and the second phase utilized an NGT and implementation mapping process. The Exploration, Preparation, Implementation and Sustainment (EPIS) framework was used to elicit discussion around determinants and strategies from the outer context (i.e., country and regions), inner organizational context of existing HIV infrastructure, bridging factors that relate to bi-directional influences, and the health innovation to be implemented (in this case cervical cancer screening and treatment). During the NGT, the group ranked implementation barriers and voted on implementation strategies using Mentimeter.

Results

Eighteen determinants to integrating cervical cancer screening and treatment into existing comprehensive HIV programs were related to human resources capacity, access to cervical cancer services, logistics management, clinic, and client-related factors. The top 3 determinants included gaps in human resources capacity, poor access to cervical cancer services, and lack of demand for services resulting from lack of awareness about the disease and servicesA set of six core implementation strategies and two enhanced implementation strategies were identified.

Conclusions

Rapid Implementation Mapping is a feasible and acceptable approach for identifying and articulating implementation determinants, mechanisms, and strategies for complex healthcare interventions in LMICs.

Acceptability, appropriateness, and feasibility of Rural School Support Strategies for behavioral interventions: a mixed methods evaluation over two years of a hybrid type 3 implementation-effectiveness trial

BACKGROUND

Positive Behavioral Interventions and Supports (PBIS) is a framework for implementing evidence-based interventions for preventing behavioral issues and improving climate in schools. The implementation of school-wide PBIS with fidelity is complex, requiring leadership commitment, teaming, and coordination of systems for tracking behaviors and consequences. Putting these components in place while ensuring alignment with the values and needs of the school community can be difficult for schools with fewer resources, such as rural schools. Implementation supports are needed, including strategies such as technical assistance, but it is unclear whether lower-cost modalities such as virtual support are acceptable, appropriate, and feasible and whether perceptions vary throughout the implementation process.

METHODS

A type 3 hybrid implementation-effectiveness trial is taking place in 40 Idaho schools, testing a bundle of implementation supports selected to meet the needs of schools in rural areas. Supports include technical assistance from an implementation support practitioner (ISP), didactic trainings, virtual learning sessions, and an online resource portal. Surveys and interviews in the first 2 years of implementation (fall 2019 to spring 2021) explored outcomes of acceptability, appropriateness, and feasibility regarding the implementation supports among more than 150 school stakeholders.

RESULTS

Evaluations showed high acceptability and appropriateness of the PBIS concepts and training. The 20 schools receiving additional implementation support rated the technical assistance and support from the project's ISPs as the most acceptable and appropriate resource. Reasons for acceptability were the relationship built with the ISP, the ISP's expertise, and being a "neutral party." Although in-person support from the ISP was preferred, remote support was acceptable and increased feasibility of attendance. Virtual learning sessions were acceptable for learning and collaboration, particularly in the second year of implementation, once ISPs had developed closer relationships with school teams.

CONCLUSIONS

School staff found training, technical assistance, and virtual learning sessions to be acceptable and appropriate. Virtual formats of training and technical assistance decreased in acceptability but increased feasibility of attendance. In-person support was preferred during initial implementation, and virtual support was more acceptable thereafter.

TRIAL REGISTRATION

This trial was prospectively registered on ClinicalTrials.gov ( NCT03736395 ), on November 9, 2018.

Ten years of implementation outcomes research: a scoping review

BACKGROUND

Proctor and colleagues' 2011 paper proposed a taxonomy of eight implementation outcomes and challenged the field to address a research agenda focused on conceptualization, measurement, and theory building. Ten years later, this paper maps the field's progress in implementation outcomes research. This scoping review describes how each implementation outcome has been studied, research designs and methods used, and the contexts and settings represented in the current literature. We also describe the role of implementation outcomes in relation to implementation strategies and other outcomes.

METHODS

Arksey and O'Malley's framework for conducting scoping reviews guided our methods. Using forward citation tracing, we identified all literature citing the 2011 paper. We conducted our search in the Web of Science (WOS) database and added citation alerts sent to the first author from the publisher for a 6-month period coinciding with the WOS citation search. This produced 1346 titles and abstracts. Initial abstract screening yielded 480 manuscripts, and full-text review yielded 400 manuscripts that met inclusion criteria (empirical assessment of at least one implementation outcome).

RESULTS

Slightly more than half (52.1%) of included manuscripts examined acceptability. Fidelity (39.3%), feasibility (38.6%), adoption (26.5%), and appropriateness (21.8%) were also commonly examined. Penetration (16.0%), sustainability (15.8%), and cost (7.8%) were less frequently examined. Thirty-two manuscripts examined implementation outcomes not included in the original taxonomy. Most studies took place in healthcare (45.8%) or behavioral health (22.5%) organizations. Two-thirds used observational designs. We found little evidence of progress in testing the relationships between implementation strategies and implementation outcomes, leaving us ill-prepared to know how to achieve implementation success. Moreover, few studies tested the impact of implementation outcomes on other important outcome types, such as service systems and improved individual or population health.

CONCLUSIONS

Our review presents a comprehensive snapshot of the research questions being addressed by existing implementation outcomes literature and reveals the need for rigorous, analytic research and tests of strategies for attaining implementation outcomes in the next 10 years of outcomes research.

The Polarization of Clinician and Service Staff Perspectives After the Use of Health Information Technology in Youth Mental Health Services: Implementation and Evaluation Study

BACKGROUND

Highly personalized care is substantially improved by technology platforms that assess and track patient outcomes. However, evidence regarding how to successfully implement technology in real-world mental health settings is limited.

OBJECTIVE

This study aimed to naturalistically monitor how a health information technology (HIT) platform was used within 2 real-world mental health service settings to gain practical insights into how HIT can be implemented and sustained to improve mental health service delivery.

METHODS

An HIT (The Innowell Platform) was naturally implemented in 2 youth mental health services in Sydney, Australia. Web-based surveys (n=19) and implementation logs were used to investigate staff attitudes toward technology before and after implementation. Descriptive statistics were used to track staff attitudes over time, whereas qualitative thematic analysis was used to explore implementation log data to gain practical insights into useful implementation strategies in real-world settings.

RESULTS

After the implementation, the staff were nearly 3 times more likely to agree that the HIT would improve care for their clients (3/12, 25% agreed before the implementation compared with 7/10, 70% after the implementation). Despite this, there was also an increase in the number of staff who disagreed that the HIT would improve care (from 1/12, 8% to 2/10, 20%). There was also decreased uncertainty (from 6/12, 50% to 3/10, 30%) about the willingness of the service to implement the technology for its intended purpose, with similar increases in the number of staff who agreed and disagreed with this statement. Staff were more likely to be uncertain about whether colleagues in my service are receptive to changes in clinical processes (not sure rose from 5/12, 42% to 7/10, 70%). They were also more likely to report that their service already provides the best mental health care (agreement rose from 7/12, 58% to 8/10, 80%). After the implementation, a greater proportion of participants reported that the HIT enabled shared or collaborative decision-making with young people (2/10, 20%, compared with 1/12, 8%), enabled clients to proactively work on their mental health care through digital technologies (3/10, 30%, compared with 2/12, 16%), and improved their response to suicidal risk (4/10, 40% compared with 3/12, 25%).

CONCLUSIONS

This study raises important questions about why clinicians, who have the same training and support in using technology, develop more polarized opinions on its usefulness after implementation. It seems that the uptake of HIT is heavily influenced by a clinician's underlying beliefs and attitudes toward clinical practice in general as well as the role of technology, rather than their knowledge or the ease of use of the HIT in question.

Individual-level associations between implementation leadership, climate, and anticipated outcomes: a time-lagged mediation analysis

BACKGROUND

Leaders can improve implementation outcomes by developing an organizational climate conducive to the implementation of evidence-based practices (EBP). This study tested the lagged associations between individual-level perceptions of implementation leadership, implementation climate, and three anticipated implementation outcomes, that is EBP acceptability, appropriateness, and feasibility.

METHODS

Screening tools and treatment methods for posttraumatic stress disorder were implemented in 43 Norwegian mental health services. A sample of 494 child and adult mental health care professionals (M = 43 years, 78% female) completed surveys addressing perceptions of first-level leaders' (n = 47) implementation leadership and their clinics' implementation climate. Single-level structural equation models estimating both direct, indirect, and total effects were used to investigate whether perceived implementation climate mediated the association between perceived implementation leadership and perceived acceptability, appropriateness, and feasibility of screening tools and treatment methods.

RESULTS

Regarding the treatment methods, implementation leadership was associated with therapists' perceptions of acceptability, appropriateness, and feasibility. Implementation climate also mediated between implementation leadership and the outcomes. Regarding the screening tools, implementation leadership was not associated with the outcomes. However, implementation climate mediated between implementation leadership and therapists' perceptions of acceptability and feasibility, but not appropriateness. Analyses with the implementation climate subscales showed stronger associations for therapists' perceptions of the treatment methods than of screening tools.

CONCLUSIONS

Leaders may promote positive implementation outcomes, both directly and through implementation climate. With regard to the effect sizes and explained variance, results indicated that both implementation leadership and implementation climate were more strongly associated with the therapists' perceptions of the treatment methods, implemented by one group of therapists, than the screening tools, implemented by all therapists. This may imply that implementation leadership and climate may have stronger effects for smaller implementation teams within a larger system than for system-wide implementations or when the clinical interventions being implemented are more complex rather than simple ones.

TRIAL REGISTRATION

ClinicalTrials NCT03719651, 25 October 2018.

Implementation of Home-Based Telerehabilitation of Patients With Stroke in the United States: Protocol for a Realist Review

BACKGROUND

Stroke is a common cause of mortality and morbidity. Insufficient and untimely rehabilitation has been associated with inadequate recovery. Telerehabilitation provides an opportunity for timely and accessible services for individuals with stroke, especially in remote areas. Telerehabilitation is defined as a health care team's use of a communication mode (eg, videoconferencing) to remotely provide rehabilitation services. Telerehabilitation is as effective as facility-based rehabilitation; however, it is infrequently used due to implementation barriers.

OBJECTIVE

The aim of the study is to explore the interaction between the implementation strategies, context, and outcomes of telerehabilitation of patients with stroke.

METHODS

This review will follow four steps: (1) defining the review scope, (2) literature search and quality appraisal, (3) data extraction and evidence synthesis, and (4) narrative development. PubMed via MEDLINE, the PEDro database, and CINAHL will be queried till June 2023 and supplemented with citation tracking and a gray literature search. The relevance and rigor of papers will be appraised using the TAPUPAS (Transparency, Accuracy, Purposivity, Utility, Propriety, Accessibility, and Specificity) and Weight of Evidence frameworks. The reviewers will extract and synthesize data iteratively and develop explanatory links between contexts, mechanisms, and outcomes. The results will be reported according to the Realist Synthesis publication standards set by Wong and colleagues in 2013.

RESULTS

The literature search and screening will be completed in July 2023. Data extraction and analysis will be completed in August 2023, and findings will be synthesized and reported in October 2023.

CONCLUSIONS

This will be the first realist synthesis, uncovering the causal mechanisms to explain how, why, and to what extent implementation strategies impact telerehabilitation adoption and implementation.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/47009.

Overcoming challenges in conducting systematic reviews in implementation science: a methods commentary

Consolidation of the literature using systematic reviews is a critical way to advance a discipline and support evidence-based decision-making in healthcare. However, unique challenges exist that impact the conduct of systematic reviews in implementation science. In this commentary, we reflect on our combined experience to describe five key challenges unique to systematic reviews of primary implementation research. These challenges include (1) descriptors used in implementation science publications, (2) distinction between evidence-based interventions and implementation strategies, (3) assessment of external validity, (4) synthesis of implementation studies with substantial clinical and methodological diversity, and (5) variability in defining implementation 'success'. We outline possible solutions and highlight resources that can be used by authors of primary implementation research, as well as systematic review and editorial teams, to overcome the identified challenges and optimise the utility of future systematic reviews in implementation science.