Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem

Utility of the Leadership and Organizational Change for Implementation-Systems Level (LOCI-SL) strategy for a statewide substance use treatment implementation effort

INTRODUCTION

Multi-level and cross-context implementation strategies are needed to support health systems, healthcare delivery organizations, and providers to adopt evidence-based practice (EBP) for substance use disorder (SUD) treatment. However, misalignment between state oversight agencies and healthcare organizations about which services to prioritize and which outcomes are reasonable to expect can hinder implementation success and widespread access to high-quality care. This study investigated the utility of the Leadership and Organizational Change for Implementation-System Level (LOCI-SL) strategy for supporting statewide EBP implementation for SUD treatment.

METHODS

Nine community mental health centers (CMHCs) contracted by a state agency participated in a combined motivational-enhancement therapy and cognitive behavioral therapy (MET/CBT) implementation effort. Five of the CMHCs also received the LOCI-SL strategy to obtain ongoing implementation support. We conducted 21 individual interviews and three small group interviews with 30 participants across CMHCs and state health agencies to investigate the utility of LOCI-SL in supporting their EBP implementation efforts. Deductive thematic analysis was guided by the Exploration, Preparation, Implementation, Sustainment Framework.

RESULTS

Five themes described CMHCs' LOCI-SL and broader contextual experiences implementing EBPs: (1) LOCI-SL supported executives in Preparation phase activities that holistically considered organizational needs and capacity to implement and sustain EBPs; (2) LOCI-SL facilitated trust and communication processes across Preparation, Implementation, and Sustainment phases to improve EBP uptake; (3) LOCI-SL increased CMHCs' use of implementation climate strengthening activities throughout the Implementation phase; (4) state contracts did not emphasize quality and thus were not sufficient bridging factors to enforce EBP fidelity during Implementation; and, (5) limited funding and low Medicaid reimbursement rates hindered EBP use throughout the Implementation and Sustainment phases.

CONCLUSIONS

LOCI-SL was viewed as a favorable and useful implementation strategy for supporting statewide adoption of EBPs. However, outer context barriers, including limited financial investments in the treatment system, impeded implementation and sustainment efforts. While previous research suggests that contracts are viable alignment-promoting bridging factors, this study demonstrates the importance of articulating implementation outcome expectations to aid state-contracted organizations in achieving EBP implementation success. This study also highlights the need for multi-level implementation strategies to effectively align implementation expectations between outer- and inner-context entities.

Randomized Controlled Trial Evaluating the Effectiveness of a Direct-to-Consumer Marketing Video About Patients' Right to Evidence-Based Mental Health Care

ABSTRACT

This study evaluated the impact of a direct-to-consumer (DTC) marketing video designed to educate the public about patients' rights to evidence-based mental health care (EBMHC). Participants ( N = 632) were randomly assigned to an active DTC video condition, a control video condition, or a control condition without a video. Participants who watched the DTC video ( vs . both control conditions) had significantly greater knowledge of patients' rights to EBMHC. Further, individuals who watched the DTC ( vs . control) video reported significantly greater comfort with accessing care and perceived their assigned video as significantly more culturally sensitive. However, participants who watched the DTC video were not significantly different from both control conditions on self-report measures of self-efficacy in working with a provider, likelihood of asking a provider about one's rights, treatment-seeking intentions, and self-stigma. Findings suggest the potential for a DTC video to promote knowledge of EBMHC, though its impact on help-seeking perceptions and intentions was less promising.

Applying the Cognitive Walkthrough for Implementation Strategies Methodology to Inform the Redesign of a Selection-Quality Implementation Toolkit for Use in Schools

Background

Implementation strategies are key to enhancing translation of new innovations but there is a need to systematically design and tailor strategies to match the targeted implementation context and address determinants. There are increasing methods to inform the redesign and tailoring of implementation strategies to maximize their usability, feasibility, and appropriateness in new settings such as the Cognitive Walkthrough for Implementation Strategies (CWIS) approach. The aim of the current project is to apply the CWIS approach to inform the redesign of a multifaceted selection-quality implementation toolkit entitled ACT SMARTS for use in middle and high schools.

Methods

We systematically applied CWIS as the second part of a community-partnered iterative redesign of ACT SMARTS for schools to evaluate the usability and inform further toolkit redesign areas. We conducted three CWIS user testing sessions with key end users of school district administrators, school principals, and educators.

Results

Our CWIS application revealed that end users found ACT SMARTS acceptable and relevant but anticipate usability issues engaging in the ACT SMARTS process. Results informed the identification of eleven usability issues and corresponding redesign solutions to enhance the usability of ACT SMARTS for use in middle and high schools.

Conclusions

Results indicated the utility of CWIS in assessing implementation strategy usability in service of informing strategy tailoring and redesign to improve alignment with user and setting needs. Recommendations regarding the use of this participatory approach are discussed.

Implementing Screening, Brief Interventions, and Referral to Treatment at Pediatric Trauma Centers: A Step Wedge Cluster Randomized Trial

BACKGROUND

Pediatric trauma centers have had challenges meeting the American College of Surgeons criteria for screening and intervening for alcohol with adolescent trauma patients. The study objective was to conduct an implementation trial to evaluate the effectiveness of the Science to Service Laboratory (SSL) implementation strategy in improving alcohol and other drugs (AOD) screening, brief intervention, and referral to treatment (SBIRT) delivery at pediatric trauma centers.

METHODS

Using a stepped wedge cross-over cluster randomized design, 10 US pediatric trauma centers received the SSL implementation strategy to deliver SBIRT with admitted adolescent (12-17 years old) trauma patients. The strategy adapted three core SSL elements: didactic training, performance feedback, and facilitation. The main outcome measured was SBIRT reach. Data were collected from each center's electronic health record (EHR) during pre- and post-implementation wedges (2018-2022).

RESULTS

EHR data from 8461 adolescent patients were extracted. Aggregated across all sites, the reach of screening with a validated AOD screening tool increased significantly from 25.2% (95% CI: 23.9, 26.5%) of adolescents during pre-implementation to 47.7% (95% CI: 46.3%, 49.2%) post-implementation. There was variability of change across centers. Brief interventions continued to be delivered at high levels to identified adolescents. Referral to primary care providers for further AOD discussion or referral to specialty service for adolescents with high risk use did not improve post-implementation and remained low.

CONCLUSIONS

The SSL implementation strategy can be successfully utilized by pediatric trauma centers to improve AOD screening, but challenges exist in connecting adolescents for continuation of AOD discussions after discharge.

LEVEL OF EVIDENCE

Level II, Therapeutic.

Women's preferences for HIV prevention service delivery in pharmacies during pregnancy in Western Kenya: a discrete choice experiment

INTRODUCTION

Pharmacy-delivered HIV prevention services might create more options for pregnant women to use HIV prevention tools earlier and more consistently during pregnancy. We quantified preferences for attributes of potential HIV prevention services among women of childbearing age in Western Kenya.

METHODS

From June to November 2023, we administered a face-to-face discrete choice experiment survey to women aged 15-44 in Kenya's Homa Bay, Kisumu and Siaya counties. The survey evaluated preferences for HIV prevention services, described by seven attributes: service location, travel time, type of HIV test, sexually transmitted infection (STI) testing, partner HIV testing, pre-exposure prophylaxis (PrEP) and service fee. Participants answered a series of 12-choice questions. Each question asked them to select one of two service options or no services-an opt-out option. We used hierarchical Bayesian modelling levels to estimate each attribute level's coefficient and understand how attributes influenced service choice.

RESULTS

Overall, 599 participants completed the survey, among whom the median age was 23 years (IQR: 18-27); 33% were married, 20% had a job and worked regularly, and 52% had been pregnant before. Participants, on average, strongly preferred having any HIV prevention service option over none (opt-out preference weight: -5.84 [95% CI: -5.97, -5.72]). The most important attributes were the availability of PrEP (relative importance 27.04% [95% CI: 25.98%, 28.11%]), followed by STI testing (relative importance 20.26% [95% CI: 19.52%, 21.01%]) and partner HIV testing (relative importance: 16.35% [95% CI: 15.79%, 16.90%]). While, on average, participants preferred obtaining services at the clinic more than pharmacies, women prioritized the availability of PrEP, STI testing and partner HIV testing more than the location or cost.

CONCLUSIONS

These findings suggest the importance of providing comprehensive HIV prevention services and ensuring PrEP, STI testing and partner HIV testing are available. If pharmacies can offer these services, women are likely to access those services at pharmacies even if they prefer clinics.

Use of concept mapping to inform a participatory engagement approach for implementation of evidence-based HPV vaccination strategies in safety-net clinics

BACKGROUND

Multiple evidence-based strategies (EBS) for promoting HPV vaccination exist. However, adolescent HPV vaccination rates remain below target levels in communities at high risk for HPV-associated cancers and served by safety-net clinics. Participatory engaged approaches are needed to leverage the expertise of community and clinical partners in selecting EBS relevant to their local context. We engaged concept mapping as a method to inform the adoption and adaptation of EBS that seeks to empower implementation partners to prioritize, select, and ultimately implement context-relevant EBS for HPV vaccination.

METHODS

Using 38 EBS statements generated from qualitative interviews and national HPV vaccine advocacy sources, we conducted a modified concept mapping activity with partners internal to safety-net clinics and external community members in two study sites of a larger implementation study (Greater Los Angeles and New Jersey), to sort EBS into clusters and rate each EBS by importance and feasibility for increasing HPV vaccination within safety-net clinics. Concept mapping findings (EBS statement ratings, ladder graphs and go-zones) were shared with leaders from a large federally qualified health center (FQHC) system (focusing on three clinic sites), to select and implement EBS over 12 months.

RESULTS

Concept mapping participants (n=23) sorted and rated statements, resulting in an eight-cluster solution: 1) Community education and outreach; 2) Advocacy and policy; 3) Data access/quality improvement monitoring; 4) Provider tracking/audit and feedback; 5) Provider recommendation/communication; 6) Expanding vaccine access; 7) Reducing missed opportunities; and 8) Nurse/staff workflow and training. The FQHC partner then selected to intervene on eight of 17 EBS statements in the "go-zone" for action, with three from "reducing missed opportunities," two from "nurse/staff workflow and training," and one each from "provider tracking/audit and feedback," "provider recommendation/communication," and "expanding vaccine access," which the research team addressed through the implementation of three multi-level intervention strategies (e.g., physician communication training, staff training and workflow assessment, audit and feedback of clinic processes).

CONCLUSIONS

Concept mapping provided a powerful participatory approach to identify multilevel EBS for HPV vaccination relevant to the local safety-net clinic context, particularly when several strategies exist, and prioritization is necessary. This study demonstrates how a clinic system benefited directly from the ratings and prioritization of EBS by multilevel clinic and community partners within the broader safety-net clinic context to identify and adapt prioritized solutions needed to advance HPV vaccine equity.

How the experiences of implementation support recipients contribute to implementation outcomes

Introduction

There is a growing body of literature on the activities and competencies of implementation support practitioners (ISPs) and the outcomes of engaging ISPs to support implementation efforts. However, there remains limited knowledge about the experiences of implementation support recipients who engage with ISPs and how these experiences shape the trajectory of implementation and contribute to implementation outcomes. This study aimed to extend the research on ISPs by describing the experiences of professionals who received implementation support and inform our understanding of the mechanisms by which ISPs produce behavior change and contribute to implementation outcomes.

Methods

Thirteen individuals with roles in supporting implementation efforts at a private foundation participated in semi-structured interviews. Data were analyzed using qualitative narrative analysis and episode profile analysis approaches. Iterative diagramming was used to visualize the pathway of experiences of implementation support recipients evidenced by the interview data.

Results

The majority of recipients described how positive experiences and trusting relationships with ISPs increased acceptance of implementation science throughout the foundation and increased the perception of implementation science as both an appropriate and feasible approach for strengthening the impact of foundation strategies. As perceptions of appropriateness and feasibility increased, recipients of implementation support described increasing knowledge and application of implementation science in their funding engagements and internal foundation strategies. Finally, recipients reported that the application of implementation science across the foundation led to sustained implementation capacity and better outcomes.

Discussion

The experiences of implementation support recipients described in this paper provide a source for further understanding the mechanisms of change for delivering effective implementation support leading to better implementation quality. Insights from these experiences can enhance our understanding for building implementation capacity and the rationales for evolving approaches that emphasize the dynamic, emotional, and highly relational nature of supporting others to use evidence in practice.

A streamlined approach to classifying and tailoring implementation strategies: recommendations to speed the translation of research to practice

BACKGROUND

Implementation science emerged from the recognized need to speed the translation of effective interventions into practice. In the US, the science has evolved to place an ever-increasing focus on implementation strategies. The long list of implementation strategies, terminology used to name strategies, and time required to tailor strategies all may contribute to delays in translating evidence-based interventions (EBIs) into practice. To speed EBI translation, we propose a streamlined approach to classifying and tailoring implementation strategies.

MAIN TEXT

A multidisciplinary team of eight scholars conducted an exercise to sort the Expert Recommendations for Implementing Change (ERIC) strategies into three classes: implementation processes (n = 25), capacity-building strategies (n = 20), and integration strategies (n = 28). Implementation processes comprise best practices that apply across EBIs and throughout the phases of implementation from exploration through sustainment (e.g., conduct local needs assessment). Capacity-building strategies target either general or EBI-specific knowledge and skills (e.g., conduct educational meetings). Integration strategies include "methods and techniques" that target barriers or facilitators to implementation of a specific EBI beyond those targeted by capacity building. Building on these three classes, the team collaboratively developed recommendations for a pragmatic, five-step approach that begins with the implementation processes and capacity-building strategies practice-settings are already using prior to tailoring integration strategies. A case study is provided to illustrate use of the five-step approach to tailor the strategies needed to implement a transitional care intervention in skilled nursing facilities.

CONCLUSIONS

Our proposed approach streamlines the formative work required prior to implementing an EBI by building on practice partner preferences, expertise, and infrastructure while also making the most of prior research findings.

Interventions targeting social determinants of mental disorders and the Sustainable Development Goals: a systematic review of reviews

Globally, mental disorders account for almost 20% of disease burden and there is growing evidence that mental disorders are socially determined. Tackling the United Nations Sustainable Development Goals (UN SDGs), which address social determinants of mental disorders, may be an effective way to reduce the global burden of mental disorders. We conducted a systematic review of reviews to examine the evidence base for interventions that map onto the UN SDGs and seek to improve mental health through targeting known social determinants of mental disorders. We included 101 reviews in the final review, covering demographic, economic, environmental events, neighborhood, and sociocultural domains. This review presents interventions with the strongest evidence base for the prevention of mental disorders and highlights synergies where addressing the UN SDGs can be beneficial for mental health.

Implementation Feasibility and Hidden Costs of Statewide Scaling of Evidence-Based Therapies for Children and Adolescents

OBJECTIVE

State mental health systems are retraining their workforces to deliver services supported by research. Knowledge about evidence-based therapies (EBTs) for child and adolescent disorders is robust, but the feasibility of their statewide scaling has not been examined. The authors reviewed implementation feasibility for 12 commonly used EBTs, defining feasibility for statewide scaling as an EBT having at least one study documenting acceptability, facilitators and barriers, or fidelity; at least one study with a racially and ethnically diverse sample; an entity for training, certification, or licensing; and fiscal data reflecting the costs of implementation.

METHODS

The authors reviewed materials for 12 EBTs being scaled in New York State and conducted a literature review with search terms relevant to their implementation. Costs and certification information were supplemented by discussions with treatment developers and implementers.

RESULTS

All 12 EBTs had been examined for implementation feasibility, but only three had been examined for statewide scaling. Eleven had been studied in populations reflecting racial-ethnic diversity, but few had sufficient power for subgroup analyses to demonstrate effectiveness with these samples. All had certifying or licensing entities. The per-clinician costs of implementation ranged from $500 to $3,500, with overall ongoing costs ranging from $100 to $6,000. A fiscal analysis of three EBTs revealed hidden costs ranging from $5,000 to $24,000 per clinician, potentially limiting sustainability.

CONCLUSIONS

The evidence necessary for embedding EBTs in state systems has notable gaps that may hinder sustainability. Research-funding agencies should prioritize studies that focus on the practical aspects of scaling to assist states as they retrain their workforces.

How can a behavioral economics lens contribute to implementation science?

BACKGROUND

Implementation science in health is an interdisciplinary field with an emphasis on supporting behavior change required when clinicians and other actors implement evidence-based practices within organizational constraints. Behavioral economics has emerged in parallel and works towards developing realistic models of how humans behave and categorizes a wide range of features of choices that can influence behavior. We argue that implementation science can be enhanced by the incorporation of approaches from behavioral economics. Main body First, we provide a general overview of implementation science and ways in which implementation science has been limited to date. Second, we review principles of behavioral economics and describe how concepts from BE have been successfully applied to healthcare including nudges deployed in the electronic health record. For example, de-implementation of low-value prescribing has been supported by changing the default in the electronic health record. We then describe what a behavioral economics lens offers to existing implementation science theories, models and frameworks, including rich and realistic models of human behavior, additional research methods such as pre-mortems and behavioral design, and low-cost and scalable implementation strategies. We argue that insights from behavioral economics can guide the design of implementation strategies and the interpretation of implementation studies. Key objections to incorporating behavioral economics are addressed, including concerns about sustainment and at what level the strategies work.

CONCLUSION

Scholars should consider augmenting implementation science theories, models, and frameworks with relevant insights from behavioral economics. By drawing on these additional insights, implementation scientists have the potential to boost efforts to expand the provision and availability of high quality care.

Organizing the dissemination and implementation field: who are we, what are we doing, and how should we do it?

Two decades into its tenure as a field, dissemination and implementation (D&I) scientists have begun a process of self-reflection, illuminating a missed opportunity to bridge the gap between research and practice-one of the field's foundational objectives. In this paper, we, the authors, assert the research-to-practice gap has persisted, in part due to an inadequate characterization of roles, functions, and processes within D&I. We aim to address this issue, and the rising tension between D&I researchers and practitioners, by proposing a community-centered path forward that is grounded in equity.We identify key players within the field and characterize their unique roles using the translational science spectrum, a model originally developed in the biomedical sciences to help streamline the research-to-practice process, as a guide. We argue that the full translational science spectrum, from basic science research, or "T0," to translation to community, or "T4," readily applies within D&I and that in using this framework to clarify roles, functions, and processes within the field, we can facilitate greater collaboration and respect across the entire D&I research-to-practice continuum. We also highlight distinct opportunities (e.g., changes to D&I scientific conference structures) to increase regular communication and engagement between individuals whose work sits at different points along the D&I translational science spectrum that can accelerate our efforts to close the research-to-practice gap and achieve the field's foundational objectives.

Healing Right Way randomised control trial enhancing rehabilitation services for Aboriginal people with brain injury in Western Australia: translation principles and activities

Background This report provides the theory, method and practice of culturally secure translation and knowledge exchange in the Healing Right Way Clinical Trial (2017-2022), outlining activities to date. Healing Right Way was a stepped wedge cluster randomised controlled trial conducted in Western Australia, aimed at enhancing rehabilitation services and quality of life for Aboriginal Australians following acquired brain injury. The trial translation plan was aspirational and action-oriented, with its implementation iterative and ongoing. Translational activities aimed to inform service and research planning for Aboriginal people with brain injury. Situated in the intercultural space, the work guards against undertaking activities that are monocultural, colonial and appropriating in favour of work that is authentically viewed through the dual lens of whiteness and Aboriginal and Torres Strait Islander ways of knowing, being and doing, and is strengths-based. Methods Three translational and knowledge exchange components were identified, relating to the role of Aboriginal Brain Injury Coordinators, cultural training of hospital staff and the research process itself. Knowledge plans were developed for key audiences, with potential translation products to be monitored for ongoing impact. Results Results demonstrate that translational and knowledge exchange were iteratively embedded throughout the trial life cycle. Data sources included community engagement, partnership meetings and interviews. Activities involved presentations to diverse audiences including bureaucrats, community and participants. Conclusions This report provides a snapshot of the first translation knowledge exchange plan and activities constructed in relation to brain injury rehabilitation services for Aboriginal people. Challenges encountered, as well as successes to date, are discussed.

Enhancing Impact: A Call to Action for Equitable Implementation Science

Despite investments in evidence-based interventions and Implementation Science, most evidence-based interventions are not widely or routinely adopted, delivered, or sustained in many real-world community and healthcare settings. This gap is even greater in settings and populations experiencing numerous social and structural barriers to health, with important implications for persistent patterns in health inequities. In this Viewpoint, as part of a Special Issue on Advancing the Adaptability of Chronic Disease Prevention and Management through Implementation Science, we outline seven calls to action for the field of Implementation Science, with the goal of encouraging researchers, practitioners, and funders to be more intentional and accountable in applying Implementation Science to have greater impact on promoting health equity. Calls to action include (1) enhance public health, community, and multi-sectoral partnerships to promote health equity and equitable implementation; (2) revisit and build the evidence base needed to promote health equity and impact at multiple levels; (3) prioritize focus on policy development, dissemination, and implementation; (4) be agile and responsive in application of Implementation Science frameworks, processes, and methods; (5) identify and redefine meaningful metrics for equity and impact; (6) disseminate scientific evidence and research to a diverse range of partners and potential beneficiaries; and (7) extend focus on de-implementation, mis-implementation, and sustainability which are central to enhancing health equity. Additionally, we outline why a focus on prevention and public health is essential to making progress towards health equity in Implementation Science, summarize important advancements that the field has made towards making equity more foundational, and pose important research questions to enhance equitable impact of work in this area.

Implementation science for cancer control: One center's experience addressing context, adaptation, equity, and sustainment

Implementation science (IS) has great potential to enhance the frequency, speed, and quality of the translation of evidence-based programs, policies, products, and guidelines into practice. Progress has been made, but with some notable exceptions, this promise has not been achieved for cancer prevention and control. We discuss five interrelated but conceptually distinct, crosscutting issues important to accelerate IS for cancer prevention and control and how our Colorado Implementation Science Center in Cancer Control (COISC3) addressed these issues. These needs and opportunities include more fully addressing changing, multi-level context; guiding rapid, iterative adaptations; evaluating innovative approaches to engagement and health equity; greater attention to costs and economic issues; and sustainability. We summarize conceptual issues; evaluation needs and capacity building activities and then provide examples of how our IS center addressed these five needs for cancer prevention and control. We discuss changes made to address priorities of (i) guiding adaptations of implementation strategies to address changing context and (ii) working on issues identified and prioritized by our primary care partners rather than the research team. We conclude with discussion of lessons learned, limitations, and directions for future research and practice in IS to enhance cancer prevention and control as well as translational behavioral medicine more generally.

[Implementing systematically: To what extent can implementation science make an educational contribution to health care practice?]

BACKGROUND

Implementation science is currently struggling with the challenge to translate its findings into implementation practice in health care. Therefore, it is of interest to explore to what extent advanced trainings are relevant for health care practice to support knowledge circulation, and what needs to be considered in their design.

DESIGN OF EDUCATIONAL INITIATIVES AND TRAINING

Internationally, numerous initiatives have emerged in recent years to make implementation science more accessible to health care implementers. They focus, in particular, on the development of curricula for actors in health care practice and the widespread implementation of these qualification programs. In German-speaking countries, the interest in health services research is particularly evident at the level of Master's degree programs to make implementation science accessible to health care. However, these efforts need to be complemented by a participatory, needs-oriented and theory-based approach to advanced training for implementers in health care practice. This will enable the required competencies to be addressed on different levels of training and the further development of training formats.

DISCUSSION

Recent literature shows that implementers in German-speaking countries can benefit from a training program tailored to their needs. International experience in the field of advanced training in implementation science for health care practice allows us to derive essential components for the development of advanced training for the German-speaking region.

Leveraging artificial intelligence to advance implementation science: potential opportunities and cautions

BACKGROUND

The field of implementation science was developed to address the significant time delay between establishing an evidence-based practice and its widespread use. Although implementation science has contributed much toward bridging this gap, the evidence-to-practice chasm remains a challenge. There are some key aspects of implementation science in which advances are needed, including speed and assessing causality and mechanisms. The increasing availability of artificial intelligence applications offers opportunities to help address specific issues faced by the field of implementation science and expand its methods.

MAIN TEXT

This paper discusses the many ways artificial intelligence can address key challenges in applying implementation science methods while also considering potential pitfalls to the use of artificial intelligence. We answer the questions of "why" the field of implementation science should consider artificial intelligence, for "what" (the purpose and methods), and the "what" (consequences and challenges). We describe specific ways artificial intelligence can address implementation science challenges related to (1) speed, (2) sustainability, (3) equity, (4) generalizability, (5) assessing context and context-outcome relationships, and (6) assessing causality and mechanisms. Examples are provided from global health systems, public health, and precision health that illustrate both potential advantages and hazards of integrating artificial intelligence applications into implementation science methods. We conclude by providing recommendations and resources for implementation researchers and practitioners to leverage artificial intelligence in their work responsibly.

CONCLUSIONS

Artificial intelligence holds promise to advance implementation science methods ("why") and accelerate its goals of closing the evidence-to-practice gap ("purpose"). However, evaluation of artificial intelligence's potential unintended consequences must be considered and proactively monitored. Given the technical nature of artificial intelligence applications as well as their potential impact on the field, transdisciplinary collaboration is needed and may suggest the need for a subset of implementation scientists cross-trained in both fields to ensure artificial intelligence is used optimally and ethically.

Partnership-building considerations for implementation science in learning health systems: a case study of the Implementation Science Collaborative in Alberta, Canada

Introduction

Implementation of health innovations is inherently collaborative, requiring trans-sectoral partnerships between implementation researchers, innovation teams, and implementation practitioners. Implementation science has been shown to improve implementation successes; however, challenges that hinder partnerships to advance implementation science continue to persist. Using a whole-system approach to assess and respond to implementation science partnership barriers may shed light on effective responses.

Methods

We conducted a case study of Alberta's learning health system, using semi-structured group and individual interviews to create a nuanced understanding of the considerations required for implementation research collaborations. We interviewed 53 participants representing 21 offices in the health system, academia, professional associations, and government who regularly plan, evaluate, and/or study health system implementation initiatives in Alberta. Using the Partnership Model for Research Capacity Building, we identified current facilitators and challenges for partnerships for conducting and using implementation science, at different levels of Alberta's health-research ecosystem.

Results

Alberta's healthcare system is well set up to readily embed intervention effectiveness and efficacy research. Infrastructure was also in place to strengthen implementation practice. However, weaknesses around exchanging knowledge and skills, providing feedback and mentoring, and accommodating diversity affected the ability of both individuals and teams to build implementation science capacity. Without this capacity, teams could not participate in embedded implementation research collaborations. We report the response of the Alberta Strategy for Patient-Oriented Research SUPPORT Unit to these barriers to provide practical guidance on various program options to strengthen individual- and organization-level implementation science capacity.

Discussion

This study applied a whole-system approach to assess factors across Alberta's health-research ecosystem, which affect partnerships to advance implementation science. Our findings illustrated that partnership considerations go beyond interpersonal factors and include system-wide considerations. With the results, health organization leaders have (1) a method for assessing organizational capability to readily embed implementation research and (2) a catalog of potential responses to create conditions to readily engage with implementation science in their day-to-day implementation processes.

What do you think it means? Using cognitive interviewing to improve measurement in implementation science: description and case example

Pragmatic measures are essential to evaluate the implementation of evidence-based interventions. Cognitive interviewing, a qualitative method that collects partner feedback throughout measure development, is particularly useful for developing pragmatic implementation measures. Measure developers can use cognitive interviewing to increase a measure's fit within a particular implementation context. However, cognitive interviewing is underused in implementation research, where most measures remain "homegrown" and used for single studies. We provide a rationale for using cognitive interviewing in implementation science studies and illustrate its use through a case example employing cognitive interviewing to inform development of a measurement-based care protocol for implementation in opioid treatment programs. Applications of cognitive interviewing, including developing a common language with partners and collecting multi-level feedback on assessment procedures, to improve measurement in implementation science are discussed.

Implementation of a Stepped Care Program to Address Emotional Recovery among Traumatic Injury Patients

BACKGROUND

Annually, over 600,000 adults served in US trauma centers (≥20%) develop posttraumatic stress disorder (PTSD) and/or depression in the first year after injury. American College of Surgeons guidelines include screening and addressing mental health recovery in trauma centers. Yet, many trauma centers do not monitor and address mental health recovery, and it is a priority to learn how to implement evidence-informed mental health programs in trauma centers.

STUDY DESIGN

This report describes our application of the Exploration, Preparation, Implementation, Sustainment model to implement the Trauma Resilience and Recovery Program (TRRP) in 3 Level I and II trauma centers to address patients' mental health needs. TRRP is a scalable and sustainable stepped model of care-one of the few in the US-that provides early intervention and direct services after traumatic injury.

RESULTS

Trauma centers are well positioned to accelerate patients' mental health recovery via early identification, education, screening, and referrals to mental health agencies that provide best-practice care. We found that TRRP was acceptable to the 3 partnering trauma centers we studied. Early engagement of patient, provider, and hospital administration stakeholders enhanced buy-in during the early stages of the implementation process and promoted sustainability. Active processes to support monitoring, evaluation, and adaptation were critical.

CONCLUSIONS

Our work demonstrates the feasibility of implementing and adapting TRRP, a cost-efficient and sustainable stepped care intervention, in Level I and II trauma centers. Several factors should be carefully considered by trauma centers seeking to integrate behavioral health interventions into their trauma program.

Essential content for teaching implementation practice in healthcare: a mixed-methods study of teams offering capacity-building initiatives

BACKGROUND

Applying the knowledge gained through implementation science can support the uptake of research evidence into practice; however, those doing and supporting implementation (implementation practitioners) may face barriers to applying implementation science in their work. One strategy to enhance individuals' and teams' ability to apply implementation science in practice is through training and professional development opportunities (capacity-building initiatives). Although there is an increasing demand for and offerings of implementation practice capacity-building initiatives, there is no universal agreement on what content should be included. In this study we aimed to explore what capacity-building developers and deliverers identify as essential training content for teaching implementation practice.

METHODS

We conducted a convergent mixed-methods study with participants who had developed and/or delivered a capacity-building initiative focused on teaching implementation practice. Participants completed an online questionnaire to provide details on their capacity-building initiatives; took part in an interview or focus group to explore their questionnaire responses in depth; and offered course materials for review. We analyzed a subset of data that focused on the capacity-building initiatives' content and curriculum. We used descriptive statistics for quantitative data and conventional content analysis for qualitative data, with the data sets merged during the analytic phase. We presented frequency counts for each category to highlight commonalities and differences across capacity-building initiatives.

RESULTS

Thirty-three individuals representing 20 capacity-building initiatives participated. Study participants identified several core content areas included in their capacity-building initiatives: (1) taking a process approach to implementation; (2) identifying and applying implementation theories, models, frameworks, and approaches; (3) learning implementation steps and skills; (4) developing relational skills. In addition, study participants described offering applied and pragmatic content (e.g., tools and resources), and tailoring and evolving the capacity-building initiative content to address emerging trends in implementation science. Study participants highlighted some challenges learners face when acquiring and applying implementation practice knowledge and skills.

CONCLUSIONS

This study synthesized what experienced capacity-building initiative developers and deliverers identify as essential content for teaching implementation practice. These findings can inform the development, refinement, and delivery of capacity-building initiatives, as well as future research directions, to enhance the translation of implementation science into practice.

Matching barriers and facilitators to implementation strategies: recommendations for community settings

BACKGROUND

Implementation science aims to improve the integration of evidence-based interventions in real-world settings. While its methods and models could potentially apply to any field with evidence-based interventions, most research thus far has originated in clinical settings. Community settings often have fewer resources, missions beyond health, and a lack of support and expertise to implement evidence-based interventions when compared to many clinical settings. Thus, selecting and tailoring implementation strategies in community settings is particularly challenging, as existing compilations are primarily operationalized through clinical setting terminology. In this debate, we (1) share the process of using an existing match tool to select implementation strategies to increase uptake of nutrition and physical activity policy, systems, and environment interventions in community settings and (2) discuss the challenges of this process to argue that selecting implementation strategies in community settings has limited transferability from clinical settings and may require a unique implementation strategy compilation and pragmatic matching tool.

MATCHING BARRIERS TO IMPLEMENTATION STRATEGIES

The impetus for this debate paper came from our work selecting implementation strategies to improve the implementation and eventual scaling of nutrition and physical activity policy, systems, and environment interventions in a community settings. We conducted focus groups with practitioners and used the Consolidated Framework for Implementation Research-Expert Recommendations for Implementing Change match tool to select potential implementation strategies to overcome prominent barriers. There was limited congruence between tool outputs and optimal strategies, which may in part be due to differences in context between clinical and community settings. Based on this, we outline needs and recommendations for developing a novel and pragmatic matching tool for researchers and practitioners in community settings.

CONCLUSIONS

More work is needed to refine the implementation barrier-strategy matching process to ensure it is relevant, rapid, and rigorous. As leading implementation strategy scholars note, as more researchers document contextual factors and strategies selected to address them, the knowledge base will increase, and refined mapping processes can emerge.

Guideline evaluation and implementation mechanisms in school health services (GuideMe): protocol for a hybrid randomized factorial trial

BACKGROUND

Norwegian school health services received a national best-practice guideline in 2017. To promote healthy life skills and identify adolescents needing support, the guideline includes strong recommendations for individual consultations with all 8th graders and increased collaboration with schools. To help implement the recommendations, a blended implementation strategy (SchoolHealth) was co-created with school nurses, students, and stakeholders. SchoolHealth consists of three implementation elements: Digital dialog and administration tool (audit and feedback +), Dialog support (external consultation), and Collaboration materials (targeted dissemination). This hybrid study will test the main and combined effects of the elements on guideline fidelity and effectiveness.

METHODS

The GuideMe study is a factorial cluster randomized controlled trial examining SchoolHealth's effectiveness on guideline fidelity and guideline effectiveness goals. Forty Norwegian secondary schools will be randomized to eight different combinations of the elements in SchoolHealth. Participants will include school nurses and school personnel from these schools, and 8th grade students (n = 1200). Primary outcomes are school nurses' fidelity to the guidelines and student's ability to cope with their life (i.e., health literacy, positive health behaviors and self-efficacy). Quantitative methods will be used to test effects and mechanisms, while mixed- and qualitative methods will be used to explore mechanisms, experiences, and other phenomena in depth. Participants will complete digital questionnaires at the start and end of the schoolyear, and after the consultation during the schoolyear. The study will run in two waves, each lasting for one school year. The multifactorial design allows testing of interactions and main effects due to equal distribution of all factors within each main effect. Sustainment and scale-up of optimized SchoolHealth elements using national infrastructure are simultaneously prepared.

DISCUSSION

The study will investigate possible effects of the implementation elements in isolation and in combination, and hypothesized implementation mechanisms. In-depth study of user experiences will inform improvements to elements in SchoolHealth. The results will yield causal knowledge about implementation strategies and the mechanisms through which they assert effects. Mixed-methods will provide insights into how and when the elements work. Optimizing guideline implementation elements can support adolescents in a crucial life phase.

TRAIL REGISTRATION

ISRCTN24173836. Registration date 8 August 2022.

Evaluation of a brief virtual implementation science training program: the Penn Implementation Science Institute

BACKGROUND

To meet the growing demand for implementation science expertise, building capacity is a priority. Various training opportunities have emerged to meet this need. To ensure rigor and achievement of specific implementation science competencies, it is critical to systematically evaluate training programs.

METHODS

The Penn Implementation Science Institute (PennISI) offers 4 days (20 h) of virtual synchronous training on foundational and advanced topics in implementation science. Through a pre-post design, this study evaluated the sixth PennISI, delivered in 2022. Surveys measures included 43 implementation science training evaluation competencies grouped into four thematic domains (e.g., items related to implementation science study design grouped into the "design, background, and rationale" competency category), course-specific evaluation criteria, and open-ended questions to evaluate change in knowledge and suggestions for improving future institutes. Mean composite scores were created for each of the competency themes. Descriptive statistics and thematic analysis were completed.

RESULTS

One hundred four (95.41% response rate) and 55 (50.46% response rate) participants completed the pre-survey and post-survey, respectively. Participants included a diverse cohort of individuals primarily affiliated with US-based academic institutions and self-reported as having novice or beginner-level knowledge of implementation science at baseline (81.73%). In the pre-survey, all mean composite scores for implementation science competencies were below one (i.e., beginner-level). Participants reported high value from the PennISI across standard course evaluation criteria (e.g., mean score of 3.77/4.00 for overall quality of course). Scores for all competency domains increased to a score between beginner-level and intermediate-level following training. In both the pre-survey and post-survey, competencies related to "definition, background, and rationale" had the highest mean composite score, whereas competencies related to "design and analysis" received the lowest score. Qualitative themes offered impressions of the PennISI, didactic content, PennISI structure, and suggestions for improvement. Prior experience with or knowledge of implementation science influenced many themes.

CONCLUSIONS

This evaluation highlights the strengths of an established implementation science institute, which can serve as a model for brief, virtual training programs. Findings provide insight for improving future program efforts to meet the needs of the heterogenous implementation science community (e.g., different disciplines and levels of implementation science knowledge). This study contributes to ensuring rigorous implementation science capacity building through the evaluation of programs.

Exploring stakeholder perspectives regarding the implementation of competency-based medical education: a qualitative descriptive study

Introduction

Competency-based medical education (CBME) offers perceived advantages and benefits for postgraduate medical education (PGME) and the training of competent physicians. The purpose of our study was to gain insights from those involved in implementing CBME in two residency programs to inform ongoing implementation practices.

Methods

We conducted a qualitative descriptive study to explore the perspectives of multiple stakeholders involved in the implementation of CBME in two residency programs (the first cohort) to launch the Royal College's Competence by Design model at one Canadian university. Semi-structured interviews were conducted with 17 participants across six stakeholder groups including residents, department chairs, program directors, faculty, medical educators, and program administrators. Data collection and analysis were iterative and reflexive to enhance the authenticity of the results.

Results

The participants' perspectives organized around three key themes including: a) contextualizing curriculum and assessment practices with educational goals of CBME, b) coordinating new administrative requirements to support implementation, and c) adaptability toward a competency-based program structure, each with sub-themes.

Conclusion

By eliciting the perspectives of different stakeholder groups who experienced the implementation processes, we developed a common understanding regarding facilitators and challenges for program directors, program administrators and educational leaders across PGME. Results from our study contribute to the scholarly conversation regarding the key aspects related to CBME implementation and serve to inform its ongoing development and application in various educational contexts.

Planning and implementing practice changes in Ontario maternal-newborn hospital units: a secondary qualitative analysis

BACKGROUND

Moving evidence into practice is complex, and pregnant and birthing people and their infants do not always receive care that aligns with the best available evidence. Implementation science can inform how to effectively move evidence into practice. While there are a growing number of examples of implementation science being studied in maternal-newborn care settings, it remains unknown how real-world teams of healthcare providers and leaders approach the overall implementation process when making practice changes. The purpose of this study was to describe maternal-newborn hospital teams' approaches to implementing practice changes. We aimed to identify what implementation steps teams take (or not) and identify strengths and potential areas for improvement based on best practices in implementation science.

METHODS

We conducted a supplementary qualitative secondary analysis of 22 interviews completed in 2014-2015 with maternal-newborn nursing leaders in Ontario, Canada. We used directed content analysis to code the data to seven steps in an implementation framework (Implementation Roadmap): identify the problem and potential best practice; assemble local evidence; select and customize best practice; discover barriers and drivers; tailor implementation strategies; field-test, plan evaluation, prepare to launch; launch, evaluate, and sustain. Frequency counts are presented for each step.

RESULTS

Participants reported completing a median of 4.5 of 7 Implementation Roadmap steps (range = 3-7), with the most common being identifying a practice problem. Other steps were described less frequently (e.g., selecting and adapting evidence, field-testing, outcome evaluation) or discussed frequently but not optimally (e.g., barriers assessment). Participants provided examples of how they engaged point-of-care staff throughout the implementation process, but provided fewer examples of engaging pregnant and birthing people and their families. Some participants stated they used a formal framework or process to guide their implementation process, with the most common being quality improvement approaches and tools.

CONCLUSIONS

We identified variability across the 22 hospitals in the implementation steps taken. While we observed many strengths, we also identified areas where further support may be needed. Future work is needed to create opportunities and resources to support maternal-newborn healthcare providers and leaders to apply principles and tools from implementation science to their practice change initiatives.

Improving Sexual Assault and Sexual Harassment Prevention from the Bottom-up: a Pilot of Getting To Outcomes in the US Military

While the Department of Defense (DoD) has given increased attention and priority to preventing sexual assault and sexual harassment (SA/SH), it remains a problem. To build its prevention capacity, DoD piloted Getting To Outcomes® (GTO®) from 2019 to 2022 at 10 military installations. GTO is an evidence-based planning and implementation support that has been used in many civilian contexts but has only recently been adapted for military SA/SH. The purpose of this study was to describe GTO use, identify its benefits and challenges, and discuss lessons the GTO effort yielded for prevention more broadly using a framework of organizational and program-level capacities needed for successful prevention in the military context, called the Prevention Evaluation Framework (PEF). GTO was piloted with 10 military installations ("sites") representing all Military Services, plus the Coast Guard and National Guard. GTO is comprised of a written guide, training, and ongoing coaching. The pilot's goal was for each site to use GTO to implement a SA/SH prevention program twice. Participants from each site were interviewed and data was collected on GTO steps completed, whether GTO spurred new evaluation activities and collaborations, and the degree of leadership support for GTO. Most sites completed all GTO steps at least once. Interviews showed that DoD participants believe GTO improved prevention understanding, planning, and evaluation capacity; strengthened confidence in chosen programs; and helped sites tailor programs to the military context. Barriers were the complexity of GTO, DoD personnel turnover, and the disruption that the COVID pandemic caused in sexual assault prevention program delivery. Many respondents were unsure if they would continue all of GTO after the coaching ended, but many believed they would continue at least some parts. According to the PEF, the GTO pilot revealed several additional prevention system gaps (e.g., need for leadership support) and changes needed to GTO (e.g., stronger leader and champion engagement), to support quality prevention. The military and other large organizations will need to focus on these issues to ensure prevention implementation and evaluation are conducted with quality.

The iPRISM webtool: an interactive tool to pragmatically guide the iterative use of the Practical, Robust Implementation and Sustainability Model in public health and clinical settings

BACKGROUND

To increase uptake of implementation science (IS) methods by researchers and implementers, many have called for ways to make it more accessible and intuitive. The purpose of this paper is to describe the iPRISM webtool (Iterative, Practical, Robust Implementation and Sustainability Model) and how this interactive tool operationalizes PRISM to assess and guide a program's (a) alignment with context, (b) progress on pragmatic outcomes, (c) potential adaptations, and (d) future sustainability across the stages of the implementation lifecycle.

METHODS

We used an iterative human-centered design process to develop the iPRISM webtool.

RESULTS

We conducted user-testing with 28 potential individual and team-based users who were English and Spanish speaking from diverse settings in various stages of implementing different types of programs. Users provided input on all aspects of the webtool including its purpose, content, assessment items, visual feedback displays, navigation, and potential application. Participants generally expressed interest in using the webtool and high likelihood of recommending it to others. The iPRISM webtool guides English and Spanish-speaking users through the process of iteratively applying PRISM across the lifecycle of a program to facilitate systematic assessment and alignment with context. The webtool summarizes assessment responses in graphical and tabular displays and then guides users to develop feasible and impactful adaptations and corresponding action plans. Equity considerations are integrated throughout.

CONCLUSIONS

The iPRISM webtool can intuitively guide individuals and teams from diverse settings through the process of using IS methods to iteratively assess and adapt different types of programs to align with the context across the implementation lifecycle. Future research and application will continue to develop and evaluate this IS resource.

Four very basic ways to think about policy in implementation science

BACKGROUND

Policy is receiving increasing attention in the field of implementation science. However, there remains a lack of clear, concise guidance about how policy can be conceptualized in implementation science research. Building on Curran's article "Implementation science made too simple"-which defines "the thing" as the intervention, practice, or innovation in need of implementation support-we offer a typology of four very basic ways to conceptualize policy in implementation science research. We provide examples of studies that have conceptualized policy in these different ways and connect aspects of the typology to established frameworks in the field. The typology simplifies and refines related typologies in the field. Four very basic ways to think about policy in implementation science research. 1) Policy as something to adopt: an evidence-supported policy proposal is conceptualized as "the thing" and the goal of research is to understand how policymaking processes can be modified to increase adoption, and thus reach, of the evidence-supported policy. Policy-focused dissemination research is well-suited to achieve this goal. 2) Policy as something to implement: a policy, evidence-supported or not, is conceptualized as "the thing" and the goal of research is to generate knowledge about how policy rollout (or policy de-implementation) can be optimized to maximize benefits for population health and health equity. Policy-focused implementation research is well-suited to achieve this goal. 3) Policy as context to understand: an evidence-supported intervention is "the thing" and policies are conceptualized as a fixed determinant of implementation outcomes. The goal of research is to understand the mechanisms through which policies affect implementation of the evidence-supported intervention. 4) Policy as strategy to use: an evidence-supported intervention is "the thing" and policy is conceptualized as a strategy to affect implementation outcomes. The goal of research is to understand, and ideally test, how policy strategies affect implementation outcomes related to the evidence-supported intervention.

CONCLUSION

Policy can be conceptualized in multiple, non-mutually exclusive ways in implementation science. Clear conceptualizations of these distinctions are important to advancing the field of policy-focused implementation science and promoting the integration of policy into the field more broadly.

Applying hybrid effectiveness-implementation studies in equity-centered policy implementation science

Policy implementation science (IS) is complex, dynamic, and fraught with unique study challenges that set it apart from biomedical or clinical research. One important consideration is the ways in which policy interacts with local contexts, such as power and social disadvantage (e.g., based on ability, race, class, sexual identity, geography). The complex nature of policy IS and the need for more intentional integration of equity principles into study approaches calls for creative adaptations to existing implementation science knowledge and guidance. Effectiveness-implementation hybrid studies were developed to enhance translation of clinical research by addressing research questions around the effectiveness of an intervention and its implementation in the same study. The original work on hybrid designs mainly focused on clinical experimental trials; however, over the last decade, researchers have applied it to a wide range of initiatives and contexts, including more widespread application in community-based studies. This perspectives article demonstrates how effectiveness-implementation hybrid studies can be adapted for and applied to equity-centered policy IS research. We draw upon principles of targeted universalism and Equity in Implementation Research frameworks to guide adaptations to hybrid study typologies, and suggest research and engagement activities to enhance equity considerations; for example, in the design and testing of implementing strategies. We also provide examples of equity-centered policy IS studies. As the field of policy IS rapidly evolves, these adapted hybrid type studies are offered to researchers as a starting guide.

Developing a single-session strategy for the implementation of take-home naloxone by community pharmacists using COM-B and design-thinking

Background

Despite the overwhelming evidence of its effectiveness, there is poor implementation of take-home naloxone by pharmacists. Barriers have been explored and mapped to the capability, opportunity, motivation-behaviour (COM-B) model of behaviour change, yet no theoretically informed implementation strategies that target known barriers have been developed. Single-session implementation strategies have been proposed as a simple, scalable way to target multiple barriers.

Methods

Qualitative participatory methods, incorporating design-thinking principles, were used to develop the key messages of a single-session implementation strategy. The key messages were drafted against COM-B mapped implementation barriers identified in the literature. A participatory workshop involving a pre-mortem exercise and incorporating design-thinking principles were used to refine the messages and generate methods for dissemination. Messages were mapped to interview questions to naturally illicit stories and delivered via storytelling from a pharmacist, a general practitioner, and a person with lived experience of using naloxone.

Results

A 3 minute 40 second video and a two-page printable infographic were developed and hosted on a website, with links to additional downloadable resources as a single-session implementation strategy. Email was the preferred method for receiving simple professional development communications, with social media also widely accessed.

Discussion

Implementation science, behavioural change theory, and participatory design methods are a complementary combination to develop implementation strategies. Some pharmacists questioned the participatory design approach to developing an implementation strategy, as it was outside of their comfort zone. However, the participatory process involving end-users resulted in unique ideas that are unlikely to have been generated using more traditional consultative approaches. The delivery as a single-session implementation strategy allows for widespread dissemination and delivery at scale.

Ten years of implementation outcomes research: a scoping review

BACKGROUND

Proctor and colleagues' 2011 paper proposed a taxonomy of eight implementation outcomes and challenged the field to address a research agenda focused on conceptualization, measurement, and theory building. Ten years later, this paper maps the field's progress in implementation outcomes research. This scoping review describes how each implementation outcome has been studied, research designs and methods used, and the contexts and settings represented in the current literature. We also describe the role of implementation outcomes in relation to implementation strategies and other outcomes.

METHODS

Arksey and O'Malley's framework for conducting scoping reviews guided our methods. Using forward citation tracing, we identified all literature citing the 2011 paper. We conducted our search in the Web of Science (WOS) database and added citation alerts sent to the first author from the publisher for a 6-month period coinciding with the WOS citation search. This produced 1346 titles and abstracts. Initial abstract screening yielded 480 manuscripts, and full-text review yielded 400 manuscripts that met inclusion criteria (empirical assessment of at least one implementation outcome).

RESULTS

Slightly more than half (52.1%) of included manuscripts examined acceptability. Fidelity (39.3%), feasibility (38.6%), adoption (26.5%), and appropriateness (21.8%) were also commonly examined. Penetration (16.0%), sustainability (15.8%), and cost (7.8%) were less frequently examined. Thirty-two manuscripts examined implementation outcomes not included in the original taxonomy. Most studies took place in healthcare (45.8%) or behavioral health (22.5%) organizations. Two-thirds used observational designs. We found little evidence of progress in testing the relationships between implementation strategies and implementation outcomes, leaving us ill-prepared to know how to achieve implementation success. Moreover, few studies tested the impact of implementation outcomes on other important outcome types, such as service systems and improved individual or population health.

CONCLUSIONS

Our review presents a comprehensive snapshot of the research questions being addressed by existing implementation outcomes literature and reveals the need for rigorous, analytic research and tests of strategies for attaining implementation outcomes in the next 10 years of outcomes research.

A mixed-method approach to generate and deliver rapid-cycle evaluation feedback: lessons learned from a multicenter implementation trial in pediatric surgery

BACKGROUND

Rapid-cycle feedback loops provide timely information and actionable feedback to healthcare organizations to accelerate implementation of interventions. We aimed to (1) describe a mixed-method approach for generating and delivering rapid-cycle feedback and (2) explore key lessons learned while implementing an enhanced recovery protocol (ERP) across 18 pediatric surgery centers.

METHODS

All centers are members of the Pediatric Surgery Research Collaborative (PedSRC, www.pedsrc.org ), participating in the ENhanced Recovery In CHildren Undergoing Surgery (ENRICH-US) trial. To assess implementation efforts, we conducted a mixed-method sequential explanatory study, administering surveys and follow-up interviews with each center's implementation team 6 and 12 months following implementation. Along with detailed notetaking and iterative discussion within our team, we used these data to generate and deliver a center-specific implementation report card to each center. Report cards used a traffic light approach to quickly visualize implementation status (green = excellent; yellow = needs improvement; red = needs significant improvement) and summarized strengths and opportunities at each timepoint.

RESULTS

We identified several benefits, challenges, and practical considerations for assessing implementation and using rapid-cycle feedback among pediatric surgery centers. Regarding potential benefits, this approach enabled us to quickly understand variation in implementation and corresponding needs across centers. It allowed us to efficiently provide actionable feedback to centers about implementation. Engaging consistently with center-specific implementation teams also helped facilitate partnerships between centers and the research team. Regarding potential challenges, research teams must still allocate substantial resources to provide feedback rapidly. Additionally, discussions and consensus are needed across team members about the content of center-specific feedback. Practical considerations include carefully balancing timeliness and comprehensiveness when delivering rapid-cycle feedback. In pediatric surgery, moreover, it is essential to actively engage all key stakeholders (including physicians, nurses, patients, caregivers, etc.) and adopt an iterative, reflexive approach in providing feedback.

CONCLUSION

From a methodological perspective, we identified three key lessons: (1) using a rapid, mixed method evaluation approach is feasible in pediatric surgery and (2) can be beneficial, particularly in quickly understanding variation in implementation across centers; however, (3) there is a need to address several methodological challenges and considerations, particularly in balancing the timeliness and comprehensiveness of feedback.

TRIAL REGISTRATION

NIH National Library of Medicine Clinical Trials.

CLINICALTRIALS

gov Identifier: NCT04060303. Registered August 7, 2019, https://clinicaltrials.gov/ct2/show/NCT04060303.

Community and partner engagement in dissemination and implementation research at the National Institutes of Health: an analysis of recently funded studies and opportunities to advance the field

BACKGROUND

As the focus has grown in recent years on both engaged research and dissemination and implementation (D&I) research, so too has federal funding to support these areas. The purpose of this analysis is to provide an overall perspective about the range of practices and approaches being used to engage partners in D&I research, with special attention to disparities-relevant research, and to identify gaps and opportunities in research funded by the US National Institutes of Health (NIH) in this space.

METHODS

This analysis examined a portfolio of active D&I research grants funded in fiscal years 2020 and 2021 across the NIH. Grant applications were deductively coded and summary statistics were calculated. Cross-tabulations were used to identify trends by engagement and disparities foci.

RESULTS

There were 103 grants included in the portfolio, of which 87% contained some form of community or partner engagement, and 50% of engaged grants were relevant to health disparities. Engagement was planned across the research continuum with each study engaging on average 2.5 different partner types. Consultation was the most common level of engagement (56%) while partnership was the least common (3%). On average, each study used 2.2 engagement strategies. Only 16% of grants indicated formally measuring engagement. Compared to non-disparities studies, disparities-relevant studies were about twice as likely to engage partners at the higher levels of partnership or collaboration (19% vs. 11%) and were also more likely to be conducted in community settings (26% vs. 5%).

CONCLUSIONS

Based on this portfolio analysis, D&I research appears to regularly integrate engagement approaches and strategies, though opportunities to deepen engagement and diversify who is engaged remain. This manuscript outlines several gaps in the portfolio and describes opportunities for increasing engagement to improve the quality of D&I research and application to advancing health equity. In addition, opportunities for leveraging the consistent and systematic application of engagement approaches and strategies to advance the science of engagement are discussed.

Connecting the science and practice of implementation - applying the lens of context to inform study design in implementation research

The saying "horses for courses" refers to the idea that different people and things possess different skills or qualities that are appropriate in different situations. In this paper, we apply the analogy of "horses for courses" to stimulate a debate about how and why we need to get better at selecting appropriate implementation research methods that take account of the context in which implementation occurs. To ensure that implementation research achieves its intended purpose of enhancing the uptake of research-informed evidence in policy and practice, we start from a position that implementation research should be explicitly connected to implementation practice. Building on our collective experience as implementation researchers, implementation practitioners (users of implementation research), implementation facilitators and implementation educators and subsequent deliberations with an international, inter-disciplinary group involved in practising and studying implementation, we present a discussion paper with practical suggestions that aim to inform more practice-relevant implementation research.

Integrating Implementation Science with Quality Improvement to Improve Perinatal Outcomes

Implementation science is an interdisciplinary field that seeks to contribute generalizable knowledge that can improve the translation of clinical evidence in routine care. To promote the integration of implementation science approaches with health care quality improvement, the authors offer a framework that links the Model for Improvement with implementation strategies and methods. Perinatal quality improvement teams can leverage the robust frameworks of implementation science to diagnose implementation barriers, select implementation strategies, and assess the strategies' contribution to improving care. Partnerships between implementation scientists and quality improvement teams could accelerate efforts by both groups to achieve measurable improvements in care.

How does facilitation in healthcare work? Using mechanism mapping to illuminate the black box of a meta-implementation strategy

BACKGROUND

Healthcare facilitation, an implementation strategy designed to improve the uptake of effective clinical innovations in routine practice, has produced promising yet mixed results in randomized implementation trials and has not been fully researched across different contexts.

OBJECTIVE

Using mechanism mapping, which applies directed acyclic graphs that decompose an effect of interest into hypothesized causal steps and mechanisms, we propose a more concrete description of how healthcare facilitation works to inform its further study as a meta-implementation strategy.

METHODS

Using a modified Delphi consensus process, co-authors developed the mechanistic map based on a three-step process. First, they developed an initial logic model by collectively reviewing the literature and identifying the most relevant studies of healthcare facilitation components and mechanisms to date. Second, they applied the logic model to write vignettes describing how facilitation worked (or did not) based on recent empirical trials that were selected via consensus for inclusion and diversity in contextual settings (US, international sites). Finally, the mechanistic map was created based on the collective findings from the vignettes.

FINDINGS

Theory-based healthcare facilitation components informing the mechanistic map included staff engagement, role clarification, coalition-building through peer experiences and identifying champions, capacity-building through problem solving barriers, and organizational ownership of the implementation process. Across the vignettes, engagement of leaders and practitioners led to increased socialization of the facilitator's role in the organization. This in turn led to clarifying of roles and responsibilities among practitioners and identifying peer experiences led to increased coherence and sense-making of the value of adopting effective innovations. Increased trust develops across leadership and practitioners through expanded capacity in adoption of the effective innovation by identifying opportunities that mitigated barriers to practice change. Finally, these mechanisms led to eventual normalization and ownership of the effective innovation and healthcare facilitation process.

IMPACT

Mapping methodology provides a novel perspective of mechanisms of healthcare facilitation, notably how sensemaking, trust, and normalization contribute to quality improvement. This method may also enable more efficient and impactful hypothesis-testing and application of complex implementation strategies, with high relevance for lower-resourced settings, to inform effective innovation uptake.

Refining Expert Recommendations for Implementing Change (ERIC) strategy surveys using cognitive interviews with frontline providers

BACKGROUND

The Expert Recommendations for Implementing Change (ERIC) compilation includes 73 defined implementation strategies clustered into nine content areas. This taxonomy has been used to track implementation strategies over time using surveys. This study aimed to improve the ERIC survey using cognitive interviews with non-implementation scientist clinicians.

METHODS

Starting in 2015, we developed and fielded annual ERIC surveys to evaluate liver care in the Veterans Health Administration (VA). We invited providers who had completed at least three surveys to participate in cognitive interviews (October 2020 to October 2021). Before the interviews, participants reviewed the complete 73-item ERIC survey and marked which strategies were unclear due to wording, conceptual confusion, or overlap with other strategies. They then engaged in semi-structured cognitive interviews to describe the experience of completing the survey and elaborate on which strategies required further clarification.

RESULTS

Twelve VA providers completed surveys followed by cognitive interviews. The "Engage Consumer" and "Support Clinicians" clusters were rated most highly in terms of conceptual and wording clarity. In contrast, the "Financial" cluster had the most wording and conceptual confusion. The "Adapt and Tailor to Context" cluster strategies were considered to have the most redundancy. Providers outlined ways in which the strategies could be clearer in terms of wording (32%), conceptual clarity (51%), and clarifying the distinction between strategies (51%).

CONCLUSIONS

Cognitive interviews with ERIC survey participants allowed us to identify and address issues with strategy wording, combine conceptually indistinct strategies, and disaggregate multi-barreled strategies. Improvements made to the ERIC survey based on these findings will ultimately assist VA and other institutions in designing, evaluating, and replicating quality improvement efforts.

Integrating pragmatic and implementation science randomized clinical trial approaches: a PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) analysis

BACKGROUND

Over the past two decades, pragmatic and implementation science clinical trial research methods have advanced substantially. Pragmatic and implementation studies have natural areas of overlap, particularly relating to the goal of using clinical trial data to leverage health care system policy changes. Few investigations have addressed pragmatic and implementation science randomized trial methods development while also considering policy impact.

METHODS

The investigation used the PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) and PRECIS-2-Provider Strategies (PRECIS-2-PS) tools to evaluate the design of two multisite randomized clinical trials that targeted patient-level effectiveness outcomes, provider-level practice changes and health care system policy. Seven raters received PRECIS-2 training and applied the tools in the coding of the two trials. Descriptive statistics were produced for both trials, and PRECIS-2 wheel diagrams were constructed. Interrater agreement was assessed with the Intraclass Correlation (ICC) and Kappa statistics. The Rapid Assessment Procedure Informed Clinical Ethnography (RAPICE) qualitative approach was applied to understanding integrative themes derived from the PRECIS-2 ratings and an end-of-study policy summit.

RESULTS

The ICCs for the composite ratings across the patient and provider-focused PRECIS-2 domains ranged from 0.77 to 0.87, and the Kappa values ranged from 0.25 to 0.37, reflecting overall fair-to-good interrater agreement for both trials. All four PRECIS-2 wheels were rated more pragmatic than explanatory, with composite mean and median scores ≥ 4. Across trials, the primary intent-to-treat analysis domain was consistently rated most pragmatic (mean = 5.0, SD = 0), while the follow-up/data collection domain was rated most explanatory (mean range = 3.14-3.43, SD range = 0.49-0.69). RAPICE field notes identified themes related to potential PRECIS-2 training improvements, as well as policy themes related to using trial data to inform US trauma care system practice change; the policy themes were not captured by the PRECIS-2 ratings.

CONCLUSIONS

The investigation documents that the PRECIS-2 and PRECIS-2-PS can be simultaneously used to feasibly and reliably characterize clinical trials with patient and provider-level targets. The integration of pragmatic and implementation science clinical trial research methods can be furthered by using common metrics such as the PRECIS-2 and PRECIS-2-PS. Future study could focus on clinical trial policy research methods development.

TRIAL REGISTRATION

DO-SBIS ClinicalTrials.gov NCT00607620. registered on January 29, 2008. TSOS ClinicalTrials.gov NCT02655354, registered on July 27, 2015.

How a Power Differential Between Clinicians and Researchers Contributes to the Research-to-Practice Gap

PURPOSE

The purpose of this article is to contend that there is a power differential between researchers and clinicians where researchers are the primary creators of knowledge and clinicians are the primary consumers of knowledge. Rooted in a sociological model illustrating interacting levels of power at macro-, meso-, and microlevels, we argue that authentic research-practice partnerships and clinician-researcher collaborations can mitigate this power differential.

CONCLUSIONS

Clinicians and researchers in our field have vastly different responsibilities and priorities that impact our ability to work collaboratively to solve the most pressing problems for the clients we serve. Although some current research practices may reinforce a power differential causing clinicians to feel less than and to only consume knowledge, there are examples of successful collaborations where this power differential is mitigated. These examples can contribute meaningfully to the dialogue on research-practice partnerships, with the goal of improving outcomes for the clients we serve.

Tailoring dissemination strategies to increase evidence-informed policymaking for opioid use disorder treatment: study protocol

BACKGROUND

Policy is a powerful tool for systematically altering healthcare access and quality, but the research to policy gap impedes translating evidence-based practices into public policy and limits widespread improvements in service and population health outcomes. The US opioid epidemic disproportionately impacts Medicaid members who rely on publicly funded benefits to access evidence-based treatment including medications for opioid use disorder (MOUD). A myriad of misaligned policies and evidence-use behaviors by policymakers across federal agencies, state Medicaid agencies, and managed care organizations limit coverage of and access to MOUD for Medicaid members. Dissemination strategies that improve policymakers' use of current evidence are critical to improving MOUD benefits and reducing health disparities. However, no research describes key determinants of Medicaid policymakers' evidence use behaviors or preferences, and few studies have examined data-driven approaches to developing dissemination strategies to enhance evidence-informed policymaking. This study aims to identify determinants and intermediaries that influence policymakers' evidence use behaviors, then develop and test data-driven tailored dissemination strategies that promote MOUD coverage in benefit arrays.

METHODS

Guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework, we will conduct a national survey of state Medicaid agency and managed care organization policymakers to identify determinants and intermediaries that influence how they seek, receive, and use research in their decision-making processes. We will use latent class methods to empirically identify subgroups of agencies with distinct evidence use behaviors. A 10-step dissemination strategy development and specification process will be used to tailor strategies to significant predictors identified for each latent class. Tailored dissemination strategies will be deployed to each class of policymakers and assessed for their acceptability, appropriateness, and feasibility for delivering evidence about MOUD benefit design.

DISCUSSION

This study will illuminate key determinants and intermediaries that influence policymakers' evidence use behaviors when designing benefits for MOUD. This study will produce a critically needed set of data-driven, tailored policy dissemination strategies. Study results will inform a subsequent multi-site trial measuring the effectiveness of tailored dissemination strategies on MOUD benefit design and implementation. Lessons from dissemination strategy development will inform future research about policymakers' evidence use preferences and offer a replicable process for tailoring dissemination strategies.

Partnering with community-based organizations to improve equitable access to depression care for underserved older adults in the U.S.: Qualitative formative research

Background

Embedding evidenced-based programs (EBPs) like PEARLS outside clinical settings can help reduce inequities in access to depression care. Trusted community-based organizations (CBOs) reach older adults who are underserved; however, PEARLS adoption has been limited. Implementation science has tried to close this know-do gap, however a more intentional focus on equity is needed to engage CBOs. We partnered with CBOs to better understand their resources and needs in order to design more equitable dissemination and implementation (D&I) strategies to support PEARLS adoption.

Methods

We conducted 39 interviews with 24 current and potential adopter organizations and other partners (February-September 2020). CBOs were purposively sampled for region, type, and priority older populations experiencing poverty (communities of color, linguistically diverse, rural). Using a social marketing framework, our guide explored barriers, benefits and process for PEARLS adoption; CBO capacities and needs; PEARLS acceptability and adaptations; and preferred communication channels. During COVID-19, interviews also addressed remote PEARLS delivery and changes in priorities. We conducted thematic analysis of transcripts using the rapid framework method to describe the needs and priorities of older adults who are underserved and the CBOs that engage them, and strategies, collaborations, and adaptations to integrate depression care in these contexts.

Results

During COVID-19, older adults relied on CBO support for basic needs such as food and housing. Isolation and depression were also urgent issues within communities, yet stigma remained for both late-life depression and depression care. CBOs wanted EBPs with cultural flexibility, stable funding, accessible training, staff investment, and fit with staff and community needs and priorities. Findings guided new dissemination strategies to better communicate how PEARLS is appropriate for organizations that engage older adults who are underserved, and what program components are core and what are adaptable to better align with organizations and communities. New implementation strategies will support organizational capacity-building through training and technical assistance, and matchmaking for funding and clinical support.

Discussion

Findings support CBOs as appropriate depression care providers for older adults who are underserved, and suggest changes to communications and resources to better fit EBPs with the resources and needs of organizations and older adults. We are currently partnering with organizations in California and Washington to evaluate whether and how these D&I strategies increase equitable access to PEARLS for older adults who are underserved.

Developing a practice-driven research agenda in implementation science: Perspectives from experienced implementation support practitioners

Background

Attention is being placed on the "ironic gap" or "secondary" research-to-practice gap in the field of implementation science. Among several challenges posited to exacerbate this research-to-practice gap, we call attention to one challenge in particular-the relative dearth of implementation research that is tethered intimately to the lived experiences of implementation support practitioners (ISPs). The purpose of this study is to feature a qualitative approach to engaging with highly experienced ISPs to inform the development of a practice-driven research agenda in implementation science. In general, we aim to encourage ongoing empirical inquiry that foregrounds practice-driven implementation research questions.

Method

Our analytic sample was comprised of 17 professionals in different child and family service systems, each with long-term experience using implementation science frameworks to support change efforts. Data were collected via in-depth, semi-structured interviews. Our analysis followed a qualitative content analysis approach. Our focal conceptual category centered on the desired areas of future research highlighted by respondents, with subcategories reflecting subsets of related research question ideas.

Results

Interviews yielded varying responses that could help shape a practice-driven research agenda for the field of implementation science. The following subcategories regarding desired areas for future research were identified in respondents' answers: (a) stakeholder engagement and developing trusting relationships, (b) evidence use, (c) workforce development, and (d) cost-effective implementation.

Conclusions

There is significant promise in bringing implementation research and implementation practice together more closely and building a practice-informed research agenda to shape implementation science. Our findings point not only to valuable practice-informed gaps in the literature that could be filled by implementation researchers, but also topics for which dissemination and translation efforts may not have yielded optimal reach. We also highlight the value in ISPs bolstering their own capacity for engaging with the implementation science literature to the fullest extent possible.

Proceedings from the Second Annual Conference of the Norwegian Network for Implementation Research

In 2019, Norwegian implementation researchers formed a network to promote implementation research and practice in the Norwegian context. On November 19th, 2021, the second annual Norwegian implementation conference was held in Oslo. Ninety participants from all regions of the country gathered to showcase the frontiers of Norwegian implementation research. The conference also hosted a panel discussion about critical next steps for implementation science in Norway. The conference included 17 presentations from diverse disciplines within health and welfare services, including schools. The themes presented included stakeholder engagement, implementation mechanisms, evaluations of the implementation of specific interventions, the use of implementation guidelines and frameworks, the development and validation of implementation measurements, and barriers and facilitators for implementation. The panel discussion highlighted several critical challenges with the implementation of evidence-informed practices in Norway, including limited implementation competence and capacity among practice leaders and workforces, few opportunities for education in implementation science, limited implementation research in the Norwegian context, scarce funding possibilities for implementation research, and a lack of long-term perspectives on implementation processes. Overall, the 2021 Norwegian implementation conference showed an encouraging sign of a maturing field of science in Norway. The more voluminous proceedings from the 2020 conference called for several important advancements to improve implementation science and practice in Norway, and the 2021 conference indicates that steps have already been taken in favorable directions in terms of, for instance, research designs and measurements. However, there are still unexploited potentials for improvements in implementation research, funding, policies, and practice. Norwegian implementation researcher should be mindful of the challenges and potential pitfalls implementation science currently face as a scientific discipline.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43477-022-00069-w.