Making Implementation Science More Rapid: Use of the RE-AIM Framework for Mid-Course Adaptations Across Five Health Services Research Projects in the Veterans Health Administration

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): a modified Delphi study

BACKGROUND

Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources.

METHOD

MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability and inclusion (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis.

RESULTS

In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application.

DISCUSSION

MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

Implementation Science and Practice-Oriented Research: Convergence and Complementarity

Implementation science is the scientific study of methods to promote the uptake of research findings and other evidence-based practices in routine care, with the goal of improving the quality and effectiveness of health services (Bauer et al., 2015). In addition to this common goal, practice-oriented psychotherapy research (and researchers) and implementation science (and scientists) share a common focus on the people and the places where treatment happens. Thus, there exists strong potential for combining these two approaches. In this article, we provide a primer on implementation science for psychotherapy researchers and highlight important areas and examples of convergence and complementarity between implementation science and practice-oriented psychotherapy research. Specifically, we (a) define and describe the core features of implementation science; (b) discuss similarities and areas of complementarity between implementation science and practice-oriented psychotherapy research; (c) discuss a case example that exemplifies the integration of implementation science and practice-oriented research; and (d) propose directions for future research and collaborations that leverage both implementation science and practice-oriented research.

A modified action framework to develop and evaluate academic-policy engagement interventions

BACKGROUND

There has been a proliferation of frameworks with a common goal of bridging the gap between evidence, policy, and practice, but few aim to specifically guide evaluations of academic-policy engagement. We present the modification of an action framework for the purpose of selecting, developing and evaluating interventions for academic-policy engagement.

METHODS

We build on the conceptual work of an existing framework known as SPIRIT (Supporting Policy In Health with Research: an Intervention Trial), developed for the evaluation of strategies intended to increase the use of research in health policy. Our aim was to modify SPIRIT, (i) to be applicable beyond health policy contexts, for example encompassing social, environmental, and economic policy impacts and (ii) to address broader dynamics of academic-policy engagement. We used an iterative approach through literature reviews and consultation with multiple stakeholders from Higher Education Institutions (HEIs) and policy professionals working at different levels of government and across geographical contexts in England, alongside our evaluation activities in the Capabilities in Academic Policy Engagement (CAPE) programme.

RESULTS

Our modifications expand upon Redman et al.'s original framework, for example adding a domain of 'Impacts and Sustainability' to capture continued activities required in the achievement of desirable outcomes. The modified framework fulfils the criteria for a useful action framework, having a clear purpose, being informed by existing understandings, being capable of guiding targeted interventions, and providing a structure to build further knowledge.

CONCLUSION

The modified SPIRIT framework is designed to be meaningful and accessible for people working across varied contexts in the evidence-policy ecosystem. It has potential applications in how academic-policy engagement interventions might be developed, evaluated, facilitated and improved, to ultimately support the use of evidence in decision-making.

Thermal Imaging for Burn Wound Depth Assessment: A Mixed-Methods Implementation Study

Background: Implementing innovations emerging from clinical research can be challenging. Thermal imagers provide an accessible diagnostic tool to increase the accuracy of burn wound depth assessment. This mixed-methods implementation study aimed to assess the barriers and facilitators, design implementation strategies, and guide the implementation process of thermal imaging in the outpatient clinic of a burn centre. Methods: This study was conducted between September 2022 and February 2023 in Beverwijk, The Netherlands. Semi-structured interviews with burn physicians guided by the Consolidated Framework for Implementation Research (CFIR) were conducted to identify barriers and facilitators. Based on the barriers, implementation strategies were developed with the CFIR-ERIC Matching Tool, and disseminated to support the uptake of the thermal imager. Subsequently, thermal imaging was implemented in daily practice, and an iterative RE-AIM approach was used to evaluate the implementation process. Results: Common facilitators for the implementation of the thermal imager were the low complexity, the relative advantage above other diagnostic tools, and benefits for patients. Common barriers were physicians' attitude towards and perceived value of the intervention, the low compatibility with the current workflow, and a lack of knowledge about existing evidence. Six implementation strategies were developed: creating a formal implementation blueprint, promoting adaptability, developing educational materials, facilitation, conducting ongoing training, and identifying early adopters. These strategies resulted in the effective implementation of the thermal imager, reflected by a >70% reach among eligible patients, and >80% effectiveness and adoption. Throughout the implementation process, compatibility, and available resources remained barriers, resulting in low ratings on RE-AIM dimensions. Conclusions: This study developed implementation strategies based on the identified CFIR constructs that impacted the implementation of a thermal imager for burn wound assessment in our outpatient clinic. The experiences and findings of this study could be leveraged to guide the implementation of thermal imaging and other innovations in burn care.

Stakeholder and Equity Data-Driven Implementation: a Mixed Methods Pilot Feasibility Study

We conducted a mixed methods pilot feasibility study of a Stakeholder and Equity Data-Driven Implementation (SEDDI) process to facilitate using healthcare data to identify patient groups experiencing gaps in the use of evidence-based interventions (EBIs) and rapidly adapt EBIs to achieve greater access and equitable outcomes. We evaluated the feasibility and acceptability of SEDDI in a pilot hybrid type 2 effectiveness-implementation trial of a paired colorectal cancer (CRC) and social needs screening intervention at four federally qualified community health centers (CHCs). An external facilitator partnered with CHC teams to support initial implementation, followed by the SEDDI phase focused on advancing health equity. Facilitation sessions were delivered over 8 months. Preliminary evaluation of SEDDI involved convergent mixed methods with quantitative survey and focus group data. CHCs used data to identify gaps in outreach and completion of CRC screening with respect to race/ethnicity, gender, age, and language. Adaptations to improve access and use of the intervention included cultural, linguistic, and health literacy tailoring. CHC teams reported that facilitation and systematic review of data were helpful in identifying and prioritizing gaps. None of the four CHCs completed rapid cycle testing of adaptations largely due to competing priorities during the COVID-19 response. SEDDI has the potential for advancing chronic disease prevention and management by providing a stakeholder and data-driven approach to identify and prioritize health equity targets and guide adaptations to improve health equity. ClinicalTrials.gov Identifier: NCT04585919.

Enhancing Impact: A Call to Action for Equitable Implementation Science

Despite investments in evidence-based interventions and Implementation Science, most evidence-based interventions are not widely or routinely adopted, delivered, or sustained in many real-world community and healthcare settings. This gap is even greater in settings and populations experiencing numerous social and structural barriers to health, with important implications for persistent patterns in health inequities. In this Viewpoint, as part of a Special Issue on Advancing the Adaptability of Chronic Disease Prevention and Management through Implementation Science, we outline seven calls to action for the field of Implementation Science, with the goal of encouraging researchers, practitioners, and funders to be more intentional and accountable in applying Implementation Science to have greater impact on promoting health equity. Calls to action include (1) enhance public health, community, and multi-sectoral partnerships to promote health equity and equitable implementation; (2) revisit and build the evidence base needed to promote health equity and impact at multiple levels; (3) prioritize focus on policy development, dissemination, and implementation; (4) be agile and responsive in application of Implementation Science frameworks, processes, and methods; (5) identify and redefine meaningful metrics for equity and impact; (6) disseminate scientific evidence and research to a diverse range of partners and potential beneficiaries; and (7) extend focus on de-implementation, mis-implementation, and sustainability which are central to enhancing health equity. Additionally, we outline why a focus on prevention and public health is essential to making progress towards health equity in Implementation Science, summarize important advancements that the field has made towards making equity more foundational, and pose important research questions to enhance equitable impact of work in this area.

Implementation science for cancer control: One center's experience addressing context, adaptation, equity, and sustainment

Implementation science (IS) has great potential to enhance the frequency, speed, and quality of the translation of evidence-based programs, policies, products, and guidelines into practice. Progress has been made, but with some notable exceptions, this promise has not been achieved for cancer prevention and control. We discuss five interrelated but conceptually distinct, crosscutting issues important to accelerate IS for cancer prevention and control and how our Colorado Implementation Science Center in Cancer Control (COISC3) addressed these issues. These needs and opportunities include more fully addressing changing, multi-level context; guiding rapid, iterative adaptations; evaluating innovative approaches to engagement and health equity; greater attention to costs and economic issues; and sustainability. We summarize conceptual issues; evaluation needs and capacity building activities and then provide examples of how our IS center addressed these five needs for cancer prevention and control. We discuss changes made to address priorities of (i) guiding adaptations of implementation strategies to address changing context and (ii) working on issues identified and prioritized by our primary care partners rather than the research team. We conclude with discussion of lessons learned, limitations, and directions for future research and practice in IS to enhance cancer prevention and control as well as translational behavioral medicine more generally.

Older adults using social support to improve self-care (OASIS): Adaptation, implementation and feasibility of peer support for older adults with T2D in appalachia: A feasibility study protocol

INTRODUCTION

The prevalence of type 2 diabetes (T2D) is 17% higher in rural dwellers compared to their urban counterparts, and it increases with age, with an estimated 25% of older adults (≥ 65 years) diagnosed. Appropriate self-care is necessary for optimal clinical outcomes. Overall, T2D self-care is consistently poor among the general population but is even worse in rural-dwellers and older adults. In rural Kentucky, up to 23% of adults in Appalachian communities have been diagnosed with T2D and, of those, 26.8% are older adults. To attain optimal clinical outcomes, social environmental factors, including social support, are vital when promoting T2D self-care. Specifically, peer support has shown to be efficacious in improving T2D self-care behaviors and clinical and psychosocial outcomes related to T2D; however, literature also suggests self-selected social support can be obstructive when engaging in healthful activities. Currently available evidence-based interventions (EBIs) using peer support have not been used to prioritize older adults, especially those living in rural communities.

METHOD

To address this gap, we conducted formative research with stakeholders, and collaboratively identified an acceptable and feasible peer support EBI-peer health coaching (PHC)-that has resulted in improved clinical and psychosocial T2D-related outcomes among participants who did not reside in rural communities nor were ≥65 years. The goal of the proposed study is to use a 2x2 factorial design to test the adapted PHC components and determine their preliminary effectiveness to promote self-care behaviors and improve glycemic control among older adults living in Appalachian Kentucky. Testing the PHC components of the peer support intervention will be instrumental in promoting care for older adults in Appalachia, as it will allow for a larger scale intervention, which if effective, could be disseminated to community partners in Appalachia.

TRIAL REGISTRATION

This study was registered at www.clinicaltrials.gov (NCT06003634) in August 2023.

Real-time evaluation and adaptation to facilitate rapid recruitment in a large, prospective cohort study

BACKGROUND

Recruiting large cohorts efficiently can speed the translation of findings into care across a range of scientific disciplines and medical specialties. Recruitment can be hampered by factors such as financial barriers, logistical concerns, and lack of resources for patients and clinicians. These and other challenges can lead to underrepresentation in groups such as rural residents and racial and ethnic minorities. Here we discuss the implementation of various recruitment strategies for enrolling participants into a large, prospective cohort study, assessing the need for adaptations and making them in real-time, while maintaining high adherence to the protocol and high participant satisfaction.

METHODS

While conducting a large, prospective trial of a multi-cancer early detection blood test at Geisinger, an integrated health system in central Pennsylvania, we monitored recruitment progress, adherence to the protocol, and participants' satisfaction. Tracking mechanisms such as paper records, electronic health records, research databases, dashboards, and electronic files were utilized to measure each outcome. We then reviewed study procedures and timelines to list the implementation strategies that were used to address barriers to recruitment, protocol adherence and participant satisfaction.

RESULTS

Adaptations to methods that contributed to achieving the enrollment goal included offering multiple recruitment options, adopting group consenting, improving visit convenience, increasing the use of electronic capture and the tracking of data and source documents, staffing optimization via leveraging resources external to the study team when appropriate, and integrating the disclosure of study results into routine clinical care without adding unfunded work for clinicians. We maintained high protocol adherence and positive participant experience as exhibited by a very low rate of protocol deviations and participant complaints.

CONCLUSION

Recruiting rapidly for large studies - and thereby facilitating clinical translation - requires a nimble, creative approach that marshals available resources and changes course according to data. Planning a rigorous assessment of a study's implementation outcomes prior to study recruitment can further ground study adaptations and facilitate translation into practice. This can be accomplished by proactively and continuously assessing and revising implementation strategies.

Aligning the planning, development, and implementation of complex interventions to local contexts with an equity focus: application of the PRISM/RE-AIM Framework

For the fields of implementation science and health equity, understanding and being responsive to local contexts is of utmost importance to better inform the development, implementation, and evaluation of healthcare and public health interventions to increase their uptake and sustainment. Contexts are multi-level and include political, historical, economic, and social factors that influence health, as well as organizational characteristics, reflecting the richness of members' views, resources, values, and needs. Poor alignment between solutions and those contextual characteristics could have an impact on inequities. The PRISM (Practical Robust Implementation and Sustainability Model) is a context-based implementation science framework that incorporates RE-AIM outcomes (Reach, Effectiveness, Adoption, Implementation, Maintenance) and offers guidance to researchers, practitioners, and their patient and community partners on how to conceptualize, assess, and address contextual domains with a focus on health equity. Drawing from systems thinking, participatory engagement, and health equity principles, this commentary expands on previous work to 1) offer a novel perspective on how to align an intervention's core functions and forms with the PRISM's contextual domains, and 2) foster an ongoing and iterative engagement process with diverse partners throughout the research and practice process using a co-creation approach. We recommend intervention-to-context alignment through iterative cycles. To that end, we present the RE-AIM Framework's 'outcomes cascade' to illustrate touch points of opportunity and gaps within and across each of the five RE-AIM outcomes to illustrate 'where things go wrong'. We present a case study to illustrate and offer recommendations for research and practice efforts to increase contextual responsiveness, and enhance alignment with context before, during, and after implementation efforts and to ensure equity is being addressed. We strive to make a conceptual contribution to advance the field of pragmatic research and implementation of evidence-based practices through the application of the contextually-based PRISM framework with a focus on health equity.

Leveraging artificial intelligence to advance implementation science: potential opportunities and cautions

BACKGROUND

The field of implementation science was developed to address the significant time delay between establishing an evidence-based practice and its widespread use. Although implementation science has contributed much toward bridging this gap, the evidence-to-practice chasm remains a challenge. There are some key aspects of implementation science in which advances are needed, including speed and assessing causality and mechanisms. The increasing availability of artificial intelligence applications offers opportunities to help address specific issues faced by the field of implementation science and expand its methods.

MAIN TEXT

This paper discusses the many ways artificial intelligence can address key challenges in applying implementation science methods while also considering potential pitfalls to the use of artificial intelligence. We answer the questions of "why" the field of implementation science should consider artificial intelligence, for "what" (the purpose and methods), and the "what" (consequences and challenges). We describe specific ways artificial intelligence can address implementation science challenges related to (1) speed, (2) sustainability, (3) equity, (4) generalizability, (5) assessing context and context-outcome relationships, and (6) assessing causality and mechanisms. Examples are provided from global health systems, public health, and precision health that illustrate both potential advantages and hazards of integrating artificial intelligence applications into implementation science methods. We conclude by providing recommendations and resources for implementation researchers and practitioners to leverage artificial intelligence in their work responsibly.

CONCLUSIONS

Artificial intelligence holds promise to advance implementation science methods ("why") and accelerate its goals of closing the evidence-to-practice gap ("purpose"). However, evaluation of artificial intelligence's potential unintended consequences must be considered and proactively monitored. Given the technical nature of artificial intelligence applications as well as their potential impact on the field, transdisciplinary collaboration is needed and may suggest the need for a subset of implementation scientists cross-trained in both fields to ensure artificial intelligence is used optimally and ethically.

Interventions connecting young people living in Africa to healthcare; a systematic review using the RE-AIM framework

Introduction

Africa's young people are among the least focused groups in healthcare linkage. The disproportionally high burden of youth-related health problems is a burden, especially in developing regions like Africa, which have a high population of young people. More information is needed about factors that impact linkages in healthcare and the sustainability of health interventions among young people in Africa.

Methods

A systematic literature search was performed from October 2020 to May 2022 in PubMed, CINAHL, Scopus, Global Health, and the Web of Science. Studies included in the review were conducted among young people aged 10-24 living in Africa, written in English, and published between 2011 and 2021. Results were reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Data was analyzed using narrative synthesis, synthesizing the details of the RE-AIM reporting component. Interventions were systematically compared using the Cochrane Collaboration risk-of-bias tool to evaluate the rigor of each intervention.

Results

A total of 2,383 potentially relevant citations were obtained after an initial database search. Retained in the final group were seventeen articles from electronic data searches; among these articles, 16 interventions were identified. Out of the seventeen studies, nine (53%) were randomized controlled trials, three (18%) were quasi-experimental designs, and five (29%) were observational studies. At the same time, the included interventions were reported on 20 (76.92%) of the 26 components of the RE-AIM dimensions. In eastern Africa, twelve (80%) interventions were conducted, and all the interventions addressed linkage to care for young people in preventing and treating HIV. The least reported RE-AIM dimensions were implementing and maintaining interventions connecting young people to care.

Discussion

Timely care remains critical to treating and preventing ailments. This review indicates that interventions created to link young people to care, especially HIV care, can help link them to health care and strengthen the programs. It is also clear that further research with more extended follow-up periods is needed to examine connections to care in all other aspects of health and to bridge the gap between research and practice in the care of young people in Africa.

Systematic Review Registration

PROSPERO [CRD42022288227].

Adaptation of the brainwriting premortem technique to inform the co-creation of COVID-19 testing strategies in underserved communities in South San Diego

INTRODUCTION

Meaningful engagement of partners in co-creating and refining health-related programs can increase the initial uptake, sustained implementation, broad reach, and effectiveness of these programs. This is especially important for underserved communities where resources are limited and need to be prioritized. Brainwriting premortem is a novel qualitative approach to partner engagement that combines the strengths of individual idea generation with the concept of premortem exercise that addresses failure points prior to the implementation of new programs.

METHODS

An adapted form of brainwriting premortem was used to inform iterative refinements to a COVID-19 testing program at a Federally Qualified Health Center (FQHC) in San Diego. Patients and providers from the FQHC participated in interviews at two time points (early- and mid-implementation of the program). Interview data were transcribed, translated, and analyzed using a rapid qualitative approach. Key themes and sub-themes were identified and used to inform refinements to the program.

RESULTS

A total of 11 patients (7 Spanish- and 4 English-speaking) and 8 providers participated in the brainwriting premortem interviews. Key themes related to possible reasons for COVID-19 testing program failure: advertising/sharing information; access to testing; handling of test results; staff and patient safety; patient beliefs and views regarding the SARS-CoV-2 virus; and COVID-19 testing options offered. Proposed solutions were offered for the key failures except for patient beliefs and views regarding the SARS-CoV-2 virus. Additional solutions offered were related to education, physical operations, and recruitment strategies. Real-time changes to the program flow and components were made in response to 7 suggestions from patients and 11 from providers. Changes related to the process of returning results were the most common, and included sending results via email with distinct workflows based on the test result.

CONCLUSION

The implementation of the adapted brainwriting premortem technique allowed us to incorporate the perspective of key partners in the delivery and iterative refinement of the COVID-19 testing program. This was an effective tool in the context of an FQHC and can be a promising and approach to incorporate iterative input from patients and providers to ensure successful program implementation. Future studies, particularly those requiring rapid response to public health emergencies, should consider the use of this technique.

Identifying opportunities to optimize mass drug administration for soil-transmitted helminths: A visualization and descriptive analysis using process mapping

BACKGROUND

The control of soil-transmitted helminths (STH) is achieved through mass drug administration (MDA) with deworming medications targeting children and other high-risk groups. Recent evidence suggests that it may be possible to interrupt STH transmission by deworming individuals of all ages via community-wide MDA (cMDA). However, a change in delivery platforms will require altering implementation processes.

METHODS

We used process mapping, an operational research methodology, to describe the activities required for effective implementation of school-based and cMDA in 18 heterogenous areas and over three years in Benin, India, and Malawi. Planned activities were identified during workshops prior to initiation of a large cMDA trial (the DeWorm3 trial). The process maps were updated annually post-implementation, including adding or removing activities (e.g., adaptations) and determining whether activities occurred according to plan. Descriptive analyses were performed to quantify differences and similarities at baseline and over three implementation years. Comparative analyses were also conducted between study sites and areas implementing school-based vs. cMDA. Digitized process maps were developed to provide a visualization of MDA processes and inspected to identify implementation bottlenecks and inefficient activity flows.

RESULTS

Across three years and all clusters, implementation of cMDA required an average of 13 additional distinct activities and was adapted more often (5.2 adaptations per year) than school-based MDA. An average of 41% of activities across both MDA platforms did not occur according to planned timelines; however, deviations were often purposeful to improve implementation efficiency or effectiveness. Visualized process maps demonstrated that receipt of drugs at the local level may be an implementation bottleneck. Many activities rely on the effective setting of MDA dates and estimating quantity of drugs, suggesting that the timing of these activities is important to meet planned programmatic outcomes.

CONCLUSION

Implementation processes were heterogenous across settings, suggesting that MDA is highly context and resource dependent and that there are many viable ways to implement MDA. Process mapping could be deployed to support a transition from a school-based control program to community-wide STH transmission interruption program and potentially to enable integration with other community-based campaigns.

TRIAL REGISTRATION

NCT03014167.

Advancing healthcare equity through dissemination and implementation science

OBJECTIVE

To provide guiding principles and recommendations for how approaches from the field of dissemination and implementation (D&I) science can advance healthcare equity.

DATA SOURCES AND STUDY SETTING

This article, part of a special issue sponsored by the Agency for Healthcare Research and Quality (AHRQ), is based on an outline drafted to support proceedings of the 2022 AHRQ Health Equity Summit and further revised to reflect input from Summit attendees.

STUDY DESIGN

This is a narrative review of the current and potential applications of D&I approaches for understanding and advancing healthcare equity, followed by discussion and feedback with Summit attendees.

DATA COLLECTION/EXTRACTION METHODS

We identified major themes in narrative and systematic reviews related to D&I science, healthcare equity, and their intersections. Based on our expertise, and supported by synthesis of published studies, we propose recommendations for how D&I science is relevant for advancing healthcare equity. We used iterative discussions internally and at the Summit to refine preliminary findings and recommendations.

PRINCIPAL FINDINGS

We identified four guiding principles and three D&I science domains with strong promise for accelerating progress toward healthcare equity. We present eight recommendations and more than 60 opportunities for action by practitioners, healthcare leaders, policy makers, and researchers.

CONCLUSIONS

Promising areas for D&I science to impact healthcare equity include the following: attention to equity in the development and delivery of evidence-based interventions; the science of adaptation; de-implementation of low-value care; monitoring equity markers; organizational policies for healthcare equity; improving the economic evaluation of implementation; policy and dissemination research; and capacity building.

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): A modified Delphi study

Background

Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources.

Method

MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis.

Results

In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application.

Discussion

MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

Utilizing PRISM and RE-AIM to implement and evaluate the Rural Telementoring Training Center (RTTC) for health care workforce development in rural communities

Introduction

Amid rural health worker shortages and hospital closures, it is imperative to build and maintain the local workforce. Telementoring (TM) or technology-enabled mentoring, is a tool for improving health care quality and access by increasing workforce capacity and support. The national Rural Telementoring Training Center (RTTC) was developed to compile and disseminate TM best practices by delivering free training, tools, and technical assistance to support the implementation, sustainability, and evaluation of new and current TM programs for rural health workers. This paper details how the Practical, Robust Implementation and Sustainability Model (PRISM) was used to understand the context that shaped implementation as well as how Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) was concurrently applied to frame outcomes.

Methods

The RTTC has three implementation strategies: outreach, training and technical assistance (TTA), and a Quality Measure Toolkit. Ongoing periodic reflections with the RTTC team, informed by PRISM, were collected, as were RE-AIM outcomes. Central to this design was the continuous review of incoming data in team meetings to inform programmatic changes by identifying challenges and applying modifications to strategies in real time.

Results

Major implementation changes discussed during reflections included providing timely and relevant messaging through various platforms, streamlining and customizing a TTA approach, and offering different options for accessing the Toolkit. The outreach strategy resulted in high Reach across the US, with over 300 organizations contacted. The effectiveness of the RTTC was demonstrated by counts of people engaging with outreach (ex. over 8,300 impressions on LinkedIn), the website (over 6,400 views), and e-bursts (33% open rate). Moreover, there were 32 TTA requests and 70 people accessing the Toolkit. Adoption was demonstrated by 27 people participating in TTA and 14 individuals utilizing the Toolkit.

Discussion

The integration of PRISM and RE-AIM frameworks promoted a holistic implementation and evaluation plan. Using PRISM, the RTTC team was able to reflect on the implementation strategies through the lens of contextual factors and make rapid programmatic changes within team meetings. That process resulted in outcomes framed by RE-AIM. The integration of two frameworks in tandem provided an adaptive and comprehensive approach to implementing a large-scale, national program.

Coming together for something good: recommendations from a scoping review for dissemination and implementation science to improve indigenous substance use disorder treatment

Introduction

Dissemination and Implementation (D&I) science is growing among Indigenous communities. Indigenous communities are adapting and implementing evidence-based treatments for substance use disorders (SUD) to fit the needs of their communities. D&I science offers frameworks, models, and theories to increase implementation success, but research is needed to center Indigenous knowledge, enhancing D&I so that it is more applicable within Indigenous contexts. In this scoping review, we examined the current state of D&I science for SUD interventions among Indigenous communities and identified best-practice SUD implementation approaches.

Methods

PubMed and PsycINFO databases were queried for articles written in English, published in the United States, Canada, Australia, and New Zealand. We included key search terms for Indigenous populations and 35 content keywords. We categorized the data using the adapted and extended Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework that emphasizes equity and sustainability. RE-AIM has also been used as a primary model to consistently identify implementation outcomes.

Results

Twenty articles were identified from the original unduplicated count of over 24,000. Over half the articles discussed processes related to Reach, Adoption, and Implementation. Effectiveness was discussed by 50% of the studies (n = 10), with 25% of the articles discussing Maintenance/sustainability (n = 4). Findings also highlighted the importance of the application of each RE-AIM domain for meaningful, well-defined community-engaged approaches.

Conclusion

Finding indicated a need to prioritize Indigenous methods to culturally center, re-align and adapt Western treatments and frameworks to increase health equity and improve SUD treatment outcomes. Utility in the use of the modified RE-AIM and the continued modification for Indigenous communities was also noted.

The iPRISM webtool: an interactive tool to pragmatically guide the iterative use of the Practical, Robust Implementation and Sustainability Model in public health and clinical settings

BACKGROUND

To increase uptake of implementation science (IS) methods by researchers and implementers, many have called for ways to make it more accessible and intuitive. The purpose of this paper is to describe the iPRISM webtool (Iterative, Practical, Robust Implementation and Sustainability Model) and how this interactive tool operationalizes PRISM to assess and guide a program's (a) alignment with context, (b) progress on pragmatic outcomes, (c) potential adaptations, and (d) future sustainability across the stages of the implementation lifecycle.

METHODS

We used an iterative human-centered design process to develop the iPRISM webtool.

RESULTS

We conducted user-testing with 28 potential individual and team-based users who were English and Spanish speaking from diverse settings in various stages of implementing different types of programs. Users provided input on all aspects of the webtool including its purpose, content, assessment items, visual feedback displays, navigation, and potential application. Participants generally expressed interest in using the webtool and high likelihood of recommending it to others. The iPRISM webtool guides English and Spanish-speaking users through the process of iteratively applying PRISM across the lifecycle of a program to facilitate systematic assessment and alignment with context. The webtool summarizes assessment responses in graphical and tabular displays and then guides users to develop feasible and impactful adaptations and corresponding action plans. Equity considerations are integrated throughout.

CONCLUSIONS

The iPRISM webtool can intuitively guide individuals and teams from diverse settings through the process of using IS methods to iteratively assess and adapt different types of programs to align with the context across the implementation lifecycle. Future research and application will continue to develop and evaluate this IS resource.

Study protocol for a multi-level cross-sectional study on the equitable reach and implementation of coordinated specialty care for early psychosis

BACKGROUND

Approximately 115,000 young adults will experience their first episode of psychosis (FEP) each year in the USA. Coordinated specialty care (CSC) for early psychosis is an evidence-based early intervention model that has demonstrated effectiveness by improving quality of life and reducing psychiatric symptoms for many individuals. Over the last decade, there has significant increase in the implementation of CSC programs throughout the USA. However, prior research has revealed difficulties among individuals and their family members accessing CSC. Research has also shown that CSC programs often report the limited reach of their program to underserved populations and communities (e.g., ethnoracial minorities, rural and low socioeconomic neighborhoods). Dissemination and implementation research focused on the equitable reach and implementation of CSC is needed to address disparities at the individual level.

METHODS

The proposed study will create a novel integrative multi-level geospatial database of CSC programs implemented throughout the USA that will include program-level data (e.g., geocoded location, capacity, setting, role availability), provider-level data (race, ethnicity, professional credentials), and neighborhood-level census data (e.g., residential segregation, ethnic density, area deprivation, rural-urban continua, public transit time). This database will be used to characterize variations in CSC programs by geographical location and examine the overall reach CSC programs to specific communities. The quantitative data will be combined with qualitative data from state administrators, providers, and service users that will inform the development of dissemination tools, such as an interactive dashboard, that can aid decision making.

DISCUSSION

Findings from this study will highlight the impact of outer contextual determinants on implementation and reach of mental health services, and will serve to inform the future implementation of CSC programs with a primary focus on equity.

Aligning intuition and theory: a novel approach to identifying the determinants of behaviours necessary to support implementation of evidence into practice

BACKGROUND

Disentangling the interplay between experience-based intuition and theory-informed implementation is crucial for identifying the direct contribution theory can make for generating behaviour changes needed for successful evidence translation. In the context of 'clinicogenomics', a complex and rapidly evolving field demanding swift practice change, we aimed to (a) describe a combined clinician intuition- and theory-driven method for identifying determinants of and strategies for implementing clinicogenomics, and (b) articulate a structured approach to standardise hypothesised behavioural pathways and make potential underlying theory explicit.

METHODS

Interview data from 16 non-genetic medical specialists using genomics in practice identified three target behaviour areas across the testing process: (1) identifying patients, (2) test ordering and reporting, (3) communicating results. The Theoretical Domains Framework (TDF) was used to group barriers and facilitators to performing these actions. Barriers were grouped by distinct TDF domains, with 'overarching' TDF themes identified for overlapping barriers. Clinician intuitively-derived implementation strategies were matched with corresponding barriers, and retrospectively coded against behaviour change techniques (BCTs). Where no intuitive strategies were provided, theory-driven strategies were generated. An algorithm was developed and applied to articulate how implementation strategies address barriers to influence behaviour change.

RESULTS

Across all target behaviour areas, 32 identified barriers were coded across seven distinct TDF domains and eight overarching TDF themes. Within the 29 intuitive strategies, 21 BCTs were represented and used on 49 occasions to address 23 barriers. On 10 (20%) of these occasions, existing empirical links were found between BCTs and corresponding distinct TDF-coded barriers. Twenty additional theory-driven implementation strategies (using 19 BCTs on 31 occasions) were developed to address nine remaining barriers.

CONCLUSION

Clinicians naturally generate their own solutions when implementing clinical interventions, and in this clinicogenomics example these intuitive strategies aligned with theoretical recommendations 20% of the time. We have matched intuitive strategies with theory-driven BCTs to make potential underlying theory explicit through proposed structured hypothesised causal pathways. Transparency and efficiency are enhanced, providing a novel method to identify determinants of implementation. Operationalising this approach to support the design of implementation strategies may optimise practice change in response to rapidly evolving scientific advances requiring swift translation into healthcare.

Bridging the "Know-Do" Gaps in Five Non-Communicable Diseases Using a Common Framework Driven by Implementation Science

According to the United Nations High-Level Meeting 2018, five non-communicable diseases (NCDs) including cardiovascular diseases, chronic respiratory diseases, diabetes mellitus, cancer, and mental health conditions accounted for two-thirds of global deaths. These five NCDs share five common risk factors including tobacco use, unhealthy diets, physical inactivity, alcohol use, and air pollution. Low- and middle-income countries (LMICs) face larger burden of NCDs than high-income countries (HICs), due to differences in ecological, technological, socioeconomic and health system development. Based on high-level evidence albeit mainly from HICs, the burden caused by NCDs can be reduced by affordable medicines and best practices. However, "know-do" gaps, ie, gaps between what we know in science and what we do in practice, has limited the impact of these strategies, especially in LMICs. Implementation science advocates the use of robust methodologies to evaluate sustainable solutions in health, education and social care aimed at informing practice and policies. In this article, physician researchers with expertise in NCDs reviewed the common challenges shared by these five NCDs with different clinical courses. They explained the principles of implementation science and advocated the use of an evidence-based framework to implement solutions focusing on early detection, prevention and empowerment, supplemented by best practices in HICs and LMICs. These successful stories can be used to motivate policymakers, payors, providers, patients and public to co-design frameworks and implement context-relevant, multi-component, evidence-based practices. In pursuit of this goal, we propose partnership, leadership, and access to continuing care as the three pillars in developing roadmaps for addressing the multiple needs during the journey of a person with or at risk of these five NCDs. By transforming the ecosystem, raising awareness and aligning context-relevant practices and policies with ongoing evaluation, it is possible to make healthcare accessible, affordable and sustainable to reduce the burden of these five NCDs.

Prioritization of Evidence-Based and Evidence-Informed Interventions for Retention in Medical Care for Persons with HIV

Up to 50% of those diagnosed with HIV in the U.S. are not retained in medical care. Care retention provides opportunity to monitor benefits of HIV therapy and enable viral suppression. To increase retention, there is a need to prioritize best practices appropriate for translation and dissemination for real-world implementation. Eighteen interventions from CDC's Compendium of Evidence-Based Interventions were scored using the RE-AIM framework to determine those most suitable for dissemination. A CDC Division of HIV Prevention workgroup developed a RE-AIM scale with emphasis on the Implementation and Maintenance dimensions and less emphasis on the Efficacy dimension since all 18 interventions were already identified as evidence-based or evidence-informed. Raters referenced primary efficacy publications and scores were averaged for a ranked RE-AIM score for interventions. Of 18 interventions, four included care linkage and 7 included viral suppression outcomes. Interventions received between 20.6 and 35.3 points (45 maximum). Scores were converted into a percentage of the total possible with ranges between 45.8 and 78.4%. Top four interventions were ARTAS (78.4%); Routine Screening for HIV (RUSH) (73.2%); Optn4Life (67.4%) and Virology Fast Track (65.9%). All four scored high on Implementation and Maintenance dimensions. Select items within the scale were applicable to health equity, covering topics such as reaching under-served focus populations and acceptability to that population. Navigation-enhanced Case Management (NAV) scored highest on the health equity subscale. RE-AIM prioritization scores will inform dissemination and translation efforts, help clinical staff select feasible interventions for implementation, and support sustainability for those interventions.

Anticipating adaptation: tracking the impact of planned and unplanned adaptations during the implementation of a complex population-based genomic screening program

In 2021, the Medical University of South Carolina (MUSC) launched In Our DNA SC. This large-scale initiative will screen 100,000 individuals in South Carolina for three preventable hereditary conditions that impact approximately two million people in the USA but often go undetected. In anticipation of inevitable changes to the delivery of this complex initiative, we developed an approach to track and assess the impact of evaluate adaptations made during the pilot phase of program implementation. We used a modified version of the Framework for Reporting Adaptations and Modification-Enhanced (FRAME) and Adaptations to code adaptations made during the 3-month pilot phase of In Our DNA SC. Adaptations were documented in real-time using a REDCap database. We used segmented linear regression models to independently test three hypotheses about the impact of adaptations on program reach (rate of enrollment in the program, rate of messages viewed) and implementation (rate of samples collected) 7 days pre- and post-adaptation. Effectiveness was assessed using qualitative observations. Ten adaptations occurred during the pilot phase of program implementation. Most adaptations (60%) were designed to increase the number and type of patient contacted (reach). Adaptations were primarily made based on knowledge and experience (40%) or from quality improvement data (30%). Of the three adaptations designed to increase reach, shortening the recruitment message potential patients received significantly increased the average rate of invitations viewed by 7.3% (p = 0.0106). There was no effect of adaptations on implementation (number of DNA samples collected). Qualitative findings support improvement in effectiveness of the intervention after shortening the consent form and short-term positive impact on uptake of the intervention as measured by team member's participation. Our approach to tracking adaptations of In Our DNA SC allowed our team to quantify the utility of modifications, make decisions about pursuing the adaptation, and understand consequences of the change. Streamlining tools for tracking and responding to adaptations can help monitor the incremental impact of interventions to support continued learning and problem solving for complex interventions being delivered in health systems based on real-time data.

Study protocol: Type III hybrid effectiveness-implementation study implementing Age-Friendly evidence-based practices in the VA to improve outcomes in older adults

BACKGROUND

Unmet care needs among older adults accelerate cognitive and functional decline and increase medical harms, leading to poorer quality of life, more frequent hospitalizations, and premature nursing home admission. The Department of Veterans Affairs (VA) is invested in becoming an "Age-Friendly Health System" to better address four tenets associated with reduced harm and improved outcomes among the 4 million Veterans aged 65 and over receiving VA care. These four tenets focus on "4Ms" that are fundamental to the care of older adults, including (1) what Matters (ensuring that care is consistent with each person's goals and preferences); (2) Medications (only using necessary medications and ensuring that they do not interfere with what matters, mobility, or mentation); (3) Mentation (preventing, identifying, treating, and managing dementia, depression, and delirium); and (4) Mobility (promoting safe movement to maintain function and independence). The Safer Aging through Geriatrics-Informed Evidence-Based Practices (SAGE) Quality Enhancement Research Initiative (QUERI) seeks to implement four evidence-based practices (EBPs) that have shown efficacy in addressing these core tenets of an "Age-Friendly Health System," leading to reduced harm and improved outcomes in older adults.

METHODS

We will implement four EBPs in 9 VA medical centers and associated outpatient clinics using a type III hybrid effectiveness-implementation stepped-wedge trial design. We selected four EBPs that align with Age-Friendly Health System principles: Surgical Pause, EMPOWER (Eliminating Medications Through Patient Ownership of End Results), TAP (Tailored Activities Program), and CAPABLE (Community Aging in Place - Advancing Better Living for Elders). Guided by the Pragmatic Robust Implementation and Sustainability Model (PRISM), we are comparing implementation as usual vs. active facilitation. Reach is our primary implementation outcome, while "facility-free days" is our primary effectiveness outcome across evidence-based practice interventions.

DISCUSSION

To our knowledge, this is the first large-scale randomized effort to implement "Age-Friendly" aligned evidence-based practices. Understanding the barriers and facilitators to implementing these evidence-based practices is essential to successfully help shift current healthcare systems to become Age-Friendly. Effective implementation of this project will improve the care and outcomes of older Veterans and help them age safely within their communities.

TRIAL REGISTRATION

Registered 05 May 2021, at ISRCTN #60,657,985.

REPORTING GUIDELINES

Standards for Reporting Implementation Studies (see attached).

Fast Methods for Drug Approval: Research Perspectives for Pandemic Preparedness

Public heath emergencies such as the outbreak of novel infectious diseases represent a major challenge for drug regulatory bodies, practitioners, and scientific communities. In such critical situations drug regulators and public health practitioners base their decisions on evidence generated and synthesised by scientists. The urgency and novelty of the situation create high levels of uncertainty concerning the safety and effectiveness of drugs. One key tool to mitigate such emergencies is pandemic preparedness. There seems to be, however, a lack of scholarly work on methodology for assessments of new or existing drugs during a pandemic. Issues related to risk attitudes, evidence production and evidence synthesis for drug approval require closer attention. This manuscript, therefore, engages in a conceptual analysis of relevant issues of drug assessment during a pandemic. To this end, we rely in our analysis on recent discussions in the philosophy of science and the philosophy of medicine. Important unanswered foundational questions are identified and possible ways to answer them are considered. Similar problems often have similar solutions, hence studying similar situations can provide important clues. We consider drug assessments of orphan drugs and drug assessments during endemics as similar to drug assessment during a pandemic. Furthermore, other scientific fields which cannot carry out controlled experiments may guide the methodology to draw defeasible causal inferences from imperfect data. Future contributions on methodologies for addressing the issues raised here will indeed have great potential to improve pandemic preparedness.

Using the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) to study adaptations in lung cancer screening delivery in the Veterans Health Administration: a cohort study

BACKGROUND

Lung cancer screening is a complex clinical process that includes identification of eligible individuals, shared decision-making, tobacco cessation, and management of screening results. Adaptations to the delivery process for lung cancer screening in situ are understudied and underreported, with the potential loss of important considerations for improved implementation. The Framework for Reporting Adaptations and Modifications-Expanded (FRAME) allows for a systematic enumeration of adaptations to implementation of evidence-based practices. We applied FRAME to study adaptations in lung cancer screening delivery processes implemented by lung cancer screening programs in a Veterans Health Administration (VHA) Enterprise-Wide Initiative.

METHODS

We prospectively conducted semi-structured interviews at baseline and 1-year intervals with lung cancer screening program navigators at 10 Veterans Affairs Medical Centers (VAMCs) between 2019 and 2021. Using this data, we developed baseline (1st) process maps for each program. In subsequent years (year 1 and year 2), each program navigator reviewed the process maps. Adaptations in screening processes were identified, documented, and mapped to FRAME categories.

RESULTS

We conducted a total of 16 interviews across 10 VHA lung cancer screening programs (n=6 in year 1, n=10 in year 2) to collect adaptations. In year 1 (2020), six programs were operational and eligible. Of these, three reported adaptations to their screening process that were planned or in response to COVID-19. In year 2 (2021), all 10 programs were operational and eligible. Programs reported 14 adaptations in year 2. These adaptations were planned and unplanned and often triggered by increased workload; 57% of year 2 adaptations were related to the identification and eligibility of Veterans and 43% were related to follow-up with Veterans for screening results. Throughout the 2 years, adaptations related to data management and patient tracking occurred in 60% of programs to improve the data collection and tracking of Veterans in the screening process.

CONCLUSIONS

Using FRAME, we found that adaptations occurred primarily in the areas of patient identification and communication of results due to increased workload. These findings highlight navigator time and resource considerations for sustainability and scalability of existing and future lung cancer screening programs as well as potential areas for future intervention.

Getting to implementation: Adaptation of an implementation playbook

Introduction

Implementation strategies supporting the translation of evidence into practice need to be tailored and adapted for maximum effectiveness, yet the field of adapting implementation strategies remains nascent. We aimed to adapt "Getting To Outcomes"® (GTO), a 10-step implementation playbook designed to help community-based organizations plan and evaluate behavioral health programs, into "Getting To Implementation" (GTI) to support the selection, tailoring, and use of implementation strategies in health care settings.

Methods

Our embedded evaluation team partnered with operations, external facilitators, and site implementers to employ participatory methods to co-design and adapt GTO for Veterans Health Administration (VA) outpatient cirrhosis care improvement. The Framework for Reporting Adaptations and Modifications to Evidenced-based Implementation Strategies (FRAME-IS) guided documentation and analysis of changes made pre- and post-implementation of GTI at 12 VA medical centers. Data from multiple sources (interviews, observation, content analysis, and fidelity tracking) were triangulated and analyzed using rapid techniques over a 3-year period.

Results

Adaptations during pre-implementation were planned, proactive, and focused on context and content to improve acceptability, appropriateness, and feasibility of the GTI playbook. Modifications during and after implementation were unplanned and reactive, concentrating on adoption, fidelity, and sustainability. All changes were collaboratively developed, fidelity consistent at the level of the facilitator and/or implementer.

Conclusion

GTO was initially adapted to GTI to support health care teams' selection and use of implementation strategies for improving guideline-concordant medical care. GTI required ongoing modification, particularly in steps regarding team building, context assessment, strategy selection, and sustainability due to difficulties with step clarity and progression. This work also highlights the challenges in pragmatic approaches to collecting and synthesizing implementation, fidelity, and adaptation data.

Trial registration

This study was registered on ClinicalTrials.gov (Identifier: NCT04178096).

Use of the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework to guide iterative adaptations: Applications, lessons learned, and future directions

Introduction

Implementation science frameworks have been used widely for planning and evaluation, but seldom to guide adaptations during program implementation. There is great potential for these frameworks to be used to inform conceptual and data-driven decisions about adaptations.

Methods

We summarize recent applications using Iterative RE-AIM to capture and guide adaptations. Iterative RE-AIM can be repeated at multiple time points customized to each project and involves the following activities: identification of key implementation partners; rating importance of and progress on each RE-AIM dimension (reach, effectiveness, adoption, implementation, and maintenance); use of summary data on ratings to identify one or two RE-AIM dimensions for adaptations and implementation strategies; and evaluation of progress and impact of adaptations. We summarize recent and ongoing Iterative RE-AIM applications across multiple care coordination and pain management projects within the Veterans Health Administration, a hypertension control trial in Guatemala, a hospital-based lung ultrasound implementation pilot, and a colorectal cancer screening program in underserved communities.

Results

Iterative RE-AIM appears feasible, helpful, and broadly applicable across diverse health care issues, interventions, contexts, and populations. In general, the RE-AIM dimension showing the largest gap between importance and progress has been Reach. The dimensions most frequently selected for improvement have been Reach and Implementation. We discuss commonalities, differences and lessons learned across these various applications of Iterative RE-AIM. Challenges include having objective real time data on which to make decisions, having key implementation staff available for all assessments, and rapidly scoring and providing actionable feedback. We discuss print and online resources and materials to support Iterative RE-AIM.

Conclusions

The use of Iterative RE-AIM to guide and support understanding of adaptations has proven feasible across diverse projects and in multiple case studies, but there are still questions about its strengths, limitations, essential components, efficiency, comparative effectiveness, and delivery details. Future directions include investigating the optimal frequency and timing for iterative applications; adding contextual assessments; developing more continuous and rapid data on which to make adaptation decisions; identifying opportunities to enhance health equity; and determining the level of facilitation that is most cost-effective.

How Dissemination and Implementation Science Can Contribute to the Advancement of Learning Health Systems

Many health systems are working to become learning health systems (LHSs), which aim to improve the value of health care by rapidly, continuously generating evidence to apply to practice. However, challenges remain to advance toward the aspirational goal of becoming a fully mature LHS. While some important challenges have been well described (i.e., building system-level supporting infrastructure and the accessibility of inclusive, integrated, and actionable data), other key challenges are underrecognized, including balancing evaluation rapidity with rigor, applying principles of health equity and classic ethics, focusing on external validity and reproducibility (generalizability), and designing for sustainability. Many LHSs focus on continuous learning cycles, but with limited consideration of issues related to the rapidity of these learning cycles, as well as the sustainability or generalizability of solutions. Some types of data have been consistently underrepresented, including patient-reported outcomes and preferences, social determinants, and behavioral and environmental data, the absence of which can exacerbate health disparities. A promising approach to addressing many challenges that LHSs face may be found in dissemination and implementation (D&I) science. With an emphasis on multilevel dynamic contextual factors, representation of implementation partner engagement, pragmatic research, sustainability, and generalizability, D&I science methods can assist in overcoming many of the challenges facing LHSs. In this article, the authors describe the current state of LHSs and challenges to becoming a mature LHS, propose solutions to current challenges, focusing on the contributions of D&I science with other methods, and propose key components and characteristics of a mature LHS model that others can use to plan and develop their LHSs.

The economics of adaptations to evidence-based practices

Background

Evidence-based practices (EBPs) are frequently adapted in response to the dynamic contexts in which they are implemented. Adaptation is defined as the degree to which an EBP is altered to fit the setting or to improve fit to local context and can be planned or unplanned. Although adaptations are common and necessary to maximizing the marginal impact of EBPs, little attention has been given to the economic consequences and how adaptations affect marginal costs.

Discussion

In assessing the economic consequences of adaptation, one should consider its impact on core components, the planned adaptive periphery, and the unplanned adaptive periphery. Guided by implementation science frameworks, we examine how various economic evaluation approaches accommodate the influence of adaptations and discuss the pros and cons of these approaches. Using the Framework for Reporting Adaptations and Modifications to Evidence-based interventions (FRAME), mixed methods can elucidate the economic reasons driving the adaptations. Micro-costing approaches are applied in research that integrates the adaptation of EBPs at the planning stage using innovative, adaptive study designs. In contrast, evaluation of unplanned adaptation is subject to confounding and requires sensitivity analysis to address unobservable measures and other uncertainties. A case study is presented using the RE-AIM framework to illustrate the costing of adaptations. In addition to empirical approaches to evaluating adaptation, simulation modeling approaches can be used to overcome limited follow-up in implementation studies.

Conclusions

As implementation science evolves to improve our understanding of the mechanisms and implications of adaptations, it is increasingly important to understand the economic implications of such adaptations, in addition to their impact on clinical effectiveness. Therefore, explicit consideration is warranted of how costs can be evaluated as outcomes of adaptations to the delivery of EBPs.

Advancing rapid adaptation for urgent public health crises: Using implementation science to facilitate effective and efficient responses

Responding rapidly to emerging public health crises is vital to reducing their escalation, spread, and impact on population health. These responses, however, are challenging and disparate processes for researchers and practitioners. Researchers often develop new interventions that take significant time and resources, with little exportability. In contrast, community-serving systems are often poorly equipped to properly adopt new interventions or adapt existing ones in a data-driven way during crises' onset and escalation. This results in significant delays in deploying evidence-based interventions (EBIs) with notable public health consequences. This prolonged timeline for EBI development and implementation results in significant morbidity and mortality that is costly and preventable. As public health emergencies have demonstrated (e.g., COVID-19 pandemic), the negative consequences often exacerbate existing health disparities. Implementation science has the potential to bridge the extant gap between research and practice, and enhance equity in rapid public health responses, but is underutilized. For the field to have a greater "real-world" impact, it needs to be more rapid, iterative, participatory, and work within the timeframes of community-serving systems. This paper focuses on rapid adaptation as a developing implementation science area to facilitate system responses during public health crises. We highlight frameworks to guide rapid adaptation for optimizing existing EBIs when responding to urgent public health issues. We also explore the economic implications of rapid adaptation. Resource limitations are frequently a central reason for implementation failure; thus, we consider the economic impacts of rapid adaptation. Finally, we provide examples and propose directions for future research and application.

Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem

BACKGROUND

Implementation science is at a sufficiently advanced stage that it is appropriate for the field to reflect on progress thus far in achieving its vision, with a goal of charting a path forward. In this debate, we offer such reflections and report on potential threats that might stymie progress, as well as opportunities to enhance the success and impact of the field, from the perspective of a group of US-based researchers.

MAIN BODY

Ten mid-career extramurally funded US-based researchers completed a "pre-mortem" or a group brainstorming exercise that leverages prospective hindsight to imagine that an event has already occurred and to generate an explanation for it - to reduce the likelihood of a poor outcome. We came to consensus on six key themes related to threats and opportunities for the field: (1) insufficient impact, (2) too much emphasis on being a "legitimate science," (3) re-creation of the evidence-to-practice gap, (4) difficulty balancing accessibility and field coherence, (5) inability to align timelines and priorities with partners, and (6) overly complex implementation strategies and approaches.

CONCLUSION

We submit this debate piece to generate further discussion with other implementation partners as our field continues to develop and evolve. We hope the key opportunities identified will enhance the future of implementation research in the USA and spark discussion across international groups. We will continue to learn with humility about how best to implement with the goal of achieving equitable population health impact at scale.

Using Iterative RE-AIM to enhance hospitalist adoption of lung ultrasound in the management of patients with COVID-19: an implementation pilot study

BACKGROUND

Lung ultrasound (LUS) is a clinician-performed evidence-based imaging modality that has multiple advantages in the evaluation of dyspnea caused by multiple disease processes, including COVID-19. Despite these advantages, few hospitalists have been trained to perform LUS. The aim of this study was to increase adoption and implementation of LUS during the 2020 COVID-19 pandemic by using recurrent assessments of RE-AIM outcomes to iteratively revise our implementation strategies.

METHODS

In an academic hospital, we implemented guidelines for the use of LUS in patients with COVID-19 in July 2020. Using a novel "RE-AIM dashboard," we used an iterative process of evaluating the high-priority outcomes of Reach, Adoption, and Implementation at twice monthly intervals to inform revisions of our implementation strategies for LUS delivery (i.e., Iterative RE-AIM process). Using a convergent mixed methods design, we integrated quantitative RE-AIM outcomes with qualitative hospitalist interview data to understand the dynamic determinants of LUS Reach, Adoption, and Implementation.

RESULTS

Over the 1-year study period, 453 LUSs were performed in 298 of 12,567 eligible inpatients with COVID-19 (Reach = 2%). These 453 LUS were ordered by 43 out of 86 eligible hospitalists (LUS order adoption = 50%). However, the LUSs were performed/supervised by only 8 of these 86 hospitalists, 4 of whom were required to complete LUS credentialing as members of the hospitalist procedure service (proceduralist adoption 75% vs 1.2% non-procedural hospitalists adoption). Qualitative and quantitative data obtained to evaluate this Iterative RE-AIM process led to the deployment of six sequential implementation strategies and 3 key findings including (1) there were COVID-19-specific barriers to LUS adoption, (2) hospitalists were more willing to learn to make clinical decisions using LUS images than obtain the images themselves, and (3) mandating the credentialing of a strategically selected sub-group may be a successful strategy for improving Reach.

CONCLUSIONS

Mandating use of a strategically selected subset of clinicians may be an effective strategy for improving Reach of LUS. Additionally, use of Iterative RE-AIM allowed for timely adjustments to implementation strategies, facilitating higher levels of LUS Adoption and Reach. Future studies should explore the replicability of these preliminary findings.

Using the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) to study lung cancer screening adaptations in the Veterans Health Administration

Background: Lung cancer screening includes identification of eligible individuals, shared decision-making inclusive of tobacco cessation, and management of screening results. Adaptations to the implemented processes for lung cancer screening in situ are understudied and underreported, with potential loss of important considerations for improved implementation. The Framework for Reporting Adaptations and Modifications-Expanded (FRAME) allows for systematic enumeration of adaptations to implementations of evidence-based practices. We used FRAME to study adaptations in lung cancer screening processes that were implemented as part of a Veterans Health Administration (VHA) Enterprise-Wide Initiative. Methods: We conducted semi-structured interviews at baseline and 1-year intervals with lung cancer screening program navigators at 10 Veterans Affairs Medical Centers (VAMC) between 2019-2021. Using this data, we developed baseline (1st) process maps for each program. In subsequent years (year 1 and year 2), each program navigator reviewed the process maps. Adaptations in screening processes were identified, recorded and mapped to FRAME categories. Results: A total of 14 program navigators across 10 VHA lung cancer screening programs participated in 20 interviews. In year 1 (2019-2020), seven programs were operational and of these, three reported adaptations to their screening process that were either planned and in response to COVID-19. In year 2 (2020-2021), all 10 programs were operational. Programs reported 14 adaptations in year 2. These adaptations were both planned and unplanned and often triggered by increased workload; 57% of year 2 adaptations were related to identification and eligibility of Veterans and 43% were related to follow-up with Veterans for screening results. Throughout the 2 years, adaptations related to data management and patient tracking occurred in 6 of 10 programs to improve the data collection and tracking of Veterans in the screening process. Conclusions: Using FRAME, we found that adaptations occurred throughout the lung cancer screening process but primarily in the areas of patient identification and communication of results. These findings highlight considerations for lung cancer screening implementation and potential areas for future intervention.

Integrating community engagement with implementation science to advance the measurement of translational science

This special communication provides an approach for applying implementation science frameworks to a Clinical and Translational Science Institutes (CTSIs) community engagement (CE) program that measures the use of implementation strategies and outcomes that promote the uptake of CE in research. Using an iterative multi-disciplinary group process, we executed a four-phased approach to developing an evaluation plan: 1) creating an evaluation model adapted from Proctor's conceptual model of implementation research; 2) mapping implementation strategies to CTSI CE program interventions that support change in research practice; 3) identifying and operationalizing measures for each strategy; and 4) conducting an evaluation. Phase 2 employed 73 implementation strategies across 9 domains generated by the Expert Recommendations for Implementing Change project. The nine domains were used to classify each CE program implementation strategy. In Phase 3, the group used the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework to define measures for each individual strategy. Phase 4 demonstrates the application of this framework and measures Year 1 outcomes for the strategy providing interactive assistance, which we implemented using a centralized consultation model. This approach can support the CTSA program in operationalizing CE program measurement to demonstrate which activities and strategies may lead to benefits derived by the program, institution, and community.

Methods to improve the translation of evidence-based interventions: A primer on dissemination and implementation science for occupational safety and health researchers and practitioners

Objective

A limited focus on dissemination and implementation (D&I) science has hindered the uptake of evidence-based interventions (EBIs) that reduce workplace morbidity and mortality. D&I science methods can be used in the occupational safety and health (OSH) field to advance the adoption, implementation, and sustainment of EBIs for complex workplaces. These approaches should be responsive to contextual factors, including the needs of partners and beneficiaries (such as employers, employees, and intermediaries).

Methods

By synthesizing seminal literature and texts and leveraging our collective knowledge as D&I science and/or OSH researchers, we developed a D&I science primer for OSH. First, we provide an overview of common D&I terminology and concepts. Second, we describe several key and evolving issues in D&I science: balancing adaptation with intervention fidelity and specifying implementation outcomes and strategies. Next, we review D&I theories, models, and frameworks and offer examples for applying these to OSH research. We also discuss widely used D&I research designs, methods, and measures. Finally, we discuss future directions for D&I science application to OSH and provide resources for further exploration.

Results

We compiled a D&I science primer for OSH appropriate for practitioners and evaluators, especially those newer to the field.

Conclusion

This article fills a gap in the OSH research by providing an overview of D&I science to enhance understanding of key concepts, issues, models, designs, methods and measures for the translation into practice of effective OSH interventions to advance the safety, health and well-being of workers.

Adaptations to relational facilitation for two national care coordination programs during COVID-19

Background

Adaptations to implementation strategies are often necessary to support adoption and scale-up of evidence-based practices. Tracking adaptations to implementation strategies is critical for understanding any impacts on outcomes. However, these adaptations are infrequently collected. In this article we present a case study of how we used a new method during COVID-19 to systematically track and report adaptations to relational facilitation, a novel implementation strategy grounded in relational coordination theory. Relational facilitation aims to assess and improve communication and relationships in teams and is being implemented to support adoption of two Quadruple Aim Quality Enhancement Research Initiative (QA QUERI) initiatives: Care Coordination and Integrated Case Management (CC&ICM) and the Transitions Nurse Program for Home Health Care (TNP-HHC) in the Veterans Health Administration (VA).

Methods

During 2021-2022, relational facilitation training, activities and support were designed as in-person and/or virtual sessions. These included a site group coaching session to create a social network map of care coordination roles and assessment of baseline relationships and communication between roles. Following this we administered the Relational Coordination Survey to assess the relational coordination strength within and between roles. COVID-19 caused challenges implementing relational facilitation, warranting adaptations. We tracked relational facilitation adaptations using a logic model, REDCap tracking tool based on the Framework for Reporting Adaptations and Modifications-Enhanced (FRAME) with expanded Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) dimensions, and member checking. Adaptations were analyzed descriptively and for themes using matrix content analysis.

Results

COVID-19's impact within the VA caused barriers for implementing relational facilitation, warranting eight unique adaptations to the implementation strategy. Most adaptations pertained to changing the format of relational facilitation activities (n = 6; 75%), were based on external factors (n = 8; 100%), were planned (n = 8; 100%) and initiated by the QA QUERI implementation team (n = 8; 100%). Most adaptations impacted adoption (n = 6; 75%) and some impacted implementation (n = 2; 25%) of the CC&ICM and TNP-HHC interventions.

Discussion

Systematically tracking and discussing adaptations to relational facilitation during the COVID-19 pandemic enhanced engagement and adoption of two VA care coordination interventions. The impact of these rapid, early course adaptations will be followed in subsequent years of CC&ICM and TNP-HHC implementation.

Applying a User Centred Design Approach to Optimise a Workplace Initiative for Wide-Scale Implementation

Translation of an effective research intervention into a program able to be implemented in practice typically requires adaptations to ensure the outcomes can be achieved within the applied setting. User centred design (UCD) methodologies can support these iterative adaptations, with this approach being particularly well suited to peer-led interventions, due to a focus on usability. We describe and reflect on the UCD approach that was applied to optimise an online, peer-led workplace health promotion initiative (BeUpstanding: ACTRN12617000682347) to be suitable for wide-scale implementation and evaluation. Optimisation was aligned against the indicators of the RE-AIM (reach, effectiveness, adoption, implementation, maintenance) framework, with UCD methodologies (discovery interviews, persona and scenario mapping, facilitated workshops, surveys and prototyping) employed to enhance the program according to all RE-AIM dimensions. The core team (content experts, implementation scientist, interaction designer, software developer, business developer) worked closely with policy and practice partners and end users (workplace champions, management and staff) to iteratively develop and test across the RE-AIM indicators. This description and reflection of the process of applying UCD and the RE-AIM framework to the optimisation of BeUpstanding is intended to provide guidance for other behaviour change research adaptations into practice.

Cardiovascular Risk Screening among Women Veterans: Identifying Provider and Patient Barriers and Facilitators to Develop a Clinical Toolkit

INTRODUCTION

Cardiovascular (CV) disease is the leading cause of death among women in the United States, making CV risk screening and management a women's health priority. Objectives were to elicit barriers and facilitators to CV risk identification and reduction among women veterans, and iteratively cocreate clinical tools to identify CV risk factors and promote goal-setting for lifestyle changes.

METHODS

We conducted three exploratory focus groups with 21 Veterans Health Administration primary care team members and piloted patient CV screeners with brief interviews with 19 patients from two Veterans Health Administration women's clinics to inform toolkit development. We then conducted two focus groups and one interview for feedback from a total of 12 staff on the proposed toolkit components. Transcripts were summarized, and a matrix analysis was used to synthesize qualitative findings.

RESULTS

Provider-identified barriers included difficulties disseminating CV information in clinic, limited patient knowledge, and lack of organized resources for provider communication and available referrals. Women's complex health needs were notable challenges to CV risk reduction. Facilitators included having a single place to track patient CV risks (e.g., an electronic template note), a patient screening worksheet, and aids to complete referrals. Patient-identified barriers included difficulties balancing health, finances, and physical and mental health concerns. Facilitators included resources for accountability and gender-specific information about CV risks and complications. Providers requested easy, accessible tools in the electronic record with gender-specific CV data and resources linked. Patients requested lifestyle change supports, including trustworthy sources vetted by providers.

CONCLUSIONS

Iteratively eliciting end-users' perspectives is critical to developing user-friendly, clinically relevant tools. CV risk reduction among women veterans will require multilevel tools and resources that meet providers' and women's needs.

Ranked determinants of telemedicine diabetic retinopathy screening performance in the United States primary care safety-net setting: an exploratory CART analysis

Background

Diabetic retinopathy (DR) is a leading cause of blindness worldwide, despite easy detection and effective treatment. Annual screening rates in the USA remain low, especially for the disadvantaged, which telemedicine-based DR screening (TDRS) during routine primary care has been shown to improve. Screening rates from such programs have varied, however, pointing to inconsistent implementation and unaddressed barriers. This work seeks to identify and prioritize modifiable barriers for targeted intervention.

Methods

In this final phase of an exploratory mixed-methods study, we developed, validated, and administered a 62-item survey to multilevel stakeholders involved with TDRS in primary care safety-net clinics. Survey items were aligned with previously identified determinants of clinic-level screening and mapped to the Consolidated Framework for Implementation Research (CFIR). Classification and Regression Tree (CART) analyses were used to identify and rank independent variables predictive of individual-level TDRS screening performance.

Results

Overall, 133 of the 341 invited professionals responded (39%), representing 20 safety-net clinics across 6 clinical systems. Respondents were predominately non-Hispanic White (77%), female (94%), and between 31 and 65 years of age (79%). Satisfaction with TDRS was high despite low self-reported screening rates. The most important screening determinants were: provider reinforcement of TDRS importance; explicit instructions by providers to staff; effective reminders; standing orders; high relative priority among routine diabetic measures; established TDRS workflows; performance feedback; effective TDRS champions; and leadership support.

Conclusions

In this survey of stakeholders involved with TDRS in safety-net clinics, screening was low despite high satisfaction with the intervention. The best predictors of screening performance mapped to the CFIR constructs Leadership Engagement, Compatibility, Goals & Feedback, Relative Priority, Champions, and Available Resources. These findings facilitate the prioritization of implementation strategies targeting determinants of TDRS performance, potentially increasing its public health impact.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-07915-5.

Evaluating the Application of the RE-AIM Planning and Evaluation Framework: An Updated Systematic Review and Exploration of Pragmatic Application

Background

RE-AIM is one of the most widely applied frameworks to plan and evaluate the implementation of public health and health behavior change interventions. The objective of this review is to provide an updated synthesis of use of the RE-AIM (Reach Effectiveness Adoption Implementation and Maintenance) planning and evaluation framework and explore pragmatic use (i.e., partial application of the framework) and how this is reported.

Methods

Systematic review. MEDLINE (R) and PsycINFO were searched, via the Ovid interface, between January 2011 and December 2017. Studies that applied RE-AIM as a planning and/or evaluation framework were included.

Results

One hundred fifty-seven articles met inclusion criteria. One hundred forty-nine reported using RE-AIM for evaluation, three for planning and five for planning and evaluation. Reach was the most frequently reported dimension (92.9%), followed by implementation (90.3%), adoption (89.7%), effectiveness (84.5%), and maintenance (77.4%). One hundred forty-seven/one hundred fifty-seven articles originated from high-income economy countries. Within a sub-set analysis (10% of included articles), 9/15 articles evaluated all dimensions. Of the 6/15 articles that did not evaluate all dimensions, five provided no justification for pragmatic application.

Conclusions

RE-AIM has gained increased use in recent years and there is evidence that it is being applied pragmatically. However, the rationale for pragmatic use is often not reported.

Systematic Review Registration

PROSPERO (CRD42017054616).

Leveraging an Implementation Science Framework to Measure the Impact of Efforts to Scale Out a Total Worker Health® Intervention to Employers

The role of dissemination and implementation (D&I) science is critical to the translation of Total Worker Health^® into practice and to the success of interventions in addressing current and future implications for worker safety, health, and well-being. D&I frameworks can guide researchers to design Total Worker Health (“TWH”) delivery approaches that use flexible implementation strategies to implement the core components of programs for employers with varying contextual factors, including small/mid/large-sized businesses and different industry types. To date, there have been very few examples of applying implementation frameworks for the translation and delivery of interventions into organizational settings that require adoption and implementation at the business level to benefit the working individuals. We present a TWH case study, Health Links™, to illustrate an approach to applying an existing implementation framework, RE-AIM, to plan, design, build, and then evaluate TWH implementation strategies. Our case study also highlights key concepts for scaling-out TWH evidence-based interventions where they are implemented in new workplace settings, new delivery systems, or both. Our example provides strong support of key implementation planning constructs including early and consistent stakeholder engagement, tailored messaging and marketing, flexibility, and adaptations in implementation strategies to maximize adoption, implementation, and maintenance among participating businesses.

A mixed methods investigation of implementation determinants for a school-based universal prevention intervention

Background

Effective implementation of evidence-based prevention interventions in schools is vital to reducing the burden of drug use and its consequences. Universal prevention interventions often fail to achieve desired public health outcomes due to poor implementation. One central reason for suboptimal implementation is the limited fit between the intervention and the setting. Research is needed to increase our understanding of how intervention characteristics and context influence intervention implementation in schools to design implementation strategies that will address barriers and improve public health impact.

Methods

Using a convergent mixed methods design we examined qualitative and quantitative data on implementation determinants for an evidence-based health curriculum, the Michigan Model for HealthTM (MMH) from the perspective of health teachers delivering the curriculum in high schools across the state. We examined data strands independently and integrated them by investigating data alignment, expansion, and divergence.

Results

We identified three mixed methods domains: (1) Acceptability, (2) intervention-context fit, and (3) adaptability. We found alignment across data strands as teachers reporting low acceptability also reported low fidelity. The fit between student needs and the curriculum predicted fidelity (expansion). Teachers mentioned instances of poor intervention-context fit (discordance), including when meeting the needs of trauma-exposed youth and keeping updated on youth drug use trends. Teachers reported high adaptability (concordance) but also instances when adaptation was challenging (discordance).

Conclusions

This investigation advances implementation research by deepening our understanding of implementation determinants for an evidence-based universal prevention intervention in schools. This will support designing effective implementation strategies to address barriers and advance the public health impact of interventions that address important risk and protective factors for all youth.

Plain Language Summary

(1) What is Already Known About the Topic? While many evidence-based interventions (EBIs) exist to address key health issues among youth including substance use and mental health, few of these interventions are effectively implemented in community settings, such as schools. Notable multilevel barriers exist to implement universal prevention in schools. Researchers identify that misalignment between the intervention and the context is a key reason why many implementation efforts do not achieve desired outcomes. (2) What Does This Paper Add? This paper combines the strengths of qualitative and quantitative research methods to identify and understand challenges to intervention-context fit for a comprehensive health curriculum, the Michigan Model for HealthTM (MMH) which is widely adopted throughout Michigan, from the perspective of end users. This paper also utilizes the consolidated framework for implementation research and implementation outcomes framework to guide our understanding of implementing complex interventions and key barriers to implementation in schools. This research provides a foundation to design effective strategies that will balance curriculum fidelity and adaptation to achieve public health objectives. (3) What are the Implications for Practice, Research, or Policy? We need implementation strategies that guide flexibility and fidelity in EBI delivery in schools. While overall teachers felt the curriculum was adaptable and met student needs, they also mentioned specific instances when they would benefit from additional implementation support, such as making adaptations to meet the needs of trauma-exposed youth and keeping up-to-date with emerging drugs. Implementation strategies designed to address these challenges can improve fidelity and ultimately student well-being.

Understanding key implementation determinants for a school-based universal prevention intervention: a qualitative study

This study examined how teachers discuss various factors as impacting their ability to execute with fidelity the Michigan Model for Health (MMH), an evidence-based health universal prevention curriculum widely adopted throughout Michigan. Researchers have found a robust relationship between fidelity and participant outcomes, including in schools. While previous studies have identified barriers that inhibit fidelity, few have focused on identifying key barriers and deepening our understanding of how these factors influence intervention fidelity. We conducted a thematic analysis using the reflexive thematic approach to identify key barriers and facilitators and deepen our understanding of how these factors influence MMH implementation. Guided by the Consolidated Framework for Implementation Research (CFIR) and the Implementation Outcomes Framework, we conducted semistructured interviews with 23 high school health teachers across Michigan. Teachers identified intervention characteristics (e.g., design quality, packaging, and program adaptability), student needs (e.g., trauma exposure, substances), and the fit between the intervention and the context as factors that contributed to acceptability. They also discussed the curriculum and its alignment with their teaching style and/or experiences as contributing to fidelity. Teachers shared how they would often go "off protocol" to improve intervention-context fit and meet students' needs. Our results identified acceptability, a perceptual implementation outcome, as demonstrating an important role in shaping the relationship between CFIR factors and fidelity. Results provide guidance for systematically designing implementation strategies that address key barriers to improve acceptability, enhance fidelity, and ultimately achieve desired public health objectives.

A RE-AIM evaluation in early adopters to iteratively improve the online BeUpstanding™ program supporting workers to sit less and move more

Background

The web-based BeUpstanding program supports desk workers to sit less and move more. Successfully translated from a research-delivered intervention, BeUpstanding has gone through iterative development and evaluation phases in preparation for wide-scale implementation. In the third planned “early-adopters” phase (01/09/2017–11/06/2019), the program was made freely-available online. An integrated delivery and evaluation platform was also developed to enable workplace champions to run and evaluate the intervention within their work team independent of researcher support. Using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework, this study reports on the extent to which the program and processes were “fit-for-purpose” for a national implementation trial across the indicators of uptake (reach and adoption), implementation and engagement, and effectiveness for behaviour change.

Methods

Data were collected via the online surveys embedded in the program and through program access analytics. Descriptive data (with linearized variance for the clustered staff-level data) and results from mixed models (repeated data and clustering for pre-post changes) are reported.

Results

Despite purposeful limited promotion, uptake was good, with 182 Australian users initially registering (208 total) and 135 (from 113 organisations) then completing the sign-up process. Recruitment reached users across Australia and in 16 of 19 Australian industries. Implementation was inconsistent and limited, with signed-up users completing 0 to 14 of the program’s 14 steps and only 7 (5.2%) completing all seven core steps. Many champions (n = 69, 51.1%) had low engagement (1 day toolkit usage) and few (n = 30, 22%) were highly engaged (> 1 day toolkit usage and surveyed staff). Although only 18 users (7 organisations) performed the pre- and post-program staff evaluations (337 and 167 staff, respectively), pre-post changes showed the program effectively reduced workplace sitting by − 9.0% (95% CI -12.0, − 5.9%).

Discussion

The program had uptake across industries and across Australia, but implementation and engagement varied widely. Few workplaces completed the evaluation components. In those that did, the program was effective for the primary outcome (workplace sitting). Conducting a planned early adopters phase and a comprehensive evaluation according to RE-AIM helped highlight necessary program improvements to make it more suitable for wide-scale implementation and evaluation.

Trial registration

Australian and New Zealand Clinic Trials Registry ACTRN12617000682347. Date registered: 12/05/2017.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-021-11993-1.

Equitable Implementation of Artificial Intelligence in Medical Imaging: What Can be Learned from Implementation Science?

Artificial intelligence (AI) has been rapidly adopted in various health care domains. Molecular imaging, accordingly, has demonstrated growing academic and commercial interest in AI. Unprepared and inequitable implementation and scale-up of AI in health care may pose challenges. Implementation of AI, as a complex intervention, may face various barriers, at individual, interindividual, organizational, health system, and community levels. To address these barriers, recommendations have been developed to consider health equity as a critical lens to sensitize implementation, engage stakeholders in implementation and evaluation, recognize and incorporate the iterative nature of implementation, and integrate equity and implementation in early-stage AI research.

The pragmatic, rapid, and iterative dissemination and implementation (PRIDI) cycle: adapting to the dynamic nature of public health emergencies (and beyond)

BACKGROUND

Public health emergencies-such as the 2020 COVID-19 pandemic-accelerate the need for both evidence generation and rapid dissemination and implementation (D&I) of evidence where it is most needed. In this paper, we reflect on how D&I frameworks and methods can be pragmatic (i.e., relevant to real-world context) tools for rapid and iterative planning, implementation, evaluation, and dissemination of evidence to address public health emergencies.

THE PRAGMATIC, RAPID, AND ITERATIVE D&I (PRIDI) CYCLE

The PRIDI cycle is based on a "double-loop" learning process that recognizes the need for responsiveness and iterative adaptation of implementation cycle (inner loop) to the moving landscapes, presented by the outer loops of emerging goals and desired outcomes, emerging interventions and D&I strategies, evolving evidence, and emerging characteristics and needs of individuals and contexts. Stakeholders iteratively evaluate these surrounding landscapes of implementation, and reconsider implementation plans and activities.

CONCLUSION

Even when the health system priority is provision of the best care to the individuals in need, and scientists are focused on development of effective diagnostic and therapeutic technologies, planning for D&I is critical. Without a flexible and adaptive process of D&I, which is responsive to emerging evidence generation cycles, and closely connected to the needs and priorities of stakeholders and target users through engagement and feedback, the interventions to mitigate public health emergencies (e.g., COVID-19 pandemic), and other emerging issues, will have limited reach and impact on populations that would most benefit. The PRIDI cycle is intended to provide a pragmatic approach to support planning for D&I throughout the evidence generation and usage processes.