Recommendations for strengthening the role of embedded researchers to accelerate implementation in health systems: Findings from a state-of-the-art (SOTA) conference workgroup

Applying Implementation Science to Advance Electronic Health Record-Driven Learning Health Systems: Case Studies, Challenges, and Recommendations

With the widespread implementation of electronic health records (EHRs), there has been significant progress in developing learning health systems (LHSs) aimed at improving health and health care delivery through rapid and continuous knowledge generation and translation. To support LHSs in achieving these goals, implementation science (IS) and its frameworks are increasingly being leveraged to ensure that LHSs are feasible, rapid, iterative, reliable, reproducible, equitable, and sustainable. However, 6 key challenges limit the application of IS to EHR-driven LHSs: barriers to team science, limited IS experience, data and technology limitations, time and resource constraints, the appropriateness of certain IS approaches, and equity considerations. Using 3 case studies from diverse health settings and 1 IS framework, we illustrate these challenges faced by LHSs and offer solutions to overcome the bottlenecks in applying IS and utilizing EHRs, which often stymie LHS progress. We discuss the lessons learned and provide recommendations for future research and practice, including the need for more guidance on the practical application of IS methods and a renewed emphasis on generating and accessing inclusive data.

Understanding the training, mentorship, and professional development priorities of early career embedded researchers

INTRODUCTION

Health systems are constantly evolving in response to existing and emerging health challenges and are increasingly adopting the Quintuple Aim to guide transformation and improvement efforts. Addressing health challenges and achieving the Quintuple Aim (enhancing patient experience, improving healthcare provider experience, promoting population health, optimising the value of healthcare services, and advancing health equity) may be enhanced with the use of a Learning Health Systems approach that fosters the real-time use of data and evidence to inform improvement efforts and harnesses embedded researchers to co-produce timely, relevant evidence to address priorities. Training programs have emerged to build embedded research capacity within health system organisations and have focused predominantly on the postdoctoral career stage, with little attention paid to the early career researcher (ECR) stage. The objective of this study was to understand ECR training and mentorship needs in the embedded research context to inform the creation new or adaptation of existing programs to build embedded ECR capacity.

METHODS

This study used a qualitative approach to garner insight from embedded and applied scholars and health systems leaders in Canada from various professional backgrounds and at various career stages using a combination of focus group discussions, key informant interviews, and an online survey. Thematic content analysis was used to examine the responses of study participants within the interview themes.

RESULTS

Twenty-six (26) participants were included in the study. Results were organised according to four key themes: (1) key competencies and skills needed by embedded ECRs; (2) additional training and capacity development needs; (3) training delivery approaches; and (4) enablers and challenges faced by embedded ECRs. Results highlight the importance of supporting ECRs to develop their leadership and organisational management capabilities; their knowledge of and ability to use research approaches that are well-suited to real-world, complex, evolving environments; and their opportunities to learn with and from each other and mentors. Results underscore the perceived importance of context, including being embedded in a supportive environment that values research and evidence and of academic incentives that recognise and value real-world research impact. The challenges of responding to shifting organisational and system priorities were identified. Additional insights from health systems leaders were also highlighted.

CONCLUSION

This study identified the multifaceted needs of embedded ECRs and the challenges they face within healthcare systems. Designing new programs or tailoring existing ones to address these needs would build their capacity, foster career progression, and ensure their impact as leaders of evidence-informed health system improvement which is crucial for achieving the Quintuple Aim.

Funding Learning Health System Research: Challenges and Strategies

PURPOSE

A growing number of health systems are establishing learning health system (LHS) programs, where research focuses on rapidly improving the health system's internal operations and performance. The authors examine funding challenges facing such initiatives and identify strategies for managing tensions between reliance on external research funding and directly contributing to improvement and learning within the researchers' own system.

METHOD

Qualitative case studies of LHS research programs in 5 health systems were performed via 38 semistructured interviews (October 2019-April 2021) with 35 diverse respondents. Inductive and deductive rapid qualitative analysis supported interview, system-level, and cross-system summaries and analysis.

RESULTS

External funding awards to LHS researchers facilitated some internal improvement and learning, scientific advancements, and the reputation of researchers and their systems, but reliance on external funding also challenged researchers' responsiveness to concerns of system leaders, managers, practitioners, and system needs. Gaps between external funding requirements and internally focused projects arose in objectives, practical applicability, audiences, timetables, routines, skill sets, and researchers' careers. To contribute more directly to system improvement, LHS researchers needed to collaborate with clinicians and other nonresearchers and pivot between long research studies and shorter, dynamic improvement, evaluation, and data analysis projects. With support from system executives, LHS program leaders employed several strategies to enhance researchers' internal contributions. They aligned funded-research topics with long-term system needs, obtained internal funding for implementing and sustaining practice change, and diversified funding sources.

CONCLUSIONS

To foster LHS research contributions to internal system learning and improvement, LHS program leaders need to manage tensions between concentrating on externally funded research and fulfilling their mission of providing research-based services to their own system. Health system executives can support LHS programs by setting clear goals for them; appropriately staffing, budgeting, and incentivizing LHS researchers; and developing supportive, system-wide teamwork, skill development programs, and data infrastructures.

Connecting the science and practice of implementation - applying the lens of context to inform study design in implementation research

The saying "horses for courses" refers to the idea that different people and things possess different skills or qualities that are appropriate in different situations. In this paper, we apply the analogy of "horses for courses" to stimulate a debate about how and why we need to get better at selecting appropriate implementation research methods that take account of the context in which implementation occurs. To ensure that implementation research achieves its intended purpose of enhancing the uptake of research-informed evidence in policy and practice, we start from a position that implementation research should be explicitly connected to implementation practice. Building on our collective experience as implementation researchers, implementation practitioners (users of implementation research), implementation facilitators and implementation educators and subsequent deliberations with an international, inter-disciplinary group involved in practising and studying implementation, we present a discussion paper with practical suggestions that aim to inform more practice-relevant implementation research.

A matter of trust: Commitment to act for health equity

We believe these recommendations constitute "minimum requirements" for health care organizations to move toward greater health equity. As health systems, standards-setting organizations, national and private purchaser organizations, and thought leaders, we represent organizations in the health care ecosystem that can both advise on strategies for adopting the recommendations and have the power and leverage to cause their implementation. We commit individually and collectively to use our leverage to propel their implementation at our own institutions and across the county. We very much hope others will join us.

Retrospective Impact Evaluation Continuing to Prove Challenging Irrespective of Setting: A Study of Research Impact Enablers and Challenges Cloaked as an Impact Evaluation? Comment on "'We're Not Providing the Best Care If We Are Not on the Cutting Edge of Research': A Research Impact Evaluation at a Regional Australian Hospital and Health Service"

The original article provides a detailed and insightful presentation of enablers and detractors for research participation, translation, and impact, at a regional Australian hospital and health service. This information builds on existing knowledge, from the perspective of a non-metropolitan healthcare organisation. It stands to inform all healthcare organisations keen to embed research into their institutions. However, what the article fails to do is present the results of the research impact evaluation in a systematic and useful way for the reader to assess the benefits of research investment by a healthcare organisation including delivery of better quality care and improved patient outcomes. This commentary suggests why such information is critical to justify continued research investment by healthcare organisations and to showcase the potential benefits of the embedded research model. It also discusses the limitations of undertaking impact evaluation retrospectively and suggests that a prospective approach coupled with proper data collection systems and processes upfront could help future reporting of organisational research impact.

Evaluating clinician-led quality improvement initiatives: A system-wide embedded research partnership at Stanford Medicine

Introduction

Many healthcare delivery systems have developed clinician-led quality improvement (QI) initiatives but fewer have also developed in-house evaluation units. Engagement between the two entities creates unique opportunities. Stanford Medicine funded a collaboration between their Improvement Capability Development Program (ICDP), which coordinates and incentivizes clinician-led QI efforts, and the Evaluation Sciences Unit (ESU), a multidisciplinary group of embedded researchers with expertise in implementation and evaluation sciences.

Aim

To describe the ICDP-ESU partnership and report key learnings from the first 2 y of operation September 2019 to August 2021.

Methods

Department-level physician and operational QI leaders were offered an ESU consultation to workshop design, methods, and overall scope of their annual QI projects. A steering committee of high-level stakeholders from operational, clinical, and research perspectives subsequently selected three projects for in-depth partnered evaluation with the ESU based on evaluability, importance to the health system, and broader relevance. Selected project teams met regularly with the ESU to develop mixed methods evaluations informed by relevant implementation science frameworks, while aligning the evaluation approach with the clinical teams' QI goals.

Results

Sixty and 62 ICDP projects were initiated during the 2 cycles, respectively, across 18 departments, of which ESU consulted with 15 (83%). Within each annual cycle, evaluators made actionable, summative findings rapidly available to partners to inform ongoing improvement. Other reported benefits of the partnership included rapid adaptation to COVID-19 needs, expanded clinician evaluation skills, external knowledge dissemination through scholarship, and health system-wide knowledge exchange. Ongoing considerations for improving the collaboration included the need for multi-year support to enable nimble response to dynamic health system needs and timely data access.

Conclusion

Presence of embedded evaluation partners in the enterprise-wide QI program supported identification of analogous endeavors (eg, telemedicine adoption) and cross-cutting lessons across QI efforts, clinician capacity building, and knowledge dissemination through scholarship.

How the VA is training the Next-Generation workforce for learning health systems

Objectives

The U.S. Department of Veterans Affairs (VA) has been a national leader in Learning Health System (LHS) implementation due to its combined mission of research, education, clinical care, and emergency preparedness. We describe the current VA LHS training ecosystem within the Veterans Health Administration's Office of Academic Affiliations (OAA), Office of Research and Development (ORD), ORD's Health Services Research and Development (HSR&D) program, and Innovation Ecosystem (IE), including lessons learned regarding their sustainment.

Methods

The VA LHS training ecosystem is based on the Learning Loop and HSR&D Quality Enhancement Research Initiative (QUERI) Roadmap, which describes VA learning opportunities, underlying infrastructures, and core competencies.

Results

VA-focused LHS educational programs include data-to-knowledge initiatives in health sciences and analytics, for example, OAA/HSR&D health services and informatics research fellowships; knowledge-to-performance opportunities in implementation and quality improvement, for example, QUERI Learning Hubs and IEs' Diffusion of Excellence Initiative; and performance-to-data embedded opportunities, for example, IE's entrepreneur fellowship programs and QUERI's Advancing Diversity in Implementation Leadership. These training programs are supported by combined VA research and clinical operations investments in funding, informatics, governance, and processes. Lessons learned include ongoing alignment of research funding with operational priorities and capacity, relentless recruitment and retention of implementation, system, and information scientists especially from under-represented groups, sustainment of data infrastructures suitable for research and quality improvement, and ensuring sustainable funding opportunities for researchers to work on system-wide health care problems.

Conclusions

There is an urgent need to expand training opportunities in LHSs, especially as health care is increasingly driven by multiple interested parties, impacted by persistent health disparities exacerbated by emerging public health threats, and rapid technology growth. With ongoing alignment of research and clinical goals, foundational support through research funding, underlying clinical operations infrastructures, and active engagement interested parties, VA's LHS training ecosystem promotes a more LHS-savvy, 21st century workforce.

How Dissemination and Implementation Science Can Contribute to the Advancement of Learning Health Systems

Many health systems are working to become learning health systems (LHSs), which aim to improve the value of health care by rapidly, continuously generating evidence to apply to practice. However, challenges remain to advance toward the aspirational goal of becoming a fully mature LHS. While some important challenges have been well described (i.e., building system-level supporting infrastructure and the accessibility of inclusive, integrated, and actionable data), other key challenges are underrecognized, including balancing evaluation rapidity with rigor, applying principles of health equity and classic ethics, focusing on external validity and reproducibility (generalizability), and designing for sustainability. Many LHSs focus on continuous learning cycles, but with limited consideration of issues related to the rapidity of these learning cycles, as well as the sustainability or generalizability of solutions. Some types of data have been consistently underrepresented, including patient-reported outcomes and preferences, social determinants, and behavioral and environmental data, the absence of which can exacerbate health disparities. A promising approach to addressing many challenges that LHSs face may be found in dissemination and implementation (D&I) science. With an emphasis on multilevel dynamic contextual factors, representation of implementation partner engagement, pragmatic research, sustainability, and generalizability, D&I science methods can assist in overcoming many of the challenges facing LHSs. In this article, the authors describe the current state of LHSs and challenges to becoming a mature LHS, propose solutions to current challenges, focusing on the contributions of D&I science with other methods, and propose key components and characteristics of a mature LHS model that others can use to plan and develop their LHSs.

Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem

BACKGROUND

Implementation science is at a sufficiently advanced stage that it is appropriate for the field to reflect on progress thus far in achieving its vision, with a goal of charting a path forward. In this debate, we offer such reflections and report on potential threats that might stymie progress, as well as opportunities to enhance the success and impact of the field, from the perspective of a group of US-based researchers.

MAIN BODY

Ten mid-career extramurally funded US-based researchers completed a "pre-mortem" or a group brainstorming exercise that leverages prospective hindsight to imagine that an event has already occurred and to generate an explanation for it - to reduce the likelihood of a poor outcome. We came to consensus on six key themes related to threats and opportunities for the field: (1) insufficient impact, (2) too much emphasis on being a "legitimate science," (3) re-creation of the evidence-to-practice gap, (4) difficulty balancing accessibility and field coherence, (5) inability to align timelines and priorities with partners, and (6) overly complex implementation strategies and approaches.

CONCLUSION

We submit this debate piece to generate further discussion with other implementation partners as our field continues to develop and evolve. We hope the key opportunities identified will enhance the future of implementation research in the USA and spark discussion across international groups. We will continue to learn with humility about how best to implement with the goal of achieving equitable population health impact at scale.

Community engagement to improve access to healthcare: a comparative case study to advance implementation science for transgender health equity

BACKGROUND

Recent calls to action have been made for Implementation Science to attend to health inequities at the intersections of race, gender, and social injustice in the United States. Transgender people, particularly Black and Latina transgender women, experience a range of health inequities and social injustices. In this study, we compared two processes of transgender community engagement in Los Angeles and in Chicago as an implementation strategy to address inequitable access to care; we adapted and extended the Exploration Planning Implementation and Sustainment (EPIS) framework for transgender health equity.

METHODS

A comparative case method and the EPIS framework were used to examine parallel implementation strategies of transgender community engagement to expand access to care. To foster conceptual development and adaptation of EPIS for trans health equity, the comparative case method required detailed description, exploration, and analyses of the community-engagement processes that led to different interventions to expand access. In both cities, the unit of analysis was a steering committee made up of local transgender and cisgender stakeholders.

RESULTS

Both steering committees initiated their exploration processes with World Café-style, transgender community-engaged events in order to assess community needs and structural barriers to healthcare. The steering committees curated activities that amplified the voices of transgender community members among stakeholders, encouraging more effective and collaborative ways to advance transgender health equity. Based on analysis and findings from the Los Angeles town hall, the steering committee worked with a local medical school, extending the transgender medicine curriculum, and incorporating elements of transgender community-engagement. The Chicago steering committee determined from their findings that the most impactful intervention on structural racism and barriers to healthcare access would be to design and pilot an employment program for Black and Latina transgender women.

CONCLUSION

In Los Angeles and Chicago, transgender community engagement guided implementation processes and led to critical insights regarding specific, local barriers to healthcare. The steering committee itself represented an important vehicle for individual-, organizational-, and community-level relationship and capacity building. This comparative case study highlights key adaptations of EPIS toward the formation of an implementation science framework for transgender health equity.

Maintaining Implementation through Dynamic Adaptations (MIDAS): protocol for a cluster-randomized trial of implementation strategies to optimize and sustain use of evidence-based practices in Veteran Health Administration (VHA) patients

Background

The adoption and sustainment of evidence-based practices (EBPs) is a challenge within many healthcare systems, especially in settings that have already strived but failed to achieve longer-term goals. The Veterans Affairs (VA) Maintaining Implementation through Dynamic Adaptations (MIDAS) Quality Enhancement Research Initiative (QUERI) program was funded as a series of trials to test multi-component implementation strategies to sustain optimal use of three EBPs: (1) a deprescribing approach intended to reduce potentially inappropriate polypharmacy; (2) appropriate dosing and drug selection of direct oral anticoagulants (DOACs); and (3) use of cognitive behavioral therapy as first-line treatment for insomnia before pharmacologic treatment. We describe the design and methods for a harmonized series of cluster-randomized control trials comparing two implementation strategies.

Methods

For each trial, we will recruit 8–12 clinics (24–36 total). All will have access to relevant clinical data to identify patients who may benefit from the target EBP at that clinic and provider. For each trial, clinics will be randomized to one of two implementation strategies to improve the use of the EBPs: (1) individual-level academic detailing (AD) or (2) AD plus the team-based Learn. Engage. Act. Process. (LEAP) quality improvement (QI) learning program. The primary outcomes will be operationalized across the three trials as a patient-level dichotomous response (yes/no) indicating patients with potentially inappropriate medications (PIMs) among those who may benefit from the EBP. This outcome will be computed using month-by-month administrative data. Primary comparison between the two implementation strategies will be analyzed using generalized estimating equations (GEE) with clinic-level monthly (13 to 36 months) percent of PIMs as the dependent variable. Primary comparative endpoint will be at 18 months post-baseline. Each trial will also be analyzed independently.

Discussion

MIDAS QUERI trials will focus on fostering sustained use of EBPs that previously had targeted but incomplete implementation. Our implementation approaches are designed to engage frontline clinicians in a dynamic optimization process that integrates the use of actional clinical data and making incremental changes, designed to be feasible within busy clinical settings.

Trial registration

ClinicalTrials.gov: NCT05065502. Registered October 4, 2021—retrospectively registered.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00297-z.

Applying User-Centered Design to Develop Practical Strategies that Address Overuse in Primary Care

INTRODUCTION

Engaging patients and frontline clinicians in re-designing clinical care is essential for improving care delivery in a complex clinical environment. This study sought to assess an innovative user-centered design approach to improving clinical care quality, focusing on the use cases of de-intensifying non-beneficial care within the following areas: (1) de-intensifying diabetes treatment in high-risk patients; (2) stopping screening for carotid artery stenosis in asymptomatic patients; and (3) stopping colorectal cancer screening in average-risk, older adults.

METHODS

The user-centered design approach, consisting of patient and patient-clinician charrettes (defined as intensive workshops where key stakeholders collaborate to develop creative solutions to a specific problem) and participant surveys, has been described previously. Following the charrettes, we used inductive coding to identify and categorize themes emerging from the de-intensification ideas prioritized by participants as well as facilitator notes and audio recordings from the charrettes.

RESULTS

Thirty-five patients participated in the patient design charrettes, generating 134 unique de-intensification ideas and prioritizing 32, which were then distilled into six patient-generated principles of de-intensification by the study team. These principles provided a starting point for a subsequent patient-clinician charrette. In this follow-up charrette, 9 patients who had participated in an earlier patient design charrette collaborated with 7 clinicians to generate 63 potential de-intensification solutions. Six of these potential solutions were developed into multi-faceted, fully operationalized de-intensification strategies.

DISCUSSION

The de-intensification strategies that patients and clinicians prioritized and operationalized during the co-design charrette process were detailed and multi-faceted. Each component of a strategy had a rationale based on feasibility, practical considerations, and ways of overcoming barriers. The charrette-based process may be a useful way to engage clinicians and patients in developing the complex and multi-faceted strategies needed to improve care delivery.

Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum

BACKGROUND

The challenges of implementing evidence-based innovations (EBIs) are widely recognized among practitioners and researchers. Context, broadly defined as everything outside the EBI, includes the dynamic and diverse array of forces working for or against implementation efforts. The Consolidated Framework for Implementation Research (CFIR) is one of the most widely used frameworks to guide assessment of contextual determinants of implementation. The original 2009 article invited critique in recognition for the need for the framework to evolve. As implementation science has matured, gaps in the CFIR have been identified and updates are needed. Our team is developing the CFIR 2.0 based on a literature review and follow-up survey with authors. We propose an Outcomes Addendum to the CFIR to address recommendations from these sources to include outcomes in the framework.

MAIN TEXT

We conducted a literature review and surveyed corresponding authors of included articles to identify recommendations for the CFIR. There were recommendations to add both implementation and innovation outcomes from these sources. Based on these recommendations, we make conceptual distinctions between (1) anticipated implementation outcomes and actual implementation outcomes, (2) implementation outcomes and innovation outcomes, and (3) CFIR-based implementation determinants and innovation determinants.

CONCLUSION

An Outcomes Addendum to the CFIR is proposed. Our goal is to offer clear conceptual distinctions between types of outcomes for use with the CFIR, and perhaps other determinant implementation frameworks as well. These distinctions can help bring clarity as researchers consider which outcomes are most appropriate to evaluate in their research. We hope that sharing this in advance will generate feedback and debate about the merits of our proposed addendum.

Balancing reality in embedded research and evaluation: Low vs high embeddedness

Embedding research and evaluation into organizations is one way to generate “practice‐based” evidence needed to accelerate implementation of evidence‐based innovations within learning health systems. Organizations and researchers/evaluators vary greatly in how they structure and operationalize these collaborations. One key aspect is the degree of embeddedness: from low embeddedness where researchers/evaluators are located outside organizations (eg, outside evaluation consultants) to high embeddedness where researchers/evaluators are employed by organizations and thus more deeply involved in program evolution and operations. Pros and cons related to the degree of embeddedness (low vs high) must be balanced when developing these relationships. We reflect on this process within the context of an embedded, mixed‐methods evaluation of the Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program. Considerations that must be balanced include: (a) low vs high alignment of goals; (b) low vs high involvement in strategic planning; (c) observing what is happening vs being integrally involved with programmatic activities; (d) reporting findings at the project's end vs providing iterative findings and recommendations that contribute to program evolution; and (e) adhering to predetermined aims vs adapting aims in response to evolving partner needs.