How Dissemination and Implementation Science Can Contribute to the Advancement of Learning Health Systems

Learning Health System to rapidly improve the implementation of a school physical activity policy

BACKGROUND

Learning Health Systems (LHS) - characterised by cycles of evidence generation and application - are increasingly recognised for their potential to improve public health interventions and optimise health impacts; however there is little evidence of their application in the context of public health practice. Here, we describe how an Australian public health unit applied a LHS approach to successfully improve a model of support for implementation of a school-based physical activity policy.

METHODS

This body of work was undertaken in the context of a strong research-practice partnership. Core LHS capabilities included: i) partnerships and stakeholder engagement; ii) workforce development and learning health communities; iii) multi-disciplinary scientific expertise; iv) practice data collection and management system; v) evidence surveillance and synthesis; and vi) governance and organisational processes of decision making. Three cycles of data generation and application were used. Within each cycle, randomised controlled trials conducted in NSW primary schools were used to generate data on the support model's effectiveness for improving schools' implementation of a government physical activity policy, its delivery costs, and process measures such as adoption and acceptability. Each type of data were analysed independently, synthesised, and then presented to a multi-disciplinary team of researchers and practitioners, in consult with stakeholders, leading to collaborative decisions for incremental improvements to the support model.

RESULTS

Cycle 1 tested the first version of the support model (composed of five implementation strategies targeting identified barriers of policy implementation) and showed the model's feasibility and efficacy for improving schools' policy implementation. Data-informed changes were made to enhance impact, including the addition of three implementation strategies to address outstanding barriers. Cycle 2 (now, testing a package of eight implementation strategies) established the model's effectiveness and cost-effectiveness for improving school's policy implementation. Data-informed changes were made to reduce delivery costs, specifically adapting the costliest strategies to reduce in-person contact from external support personnel. Cycle 3 showed that the adaptations minimised the relative cost of delivery without adversely impacting on the effect.

CONCLUSIONS

Through this process, we identified an effective, cost-effective, acceptable and scalable policy implementation support model for service delivery. This provides important information to inform or support LHS approaches for other agencies seeking to optimise the health impact of evidence-based interventions.

Leveraging the health equity implementation framework to foster an equity focus in medical education

Teaching equitable clinical practice is of critical importance, yet how best to do so remains unknown. Educators utilize implementation science frameworks to disseminate clinical evidence-based practices (EBP). The Health Equity Implementation Framework (HEIF) is one of these frameworks, and it delineates how health equity may be concomitantly assessed and addressed in planning the implementation of an EBP. The HEIF therefore lays a strong foundation to understand and explain barriers and facilitators to implementation through an equity lens, making it well-suited for use by medical educators. Three equity-focused frames of reference within the model include (1) the clinical encounter, (2) societal context, and (3) culturally relevant factors, herein referred to as domains. The HEIF provides a structure for prospective and retrospective assessment of how EBP are taught and ultimately incorporated into clinical practice by trainees, with specific attention to delivering equitable care. We present three examples of common topics in internal medicine, contextualized by the three equity domains of the HEIF. We additionally acknowledge the limitations of this framework as a research tool with complex features that may not be suitable for brief teaching in the clinical environment. We propose a 360-degree learner assessment to ensure implementation of this framework is successful. By encouraging trainees to explore the narrative experiences of their patients and examine their own implicit biases, the HEIF provides a structure to address gaps in knowledge about delivering equitable care.

Use of the iPRISM webtool in a learning community to assess implementation context and fit of a novel clinical decision support tool for physical therapy triage in acute care hospitals

BACKGROUND

The iPRISM webtool is an interactive tool designed to aid the process of applying the Practical, Robust Implementation and Sustainability Model (PRISM) for the assessment of and fit with context. A learning community (LC) is a multidisciplinary group of partners addressing a complex problem. Our LC coproduced the Physical TheraPy frEqueNcy Clinical decIsion support tooL (PT-PENCIL) to guide the use of physical therapist services in acute care hospitals.

OBJECTIVE

To describe our LC's activities to co-produce the PT-PENCIL, use of the iPRISM webtool to assess its preimplementation context and fit, and develop a multicomponent implementation strategy for the PT-PENCIL.

DESIGN

A descriptive research design.

SETTING

Three tertiary care hospitals.

PARTICIPANTS

Thirteen LC partners: six clinical physical therapists, three rehabilitation managers, three researchers, and a bioinformaticist.

INTERVENTIONS

Not applicable.

OUTCOME MEASURES

Using the iPRISM webtool, expected fit of the PT-PENCIL was rated 1 (not aligned) to 6 (well aligned) for each PRISM domain and expected reach, effectiveness, adoption, implementation, and maintenance were rated 1 (not likely at all) to 6 (very likely). Discrete implementation strategies were identified from the Expert Recommendations for Implementing Change.

RESULTS

The process spanned 18 meetings over 8 months. Ten LC partners completed the iPRISM webtool. PRISM domains with the lowest expected alignment were the "implementation and sustainability infrastructure" (mean = 4.7 out of 6; range = 3-6) and the "external environment" (mean = 4.9 of 6; range = 4-6). Adoption was the outcome with the lowest expected likelihood (mean = 4.5 out of 6; range = 1-6). Six discrete implementation strategies were identified and combined into a multicomponent strategy.

CONCLUSIONS

Within a LC, we used existing implementation science resources to co-produce a novel clinical decision support tool for acute care physical therapists and develop a strategy for its implementation. Our methodology can be replicated for similar projects given the public availability of each resource used.

Funding Learning Health System Research: Challenges and Strategies

PURPOSE

A growing number of health systems are establishing learning health system (LHS) programs, where research focuses on rapidly improving the health system's internal operations and performance. The authors examine funding challenges facing such initiatives and identify strategies for managing tensions between reliance on external research funding and directly contributing to improvement and learning within the researchers' own system.

METHOD

Qualitative case studies of LHS research programs in 5 health systems were performed via 38 semistructured interviews (October 2019-April 2021) with 35 diverse respondents. Inductive and deductive rapid qualitative analysis supported interview, system-level, and cross-system summaries and analysis.

RESULTS

External funding awards to LHS researchers facilitated some internal improvement and learning, scientific advancements, and the reputation of researchers and their systems, but reliance on external funding also challenged researchers' responsiveness to concerns of system leaders, managers, practitioners, and system needs. Gaps between external funding requirements and internally focused projects arose in objectives, practical applicability, audiences, timetables, routines, skill sets, and researchers' careers. To contribute more directly to system improvement, LHS researchers needed to collaborate with clinicians and other nonresearchers and pivot between long research studies and shorter, dynamic improvement, evaluation, and data analysis projects. With support from system executives, LHS program leaders employed several strategies to enhance researchers' internal contributions. They aligned funded-research topics with long-term system needs, obtained internal funding for implementing and sustaining practice change, and diversified funding sources.

CONCLUSIONS

To foster LHS research contributions to internal system learning and improvement, LHS program leaders need to manage tensions between concentrating on externally funded research and fulfilling their mission of providing research-based services to their own system. Health system executives can support LHS programs by setting clear goals for them; appropriately staffing, budgeting, and incentivizing LHS researchers; and developing supportive, system-wide teamwork, skill development programs, and data infrastructures.

Fall Tailoring Interventions for Patient Safety Brazil Program: an evaluability study in a teaching hospital

OBJECTIVES

to present the theoretical model, logic model, and the analysis and judgment matrix of the Fall TIPS Brazil Program.

METHODS

a qualitative, participatory research approach, in the form of an evaluability study, encompassing the phases (1) problem analysis; (2) program design, development, and adaptation to the Brazilian context; (3) program dissemination. Data were collected through document analysis and workshops.

RESULTS

through document analysis, workshops with stakeholders from the participating institution, and validation with key informants, it was possible to identify the program's objectives, expected outcomes, and the target audience. This allowed the construction of theoretical and logic models and, through evaluative questions, the identification of indicators for the evaluation of the Fall TIPS Brazil Program.

FINAL CONSIDERATIONS

this study has provided insights into the Fall TIPS program, the topic of hospital fall prevention, and the proposed models and indicators can be employed in the implementation and future evaluative processes of the program.

Social Determinants of Health Impacting the Experience of Young Adults With Cancer at a Single Community Urban Hospital: A Retrospective Cohort Study

Adolescent and young adult (AYA) cancer patients receive palliative medicine consultation at a late stage and face diagnostic delays. Failure to address social determinants of health (SDOH) and AYA-specific needs can adversely impact patient experience. This retrospective observational cohort study used data from chart review to assess the frequency of SDOH impacting AYA patients and setting of initial diagnosis at a US urban safety-net hospital. The association of SDOH variables with delays in treatment, loss of follow-up, and no-shows was tested using Chi-square and t-tests. One hundred seventy five patient charts were reviewed. Sixty-two percent were diagnosed in acute care settings. Substance use disorders, financial, employment, and insurance issues were associated with delayed treatment, with weak to moderate effect sizes. Mental health diagnoses, substance use disorder, homelessness, and financial burdens were associated with patient no-shows, with moderate to large effect sizes. Twenty-five percent of patients received palliative medicine consultation; 70% of these occurred at end of life. This study demonstrates the impact of SDOH on AYA cancer care and the need for policy allowing for intervention on SDOH.

Implementation science for cancer control: One center's experience addressing context, adaptation, equity, and sustainment

Implementation science (IS) has great potential to enhance the frequency, speed, and quality of the translation of evidence-based programs, policies, products, and guidelines into practice. Progress has been made, but with some notable exceptions, this promise has not been achieved for cancer prevention and control. We discuss five interrelated but conceptually distinct, crosscutting issues important to accelerate IS for cancer prevention and control and how our Colorado Implementation Science Center in Cancer Control (COISC3) addressed these issues. These needs and opportunities include more fully addressing changing, multi-level context; guiding rapid, iterative adaptations; evaluating innovative approaches to engagement and health equity; greater attention to costs and economic issues; and sustainability. We summarize conceptual issues; evaluation needs and capacity building activities and then provide examples of how our IS center addressed these five needs for cancer prevention and control. We discuss changes made to address priorities of (i) guiding adaptations of implementation strategies to address changing context and (ii) working on issues identified and prioritized by our primary care partners rather than the research team. We conclude with discussion of lessons learned, limitations, and directions for future research and practice in IS to enhance cancer prevention and control as well as translational behavioral medicine more generally.

Implementation Mapping for Managing Patients at High Risk for Hereditary Cancer

INTRODUCTION

Currently, no standard workflow exists for managing patients with pathogenic variants that put them at higher risk for hereditary cancers. Therefore, follow-up care for individuals with pathogenic variants is logistically challenging and results in poor guideline adherence. To address this challenge, authors created clinical management strategies for individuals identified at high risk for hereditary cancers.

METHODS

An implementation mapping approach was used to develop and evaluate the establishment of a Hereditary Cancer Clinic at the Medical University of South Carolina throughout in 2022. This approach consisted of 5 steps: conduct a needs assessment, identify objectives, select implementation strategies, produce implementation protocols, and develop an evaluation plan. The needs assessment consisted of qualitative interviews with patients (n=11), specialists (n=9), and members of the implementation team (n=4). Interviews were coded using the Consolidated Framework for Implementation Research to identify barriers and facilitators to establishment of the Hereditary Cancer Clinic. Objectives were identified, and then the team selected implementation strategies and produced implementation protocols to address concerns identified during the needs assessment. Authors conducted a second round of patient interviews to assess patient education materials.

RESULTS

The research team developed a long-term evaluation plan to guide future assessment of implementation, service, and clinical/patient outcomes.

CONCLUSIONS

This approach provides the opportunity for real-time enhancements and impact, with strategies for care specialists, patients, and implementation teams. Findings support ongoing efforts to improve patient management and outcomes while providing an opportunity for long-term evaluation of implementation strategies and guidelines for patients at high risk for hereditary cancers.

Leveraging artificial intelligence to advance implementation science: potential opportunities and cautions

BACKGROUND

The field of implementation science was developed to address the significant time delay between establishing an evidence-based practice and its widespread use. Although implementation science has contributed much toward bridging this gap, the evidence-to-practice chasm remains a challenge. There are some key aspects of implementation science in which advances are needed, including speed and assessing causality and mechanisms. The increasing availability of artificial intelligence applications offers opportunities to help address specific issues faced by the field of implementation science and expand its methods.

MAIN TEXT

This paper discusses the many ways artificial intelligence can address key challenges in applying implementation science methods while also considering potential pitfalls to the use of artificial intelligence. We answer the questions of "why" the field of implementation science should consider artificial intelligence, for "what" (the purpose and methods), and the "what" (consequences and challenges). We describe specific ways artificial intelligence can address implementation science challenges related to (1) speed, (2) sustainability, (3) equity, (4) generalizability, (5) assessing context and context-outcome relationships, and (6) assessing causality and mechanisms. Examples are provided from global health systems, public health, and precision health that illustrate both potential advantages and hazards of integrating artificial intelligence applications into implementation science methods. We conclude by providing recommendations and resources for implementation researchers and practitioners to leverage artificial intelligence in their work responsibly.

CONCLUSIONS

Artificial intelligence holds promise to advance implementation science methods ("why") and accelerate its goals of closing the evidence-to-practice gap ("purpose"). However, evaluation of artificial intelligence's potential unintended consequences must be considered and proactively monitored. Given the technical nature of artificial intelligence applications as well as their potential impact on the field, transdisciplinary collaboration is needed and may suggest the need for a subset of implementation scientists cross-trained in both fields to ensure artificial intelligence is used optimally and ethically.

The iPRISM webtool: an interactive tool to pragmatically guide the iterative use of the Practical, Robust Implementation and Sustainability Model in public health and clinical settings

BACKGROUND

To increase uptake of implementation science (IS) methods by researchers and implementers, many have called for ways to make it more accessible and intuitive. The purpose of this paper is to describe the iPRISM webtool (Iterative, Practical, Robust Implementation and Sustainability Model) and how this interactive tool operationalizes PRISM to assess and guide a program's (a) alignment with context, (b) progress on pragmatic outcomes, (c) potential adaptations, and (d) future sustainability across the stages of the implementation lifecycle.

METHODS

We used an iterative human-centered design process to develop the iPRISM webtool.

RESULTS

We conducted user-testing with 28 potential individual and team-based users who were English and Spanish speaking from diverse settings in various stages of implementing different types of programs. Users provided input on all aspects of the webtool including its purpose, content, assessment items, visual feedback displays, navigation, and potential application. Participants generally expressed interest in using the webtool and high likelihood of recommending it to others. The iPRISM webtool guides English and Spanish-speaking users through the process of iteratively applying PRISM across the lifecycle of a program to facilitate systematic assessment and alignment with context. The webtool summarizes assessment responses in graphical and tabular displays and then guides users to develop feasible and impactful adaptations and corresponding action plans. Equity considerations are integrated throughout.

CONCLUSIONS

The iPRISM webtool can intuitively guide individuals and teams from diverse settings through the process of using IS methods to iteratively assess and adapt different types of programs to align with the context across the implementation lifecycle. Future research and application will continue to develop and evaluate this IS resource.

The Academic Learning Health System: A Framework for Integrating the Multiple Missions of Academic Medical Centers

The learning health system (LHS) has emerged over the past 15 years as a concept for improving health care delivery. Core aspects of the LHS concept include: promoting improved patient care through organizational learning, innovation, and continuous quality improvement; identifying, critically assessing, and translating knowledge and evidence into improved practices; building new knowledge and evidence around how to improve health care and health outcomes; analyzing clinical data to support learning, knowledge generation, and improved patient care; and engaging clinicians, patients, and other stakeholders in processes of learning, knowledge generation, and translation. However, the literature has paid less attention to how these LHS aspects may integrate with the multiple missions of academic medical centers (AMCs). The authors define an academic learning health system (aLHS) as an LHS built around a robust academic community and central academic mission, and they propose 6 features that emphasize how an aLHS differs from an LHS. An aLHS capitalizes on embedded academic expertise in health system sciences; engages the full spectrum of translational investigation from mechanistic basic sciences to population health; builds pipelines of experts in LHS sciences and clinicians with fluency in practicing in an LHS; applies core LHS principles to the development of curricula and clinical rotations for medical students, housestaff, and other learners; disseminates knowledge more broadly to advance the evidence for clinical practice and health systems science methods; and addresses social determinants of health, creating community partnerships to mitigate disparities and improve health equity. As AMCs evolve, the authors expect that additional differentiating features and ways to operationalize the aLHS will be identified and hope this article stimulates further discussion around the intersection of the LHS concept and AMCs.

Comparative effectiveness of generic commercial versus locally customized clinical decision support tools to reduce prescription of nonsteroidal anti-inflammatory drugs for patients with heart failure

OBJECTIVE

To compare the effectiveness of 2 clinical decision support (CDS) tools to avoid prescription of nonsteroidal anti-inflammatory drugs (NSAIDs) in patients with heart failure (HF): a "commercial" and a locally "customized" alert.

METHODS

We conducted a retrospective cohort study of 2 CDS tools implemented within a large integrated health system. The commercial CDS tool was designed according to third-party drug content and EHR vendor specifications. The customized CDS tool underwent a user-centered design process informed by implementation science principles, with input from a cross disciplinary team. The customized CDS tool replaced the commercial CDS tool. Data were collected from the electronic health record via analytic reports and manual chart review. The primary outcome was effectiveness, defined as whether the clinician changed their behavior and did not prescribe an NSAID.

RESULTS

A random sample of 366 alerts (183 per CDS tool) was evaluated that represented 355 unique patients. The commercial CDS tool was effective for 7 of 172 (4%) patients, while the customized CDS tool was effective for 81 of 183 (44%) patients. After adjusting for age, chronic kidney disease, ejection fraction, NYHA class, concurrent prescription of an opioid or acetaminophen, visit type (inpatient or outpatient), and clinician specialty, the customized alerts were at 24.3 times greater odds of effectiveness compared to the commercial alerts (OR: 24.3 CI: 10.20-58.06).

CONCLUSION

Investing additional resources to customize a CDS tool resulted in a CDS tool that was more effective at reducing the total number of NSAID orders placed for patients with HF compared to a commercially available CDS tool.

Advancing health equity through implementation science: Identifying and examining measures of the outer setting

BACKGROUND

Implementation science (IS) could accelerate progress toward achieving health equity goals. However, the lack of attention to the outer setting where interventions are implemented limits applicability and generalizability of findings to different populations, settings, and time periods. We developed a data resource to assess outer setting across seven centers funded by the National Cancer Institute's IS Centers in Cancer Control (ISC3) Network Program.

OBJECTIVE

To describe the development of the Outer Setting Data Resource and characterize the county-level outer context across Centers.

METHODS

Our Data Resource captures seven key environments, including: (1) food; (2) physical; (3) economic; (4) social; (5) health care; (6) cancer behavioral and screening; and (7) cancer-related policy. Data were obtained from public sources including the US Census and American Community Survey. We present medians and interquartile ranges based on the distribution of all counties in the US, all ISC3 centers, and within each Center for twelve selected measures. Distributions of each factor are compared with the national estimate using single sample sign tests.

RESULTS

ISC3 centers' catchment areas include 458 counties and over 126 million people across 28 states. The median percentage of population living within ½ mile of a park is higher in ISC3 counties (38.0%, interquartile range (IQR): 16.0%-59.0%) compared to nationally (18.0%, IQR: 7.0%-38.0%; p < 0.0001). The median percentage of households with no broadband access is significantly lower in ISC3 counties (28.4%, IQR: 21.4%-35.6%) compared the nation overall (32.8%, IQR: 25.8%-41.2%; p < 0.0001). The median unemployment rate was significantly higher in ISC3 counties (5.2%, IQR: 4.1%-6.4%) compared to nationally (4.9%, 3.6%-6.3%, p = 0.0006).

CONCLUSIONS

Our results indicate that the outer setting varies across Centers and often differs from the national level. These findings demonstrate the importance of assessing the contextual environment in which interventions are implemented and suggest potential implications for intervention generalizability and scalability.

An Empirical Study on the Use of Digital Technologies to Achieve Cost-Effectiveness in Healthcare Management

Objectives: Healthcare cost reduction is one of the major challenges of the current era. This study was based on the general system theory-based view to assess the significance of sensing communication technologies and processing actuation technologies in improving healthcare quality, leading to cost reduction. Moreover, the contingent rule of healthcare supply chain management in enhancing the influence of improved quality on healthcare cost reduction was also empirically tested. Methods: The sample of the study comprised 337 middle and senior healthcare managers employed in various government and private hospitals and health institutions in Jakarta, Indonesia. The administrative departments of each hospital and health institution was visited to take their consent to conduct this survey at their clinical and non-clinical departments. The data collected was analyzed using SmartPLS ver. 4 software. Results: Results reveal a significant direct and indirect influence of sensing communication technologies and processing actuation technologies on achieving cost-effectiveness in the healthcare sector, in the presence of perceived quality improvement as a mediator. However, the strength of the associations varied and was based on highly reliable and familiar nature of sensing communication technologies compared to processing actuation technologies which were emerging and gaining popularity in recent years. Conclusion: Considering the healthcare cost as a critical factor based on limited resources in emerging economies, healthcare institutions/centers should use digital technologies to achieve cost-effectiveness for providing healthcare facilities in the industry 4.0 era.

How the VA is training the Next-Generation workforce for learning health systems

Objectives

The U.S. Department of Veterans Affairs (VA) has been a national leader in Learning Health System (LHS) implementation due to its combined mission of research, education, clinical care, and emergency preparedness. We describe the current VA LHS training ecosystem within the Veterans Health Administration's Office of Academic Affiliations (OAA), Office of Research and Development (ORD), ORD's Health Services Research and Development (HSR&D) program, and Innovation Ecosystem (IE), including lessons learned regarding their sustainment.

Methods

The VA LHS training ecosystem is based on the Learning Loop and HSR&D Quality Enhancement Research Initiative (QUERI) Roadmap, which describes VA learning opportunities, underlying infrastructures, and core competencies.

Results

VA-focused LHS educational programs include data-to-knowledge initiatives in health sciences and analytics, for example, OAA/HSR&D health services and informatics research fellowships; knowledge-to-performance opportunities in implementation and quality improvement, for example, QUERI Learning Hubs and IEs' Diffusion of Excellence Initiative; and performance-to-data embedded opportunities, for example, IE's entrepreneur fellowship programs and QUERI's Advancing Diversity in Implementation Leadership. These training programs are supported by combined VA research and clinical operations investments in funding, informatics, governance, and processes. Lessons learned include ongoing alignment of research funding with operational priorities and capacity, relentless recruitment and retention of implementation, system, and information scientists especially from under-represented groups, sustainment of data infrastructures suitable for research and quality improvement, and ensuring sustainable funding opportunities for researchers to work on system-wide health care problems.

Conclusions

There is an urgent need to expand training opportunities in LHSs, especially as health care is increasingly driven by multiple interested parties, impacted by persistent health disparities exacerbated by emerging public health threats, and rapid technology growth. With ongoing alignment of research and clinical goals, foundational support through research funding, underlying clinical operations infrastructures, and active engagement interested parties, VA's LHS training ecosystem promotes a more LHS-savvy, 21st century workforce.