An Analysis of the Learning Health System in Its First Decade in Practice: Scoping Review

Alpha NSW: What would it take to create a state-wide paediatric population-level learning health system?

BACKGROUND

The health and well-being of children in the first 2000 days has a lasting effect on educational achievement and long-term chronic disease in later life. However, the lack of integration between high-quality data, analytic capacity and timely health improvement initiatives means practitioners, service leaders and policymakers cannot use data effectively to plan and evaluate early intervention services and monitor high-level health outcomes.

OBJECTIVE

Our exploratory study aimed to develop an in-depth understanding of the system and clinical requirements of a state-wide paediatric learning health system (LHS) that uses routinely collected data to not only identify where the inequities and variation in care are, but also to also inform service development and delivery where it is needed most.

METHOD

Our approach included reviewing exemplars of how administrative data are used in Australia; consulting with clinical, policy and data stakeholders to determine their needs for a child health LHS; mapping the existing data points collected across the first 2000 days of a child's life and geospatially locating patterns of key indicators for child health needs.

RESULTS

Our study identified the indicators that are available and accessible to inform service delivery and demonstrated the potential of using routinely collected administrative data to identify the gap between health needs and service availability.

CONCLUSION

We recommend improving data collection, accessibility and integration to establish a state-wide LHS, whereby there is a streamlined process for data cleaning, analysis and visualisation to help identify populations in need in a timely manner.

Lessons learned from a pay-for-performance scheme for appropriate prescribing using electronic health records from general practices in the Netherlands

INTRODUCTION

A nationwide pay-for-performance (P4P) scheme was introduced in the Netherlands between 2018 and 2023 to incentivize appropriate prescribing in general practice. Appropriate prescribing was operationalised as adherence to prescription formularies and measured based on electronic health records (EHR) data. We evaluated this P4P scheme from a learning health systems perspective.

METHODS

We conducted semi-structured interviews with 15 participants representing stakeholders of the scheme: general practitioners (GPs), health insurers, pharmacists, EHR suppliers and formulary committees. We used a thematic approach for data analysis.

RESULTS

Using EHR data showed several benefits, but lack of uniformity of EHR systems hindered consistent measurements. Specific indicators were favoured over general indicators as they allow GPs to have more control over their performance. Most participants emphasized the need for GPs to jointly reflect on their performance. Communication to GPs appeared to be challenging. Partly because of these challenges, impact of the scheme on prescribing behaviour was perceived as limited. However, several unexpected positive effects of the scheme were mentioned, such as better EHR recording habits.

CONCLUSIONS

This study identified benefits and challenges useful for future P4P schemes in promoting appropriate care with EHR data. Enhancing uniformity in EHR systems is crucial for more consistent quality measurements. Future P4P schemes should focus on high-quality feedback, peer-to-peer learning and establish a single point of communication for healthcare providers.

Implementing AI in Hospitals to Achieve a Learning Health System: Systematic Review of Current Enablers and Barriers

BACKGROUND

Efforts are underway to capitalize on the computational power of the data collected in electronic medical records (EMRs) to achieve a learning health system (LHS). Artificial intelligence (AI) in health care has promised to improve clinical outcomes, and many researchers are developing AI algorithms on retrospective data sets. Integrating these algorithms with real-time EMR data is rare. There is a poor understanding of the current enablers and barriers to empower this shift from data set-based use to real-time implementation of AI in health systems. Exploring these factors holds promise for uncovering actionable insights toward the successful integration of AI into clinical workflows.

OBJECTIVE

The first objective was to conduct a systematic literature review to identify the evidence of enablers and barriers regarding the real-world implementation of AI in hospital settings. The second objective was to map the identified enablers and barriers to a 3-horizon framework to enable the successful digital health transformation of hospitals to achieve an LHS.

METHODS

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were adhered to. PubMed, Scopus, Web of Science, and IEEE Xplore were searched for studies published between January 2010 and January 2022. Articles with case studies and guidelines on the implementation of AI analytics in hospital settings using EMR data were included. We excluded studies conducted in primary and community care settings. Quality assessment of the identified papers was conducted using the Mixed Methods Appraisal Tool and ADAPTE frameworks. We coded evidence from the included studies that related to enablers of and barriers to AI implementation. The findings were mapped to the 3-horizon framework to provide a road map for hospitals to integrate AI analytics.

RESULTS

Of the 1247 studies screened, 26 (2.09%) met the inclusion criteria. In total, 65% (17/26) of the studies implemented AI analytics for enhancing the care of hospitalized patients, whereas the remaining 35% (9/26) provided implementation guidelines. Of the final 26 papers, the quality of 21 (81%) was assessed as poor. A total of 28 enablers was identified; 8 (29%) were new in this study. A total of 18 barriers was identified; 5 (28%) were newly found. Most of these newly identified factors were related to information and technology. Actionable recommendations for the implementation of AI toward achieving an LHS were provided by mapping the findings to a 3-horizon framework.

CONCLUSIONS

Significant issues exist in implementing AI in health care. Shifting from validating data sets to working with live data is challenging. This review incorporated the identified enablers and barriers into a 3-horizon framework, offering actionable recommendations for implementing AI analytics to achieve an LHS. The findings of this study can assist hospitals in steering their strategic planning toward successful adoption of AI.

Leveraging data to support health equity in an integrated delivery and finance system

Introduction

To accelerate healthcare transformation and advance health equity, scientists in learning health systems (LHSs) require ready access to integrated, comprehensive data that includes information on social determinants of health (SDOH).

Methods

We describe how an integrated delivery and finance system leveraged its learning ecosystem to advance health equity through (a) a cross-sector initiative to integrate healthcare and human services data for better meeting clients' holistic needs and (b) a system-level initiative to collect and use patient-reported SDOH data for connecting patients to needed resources.

Results

Through these initiatives, we strengthened our health system's capacity to meet diverse patient needs, address health disparities, and improve health outcomes. By sharing and integrating healthcare and human services data, we identified 281 000 Shared Services Clients and enhanced care management for 100 adult Medicaid/Special Needs Plan members. Over a 1-year period, we screened 9173 (37%) patients across UPMC's Women's Health Services Line and connected over 700 individuals to social services and supports.

Conclusions

Opportunities exist for LHSs to improve, expand, and sustain their innovative data practices. As learnings continue to emerge, LHSs will be well positioned to accelerate healthcare transformation and advance health equity.

Harmonizing the Tripartite Mission in Academic Family Medicine: A Longitudinal Case Example

Academic practices and departments are defined by a tripartite mission of care, education, and research, conceived as being mutually reinforcing. But in practice, academic faculty have often experienced these 3 missions as competing rather than complementary priorities. This siloed approach has interfered with innovation as a learning health system in which the tripartite missions reinforce each other in practical ways. This paper presents a longitudinal case example of harmonizing academic missions in a large family medicine department so that missions and people interact in mutually beneficial ways to create value for patients, learners, and faculty. We describe specific experiences, implementation, and examples of harmonizing missions as a feasible strategy and culture. "Harmonized" means that no one mission subordinates or drives out the others; each mission informs and strengthens the others (quickly in practice) while faculty experience the triparate mission as a coherent whole faculty job. Because an academic department is a complex system of work and relationships, concepts for leading a complex adaptive system were employed: (1) a "good enough" vision, (2) frequent and productive interactions, and (3) a few simple rules. These helped people harmonize their work without telling them exactly what to do, when, and how. Our goal here is to highlight concrete examples of harmonizing missions as a feasible operating method, suggesting ways it builds a foundation for a learning health system and potentially improving faculty well-being.

Now Is the Time to Strengthen Government-Academic Data Infrastructures to Jump-Start Future Public Health Crisis Response

During public health crises, the significance of rapid data sharing cannot be overstated. In attempts to accelerate COVID-19 pandemic responses, discussions within society and scholarly research have focused on data sharing among health care providers, across government departments at different levels, and on an international scale. A lesser-addressed yet equally important approach to sharing data during the COVID-19 pandemic and other crises involves cross-sector collaboration between government entities and academic researchers. Specifically, this refers to dedicated projects in which a government entity shares public health data with an academic research team for data analysis to receive data insights to inform policy. In this viewpoint, we identify and outline documented data sharing challenges in the context of COVID-19 and other public health crises, as well as broader crisis scenarios encompassing natural disasters and humanitarian emergencies. We then argue that government-academic data collaborations have the potential to alleviate these challenges, which should place them at the forefront of future research attention. In particular, for researchers, data collaborations with government entities should be considered part of the social infrastructure that bolsters their research efforts toward public health crisis response. Looking ahead, we propose a shift from ad hoc, intermittent collaborations to cultivating robust and enduring partnerships. Thus, we need to move beyond viewing government-academic data interactions as 1-time sharing events. Additionally, given the scarcity of scholarly exploration in this domain, we advocate for further investigation into the real-world practices and experiences related to sharing data from government sources with researchers during public health crises.

Lessons learned from the development of a national registry on dementia care and support based on linked national health and administrative data

Introduction

This paper provides insight into the development of the Dutch Dementia Care and Support Registry and the lessons that can be learned from it. The aim of this Registry was to contribute to quality improvement in dementia care and support.

Methods

This paper describes how the Registry was set up in four stages, reflecting the four FAIR principles: the selection of data sources (Findability); obtaining access to the selected data sources (Accessibility); data linkage (Interoperability); and the reuse of data (Reusability).

Results

The linkage of 16 different data sources, including national routine health and administrative data appeared to be technically and legally feasible. The linked data in the Registry offers rich information about (the use of) care for persons with dementia across various healthcare settings, including but not limited to primary care, secondary care, long-term care and medication use, that cannot be obtained from single data sources.

Conclusions

A key lesson learned is that in order to reuse the data for quality improvement in practice, it is essential to involve healthcare professionals in setting up the Registry and to guide them in the interpretation of the data.

Building Learning Healthcare Systems for Critical Care Medicine

Learning healthcare systems are an evolving way of integrating informatics, analytics, and continuous improvement into daily practice in healthcare. This article discusses strategies to build learning healthcare systems for critical care medicine.

Progress with the Learning Health System 2.0: a rapid review of Learning Health Systems' responses to pandemics and climate change

BACKGROUND

Pandemics and climate change each challenge health systems through increasing numbers and new types of patients. To adapt to these challenges, leading health systems have embraced a Learning Health System (LHS) approach, aiming to increase the efficiency with which data is translated into actionable knowledge. This rapid review sought to determine how these health systems have used LHS frameworks to both address the challenges posed by the COVID-19 pandemic and climate change, and to prepare for future disturbances, and thus transition towards the LHS2.0.

METHODS

Three databases (Embase, Scopus, and PubMed) were searched for peer-reviewed literature published in English in the five years to March 2023. Publications were included if they described a real-world LHS's response to one or more of the following: the COVID-19 pandemic, future pandemics, current climate events, future climate change events. Data were extracted and thematically analyzed using the five dimensions of the Institute of Medicine/Zurynski-Braithwaite's LHS framework: Science and Informatics, Patient-Clinician Partnerships, Continuous Learning Culture, Incentives, and Structure and Governance.

RESULTS

The search yielded 182 unique publications, four of which reported on LHSs and climate change. Backward citation tracking yielded 13 additional pandemic-related publications. None of the climate change-related papers met the inclusion criteria. Thirty-two publications were included after full-text review. Most were case studies (n = 12, 38%), narrative descriptions (n = 9, 28%) or empirical studies (n = 9, 28%). Science and Informatics (n = 31, 97%), Continuous Learning Culture (n = 26, 81%), Structure and Governance (n = 23, 72%) were the most frequently discussed LHS dimensions. Incentives (n = 21, 66%) and Patient-Clinician Partnerships (n = 18, 56%) received less attention. Twenty-nine papers (91%) discussed benefits or opportunities created by pandemics to furthering the development of an LHS, compared to 22 papers (69%) that discussed challenges.

CONCLUSIONS

An LHS 2.0 approach appears well-suited to responding to the rapidly changing and uncertain conditions of a pandemic, and, by extension, to preparing health systems for the effects of climate change. LHSs that embrace a continuous learning culture can inform patient care, public policy, and public messaging, and those that wisely use IT systems for decision-making can more readily enact surveillance systems for future pandemics and climate change-related events.

TRIAL REGISTRATION

PROSPERO pre-registration: CRD42023408896.

Using a Rapid Learning Health System for Stratified Care in Emerging Adult Mental Health Services: Protocol for the Implementation of Patient-Reported Outcome Measures

BACKGROUND

Mental illness among emerging adults is often difficult to ameliorate due to fluctuating symptoms and heterogeneity. Recently, innovative approaches have been developed to improve mental health care for emerging adults, including (1) implementing patient-reported outcome measures (PROMs) to assess illness severity and inform stratified care to assign emerging adults to a treatment modality commensurate with their level of impairment and (2) implementing a rapid learning health system in which data are continuously collected and analyzed to generate new insights, which are then translated to clinical practice, including collaboration among clients, health care providers, and researchers to co-design and coevaluate assessment and treatment strategies.

OBJECTIVE

The aim of the study is to determine the feasibility and acceptability of implementing a rapid learning health system to enable a measurement-based, stratified care treatment strategy for emerging adults.

METHODS

This study takes place at a specialty clinic serving emerging adults (age 16-24 years) in Calgary, Canada, and involves extensive collaboration among researchers, providers, and youth. The study design includes six phases: (1) developing a transdiagnostic platform for PROMs, (2) designing an initial stratified care model, (3) combining the implementation of PROMs with stratified care, (4) evaluating outcomes and disseminating results, (5) modification of stratified care based on data derived from PROMs, and (6) spread and scale to new sites. Qualitative and quantitative feedback will be collected from health care providers and youth throughout the implementation process. These data will be analyzed at regular intervals and used to modify the way future services are delivered. The RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework is used to organize and evaluate implementation according to 3 key objectives: improving treatment selection, reducing average wait time and treatment duration, and increasing the value of services.

RESULTS

This project was funded through a program grant running from 2021 to 2026. Ethics approval for this study was received in February 2023. Presently, we have developed a system of PROMs and organized clinical services into strata of care. We will soon begin using PROMs to assign clients to a stratum of care and using feedback from youth and clinicians to understand how to improve experiences and outcomes.

CONCLUSIONS

This study has key implications for researchers and clinicians looking to understand how to customize emerging adult mental health services to improve the quality of care and satisfaction with care. This study has significant implications for mental health care systems as part of a movement toward value-based health care.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/51667.

An Intelligent Search & Retrieval System (IRIS) and Clinical and Research Repository for Decision Support Based on Machine Learning and Joint Kernel-based Supervised Hashing

Large-scale, multi-site collaboration is becoming indispensable for a wide range of research and clinical activities in oncology. To facilitate the next generation of advances in cancer biology, precision oncology and the population sciences it will be necessary to develop and implement data management and analytic tools that empower investigators to reliably and objectively detect, characterize and chronicle the phenotypic and genomic changes that occur during the transformation from the benign to cancerous state and throughout the course of disease progression. To facilitate these efforts it is incumbent upon the informatics community to establish the workflows and architectures that automate the aggregation and organization of a growing range and number of clinical data types and modalities ranging from new molecular and laboratory tests to sophisticated diagnostic imaging studies. In an attempt to meet those challenges, leading health care centers across the country are making steep investments to establish enterprise-wide, data warehouses. A significant limitation of many data warehouses, however, is that they are designed to support only alphanumeric information. In contrast to those traditional designs, the system that we have developed supports automated collection and mining of multimodal data including genomics, digital pathology and radiology images. In this paper, our team describes the design, development and implementation of a multi-modal, Clinical & Research Data Warehouse (CRDW) that is tightly integrated with a suite of computational and machine-learning tools to provide actionable insight into the underlying characteristics of the tumor environment that would not be revealed using standard methods and tools. The System features a flexible Extract, Transform and Load (ETL) interface that enables it to adapt to aggregate data originating from different clinical and research sources depending on the specific EHR and other data sources utilized at a given deployment site.

Cultural health capital and patient partner recruitment into healthcare improvement work

A rising emphasis on patient involvement in clinical research and healthcare improvement has led to the steady incorporation of patients and caregivers into this work. However, interactional factors shaping recruitment processes are not well understood. In this paper, we present a qualitative analysis of interviews with twenty-six patients, family members, engagement staff and healthcare providers who are engaged in healthcare improvement work in the United States. We focus on how stakeholders account for recruitment decisions to participate in healthcare improvement work. We find that expressions of and judgments about patients' and caregivers' cultural health capital shape providers' decisions to extend invitations to participate in healthcare improvement work. These findings extend current conceptualizations of cultural health capital beyond the clinical encounter to reveal factors shaping patient recruitment into healthcare improvement work. In theorizing how cultural health capital shapes action in this new setting, we found that healthcare providers, engagement staff, and patients/caregivers attended to different aspects of cultural health capital when accounting for why they extended or received a recruitment pitch. We further found that participating in healthcare improvement work led to a boost in cultural health capital for patients and caregivers, which they could use to develop transmissible forms of cultural health capital for less centrally involved patients and caregivers. Finally, we describe how participants in healthcare improvement collaboratives account for a lack of diversity among partners. These findings help us hypothesize the consequences of recruitment processes that rely on displays and judgments of cultural health capital and identify possibilities for change. Using the case of healthcare improvement work in Collaborative Learning Health Systems, our findings advance past work on cultural health capital in medical sociology by theorizing the role of cultural health capital in recruitment processes.

Frameworks, guidelines, and tools to develop a learning health system for Indigenous health: An environmental scan for Canada

Introduction

First Nations, Inuit, and Métis (FNIM) peoples experience systemic health disparities within Ontario's healthcare system. Learning health systems (LHS) is a rapidly growing interdisciplinary area with the potential to address these inequitable health outcomes through a comprehensive health system that draws on science, informatics, incentives, and culture for ongoing innovation and improvement. However, global literature is in its infancy with grounding theories and principles still emerging. In addition, there is inadequate information on LHS within Ontario's health care context.

Methods

We conducted an environmental scan between January and April 2021 and again in June 2022 to identify existing frameworks, guidelines, and tools for designing, developing, implementing, and evaluating an LHS.

Results

We found 37 relevant sources. This paper maps the literature and identifies gaps in knowledge based on five key pillars: (a) data and evidence-driven, (b) patient-centeredness, (c) system-supported, (d) cultural competencies enabled, and (e) the learning health system.

Conclusion

We provide recommendations for implementation accordingly. The literature on LHS provides a starting point to address the health disparities of FNIM peoples within the healthcare system but Indigenous community partnerships in LHS development and operation will be key to success.

Learning health system benefits: Development and initial validation of a framework

Introduction

Implementation of research findings in clinical practice often is not realized or only partially achieved, and if so, with a significant delay. Learning health systems (LHSs) hold promise to overcome this problem by embedding clinical research and evidence-based best practices into care delivery, enabling innovation and continuous improvement. Implementing an LHS is a complex process that requires participation and resources of a wide range of stakeholders, including healthcare leaders, clinical providers, patients and families, payers, and researchers. Engaging these stakeholders requires communicating clear, tangible value propositions. Existing models identify broad categories of benefits but do not explicate the full range of benefits or ways they can manifest in different organizations.

Methods

To develop such a framework, a working group with representatives from six Clinical and Translational Science Award (CTSA) hubs reviewed existing literature on LHS characteristics, models, and goals; solicited expert input; and applied the framework to their local LHS experiences.

Results

The Framework of LHS Benefits includes six categories of benefits (quality, safety, equity, patient satisfaction, reputation, and value) relevant for a range of stakeholders and defines key concepts within each benefit. Applying the framework to five LHS case examples indicated preliminary face validity across varied LHS approaches and revealed three dimensions in which the framework is relevant: defining goals of individual LHS projects, facilitating collaboration based on shared values, and establishing guiding tenets of an LHS program or mission.

Conclusion

The framework can be used to communicate the value of an LHS to different stakeholders across varied contexts and purposes, and to identify future organizational priorities. Further validation will contribute to the framework's evolution and support its potential to inform the development of tools to evaluate LHS impact.

21st century (clinical) decision support in nursing and allied healthcare. Developing a learning health system: a reasoned design of a theoretical framework

In this paper, we present a framework for developing a Learning Health System (LHS) to provide means to a computerized clinical decision support system for allied healthcare and/or nursing professionals. LHSs are well suited to transform healthcare systems in a mission-oriented approach, and is being adopted by an increasing number of countries. Our theoretical framework provides a blueprint for organizing such a transformation with help of evidence based state of the art methodologies and techniques to eventually optimize personalized health and healthcare. Learning via health information technologies using LHS enables users to learn both individually and collectively, and independent of their location. These developments demand healthcare innovations beyond a disease focused orientation since clinical decision making in allied healthcare and nursing is mainly based on aspects of individuals' functioning, wellbeing and (dis)abilities. Developing LHSs depends heavily on intertwined social and technological innovation, and research and development. Crucial factors may be the transformation of the Internet of Things into the Internet of FAIR data & services. However, Electronic Health Record (EHR) data is in up to 80% unstructured including free text narratives and stored in various inaccessible data warehouses. Enabling the use of data as a driver for learning is challenged by interoperability and reusability.To address technical needs, key enabling technologies are suitable to convert relevant health data into machine actionable data and to develop algorithms for computerized decision support. To enable data conversions, existing classification and terminology systems serve as definition providers for natural language processing through (un)supervised learning.To facilitate clinical reasoning and personalized healthcare using LHSs, the development of personomics and functionomics are useful in allied healthcare and nursing. Developing these omics will be determined via text and data mining. This will focus on the relationships between social, psychological, cultural, behavioral and economic determinants, and human functioning.Furthermore, multiparty collaboration is crucial to develop LHSs, and man-machine interaction studies are required to develop a functional design and prototype. During development, validation and maintenance of the LHS continuous attention for challenges like data-drift, ethical, technical and practical implementation difficulties is required.

Implementation strategies and outcome measures for advancing learning health systems: a mixed methods systematic review

BACKGROUND

Learning health systems strive to continuously integrate data and evidence into practice to improve patient outcomes and ensure value-based healthcare. While the LHS concept is gaining traction, the operationalization of LHSs is underexplored.

OBJECTIVE

To identify and synthesize the existing evidence on the implementation and evaluation of advancing learning health systems across international health care settings.

METHODS

A mixed methods systematic review was conducted. Six databases (CINAHL, Embase, Medline, PAIS, Scopus and Nursing at Allied Health Database) were searched up to July 2022 for terms related to learning health systems, implementation, and evaluation measures. Any study design, health care setting and population were considered for inclusion. No limitations were placed on language or date of publication. Two reviewers independently screened the titles, abstracts, and full texts of identified articles. Data were extracted and synthesized using a convergent integrated approach. Studies were critically appraised using relevant JBI critical appraisal checklists.

RESULTS

Thirty-five studies were included in the review. Most studies were conducted in the United States (n = 21) and published between 2019 and 2022 (n = 24). Digital data capture was the most common LHS characteristic reported across studies, while patient engagement, aligned governance and a culture of rapid learning and improvement were reported least often. We identified 33 unique strategies for implementing LHSs including: change record systems, conduct local consensus discussions and audit & provide feedback. A triangulation of quantitative and qualitative data revealed three integrated findings related to the implementation of LHSs: (1) The digital infrastructure of LHSs optimizes health service delivery; (2) LHSs have a positive impact on patient care and health outcomes; and (3) LHSs can influence health care providers and the health system.

CONCLUSION

This paper provides a comprehensive overview of the implementation of LHSs in various healthcare settings. While this review identified key implementation strategies, potential outcome measures, and components of functioning LHSs, further research is needed to better understand the impact of LHSs on patient, provider and population outcomes, and health system costs. Health systems researchers should continue to apply the LHS concept in practice, with a stronger focus on evaluation.

Design, implementation, and inferential issues associated with clinical trials that rely on data in electronic medical records: a narrative review

Real world evidence is now accepted by authorities charged with assessing the benefits and harms of new therapies. Clinical trials based on real world evidence are much less expensive than randomized clinical trials that do not rely on "real world evidence" such as contained in electronic health records (EHR). Consequently, we can expect an increase in the number of reports of these types of trials, which we identify here as 'EHR-sourced trials.' 'In this selected literature review, we discuss the various designs and the ethical issues they raise. EHR-sourced trials have the potential to improve/increase common data elements and other aspects of the EHR and related systems. Caution is advised, however, in drawing causal inferences about the relationships among EHR variables. Nevertheless, we anticipate that EHR-CTs will play a central role in answering research and regulatory questions.

Implementation model for a national learning health system (IMPLEMENT-National LHS): a concept analysis and systematic review protocol

INTRODUCTION

Despite efforts and repeated calls to improve the organisation and quality of healthcare and services, and in view of the many challenges facing health systems, the results and capacity to adapt and integrate innovations and new knowledge remain suboptimal. Learning health systems (LHS) may be an effective model to accelerate the application of research for real quality improvement in healthcare. However, while recognising the enormous potential of LHS, the literature suggests the model remains more of an aspiration than a reality.

METHODS AND ANALYSIS

To reach a fine understanding of the implementation of the concepts involved in LHS, we will use a hybrid method which combines concept analyses with systematic review methodology. We will use a two-step analysis, a content analysis to analyse the definitions, uses and attributes of the concept and a systematic review to analyse the concept's implementation mechanisms. We will search eight databases and grey literature and present a broad synthesis of the available evidence regarding design, implementation and evaluation of LHS in a multilevel perspective. We will follow the latest Preferred Reporting Items for Systematic Review and Meta-Analysis statement for conducting and reporting a systematic review. Two reviewers will independently screen the titles and abstracts against the eligibility criteria followed by full-text screening of potentially relevant articles for final inclusion decision. Conflicts will be resolved with a senior author. We will include published primary studies that use qualitative, quantitative or mixed methods. The assessment of risk of bias will be made using the Mixed-Methods Appraisal Tool.

ETHICS AND DISSEMINATION

This systematic review is exempt from ethics approval. The results formulated will highlight evidence-based interventions that support the implementation of a national LHS. They will be of particular interest to decision makers, researchers, managers, clinicians and patients allowing finally to implement the promising proposal of LHSs at national scale.

PROSPERO REGISTRATION NUMBER

CRD42023393565.

The typing is on the wall: Australia's healthcare future needs a digitally capable workforce

Digital health technologies are a proposed solution to improve healthcare delivery and reduce pressures on the healthcare system, but these technologies are new to much of the health workforce. This perspective paper highlights lessons learned from the global experience of rapid digital transformation of health workforces, including fostering a culture of learning, ensuring accreditation and recognition, and adopting a transdisciplinary approach. Evidence-based actions are proposed to address recommendations to (1) ensure foundational workforce digital health capability and (2) build specialist digital health career pathways. Australia must take a national approach and strategically leverage strong collaborations across sectors including healthcare, education and government to ensure a consistent, regulated and sustainable digital workforce capability.

Reducing hospital acquired pressure injury in a learning health center: Making the case for quality

Introduction

The purpose of this descriptive study is to examine a learning health system (LHS) continuous improvement and learning approach as a case for increased quality, standardized processes, redesigned workflows, and better resource utilization. Hospital acquired pressure injuries (HAPI) commonly occur in the hospitalized patient and are costly and preventable. This study examines the effect of a LHS approach to reducing HAPI within a large academic medical center.

Methods

Our learning health center implemented a 6-year series of iterative improvements that included both process and technology changes, with robust data and analytical reforms. In this descriptive, observational study, we retrospectively examined longitudinal data from April 1, 2018 to March 31, 2022, examining the variables of total number of all-stage HAPI counts and average length of stay (ALOS). We also analyzed patient characteristics observed/expected mortality ratios, as well as total patient days, and the case-mix index to determine whether these factors varied over the study period. We used the Agency for Healthcare Research and Quality cost estimates to identify the estimated financial benefit of HAPI reductions on an annualized basis.

Results

HAPI per 1000 patient days for FY 20 (October 1-September 30) and FY 21, decreased from 2.30 to 1.30 and annualized event AHRQ cost estimates for HAPI decreased by $4 786 980 from FY 20 to FY 21. A strong, statistically significant, negative and seemingly counterintuitive correlation was found (r = -.524, P = .003) between HAPI and ALOS.

Conclusions

The LHS efforts directed toward HAPI reduction led to sustained improvements during the study period. These results demonstrate the benefits of a holistic approach to quality improvement offered by the LHS model. The LHS model goes beyond a problem-based approach to process improvement. Rather than targeting a specific problem to solve, the LHS system creates structures that yield process improvement benefits over a continued time period.

Gathering speed and countering tensions in the rapid learning health system

The vision of the learning health system (LHS), conceptualized 15 years ago, is for the rapid generation, use, and spread of high-quality evidence that yields better health experiences, outcomes, efficiencies, and equity in everyday practice settings across communities. However, despite the emergence of many useful LHS frameworks and examples to guide adoption, large gaps remain in the speed and consistency with which evidence is generated and used across the range of settings from the bedside to the policy table. Gaps in progress are not surprising, however, given the tensions that predictably arise when key stakeholders-researchers, health systems, and funders-comingle in these efforts. This commentary examines eight core tensions that naturally arise and offers practical actions that stakeholders can take to address these tensions and speed LHS adoption. The urgency for attenuating these tensions and accelerating health system improvements has never been higher. Timeliness, rigor, and prioritization can be aligned across stakeholders, but only if all partners are intentional about the operational and cultural challenges that exist.

Stroke Learning Health Systems: A Topical Narrative Review With Case Examples

To our knowledge, the adoption of Learning Health System (LHS) concepts or approaches for improving stroke care, patient outcomes, and value have not previously been summarized. This topical review provides a summary of the published evidence about LHSs applied to stroke, and case examples applied to different aspects of stroke care from high and low-to-middle income countries. Our attempt to systematically identify the relevant literature and obtain real-world examples demonstrated the dissemination gaps, the lack of learning and action for many of the related LHS concepts across the continuum of care but also elucidated the opportunity for continued dialogue on how to study and scale LHS advances. In the field of stroke, we found only a few published examples of LHSs and health systems globally implementing some selected LHS concepts, but the term is not common. A major barrier to identifying relevant LHS examples in stroke may be the lack of an agreed taxonomy or terminology for classification. We acknowledge that health service delivery settings that leverage many of the LHS concepts do so operationally and the lessons learned are not shared in peer-reviewed literature. It is likely that this topical review will further stimulate the stroke community to disseminate related activities and use keywords such as learning health system so that the evidence base can be more readily identified.

How alignment between health systems and their embedded research units contributes to system learning

BACKGROUND

There is growing interest in the contributions of embedded, learning health system (LHS), research within healthcare delivery systems. We examined the organization of LHS research units and conditions affecting their contributions to system improvement and learning.

METHODS

We conducted 12 key-informant and 44 semi-structured interviews in six delivery systems engaged in LHS research. Using rapid qualitative analysis, we identified themes and compared: successful versus challenging projects; LHS units and other research units in the same system; and LHS units in different systems.

RESULTS

LHS units operate both independently and as subunits within larger research centers. Contributions of LHS units to improvements and learning are influenced by alignment of facilitating factors within units, within the broader system, and between unit and host system. Key alignment factors were availability of internal (system) funding directing researchers' work toward system priorities; researchers' skills and experiences that fit a system's operational needs; LHS unit subculture supporting system improvement and collaboration with clinicians and other internal stakeholders; applications of external funding to system priorities; and executive leadership for system-wide learning. Mutual understanding and collaboration between researchers, clinicians, and leaders was fostered through direct consultation between LHS unit leaders and system executives and engagement of researchers in clinical and operational activities.

CONCLUSIONS

Embedded researchers face significant challenges to contributing to system improvement and learning. Nevertheless, when appropriately led, organized, and supported by internal funding, they may learn to collaborate effectively with clinicians and system leaders in advancing care delivery toward the learning health system ideal.

Determinants of Implementation of a Critical Care Registry in Asia: Lessons From a Qualitative Study

BACKGROUND

The Collaboration for Research, Implementation, and Training in Critical Care in Asia (CCA) is implementing a critical care registry to capture real-time data to facilitate service evaluation, quality improvement, and clinical studies.

OBJECTIVE

The purpose of this study is to examine stakeholder perspectives on the determinants of implementation of the registry by examining the processes of diffusion, dissemination, and sustainability.

METHODS

This study is a qualitative phenomenological inquiry using semistructured interviews with stakeholders involved in registry design, implementation, and use in 4 South Asian countries. The conceptual model of diffusion, dissemination, and sustainability of innovations in health service delivery guided interviews and analysis. Interviews were coded using the Rapid Identification of Themes from Audio recordings procedure and were analyzed based on the constant comparison approach.

RESULTS

A total of 32 stakeholders were interviewed. Analysis of stakeholder accounts identified 3 key themes: innovation-system fit; influence of champions; and access to resources and expertise. Determinants of implementation included data sharing, research experience, system resilience, communication and networks, and relative advantage and adaptability.

CONCLUSIONS

The implementation of the registry has been possible due to efforts to increase the innovation-system fit, influence of motivated champions, and the support offered by access to resources and expertise. The reliance on individuals and the priorities of other health care actors pose a risk to sustainability.

Setting up a Governance Framework for Secondary Use of Routine Health Data in Nursing Homes: Development Study Using Qualitative Interviews

BACKGROUND

In the nursing home sector, reusing routinely recorded data from electronic health records (EHRs) for knowledge development and quality improvement is still in its infancy. Trust in appropriate and responsible reuse is crucial for patients and nursing homes deciding whether to share EHR data for these purposes. A data governance framework determines who may access the data, under what conditions, and for what purposes. This can help obtain that trust. Although increasing attention is being paid to data governance in the health care sector, little guidance is available on development and implementation of a data governance framework in practice.

OBJECTIVE

This study aims to describe the development process of a governance framework for the "Registry Learning from Data in Nursing Homes," a national registry for EHR data on care delivered by nursing home physicians (in Dutch: specialist ouderengeneeskunde) in Dutch nursing homes-to allow data reusage for research and quality improvement of care.

METHODS

Relevant stakeholders representing practices, policies, and research in the nursing home sector were identified. Semistructured interviews were conducted with 20 people from 14 stakeholder organizations. The main aim of the interviews was to explore stakeholders' perspectives regarding the Registry's aim, data access criteria, and governing bodies' tasks and composition. Interview topics and analyses were guided by 8 principles regarding governance for reusing health data, as described in the literature. Interview results, together with legal advice and consensus discussions by the Registry's consortium partners, were used to shape the rules, regulations, and governing bodies of the governance framework.

RESULTS

Stakeholders valued the involvement of nursing home residents and their representatives, nursing home physicians, nursing homes' boards of directors, and scientists and saw this as a prerequisite for a trustworthy data governance framework. For the Registry, involvement of these groups can be achieved through a procedure in which residents can provide their consent or objection to the reuse of the data, transparency about the decisions made, and providing them a position in a governing body. In addition, a data request approval procedure based on predefined assessment criteria indicates that data reuse by third parties aligns with the aims of the Registry, benefits the nursing home sector, and protects the privacy of data subjects.

CONCLUSIONS

The stakeholders' views, expertise, and knowledge of other frameworks and relevant legislation serve to inform the application of governance principles to the contexts of both the nursing home sector and the Netherlands. Many different stakeholders were involved in the development of the Registry Learning from Data in Nursing Homes' governance framework and will continue to be involved. Engagement of the full range of stakeholders in an early stage of governance framework development is important to generate trust in appropriate and responsible data reuse.

Beyond prediction: Off-target uses of artificial intelligence-based predictive analytics in a learning health system

Introduction

Artificial-intelligence (AI)-based predictive analytics provide new opportunities to leverage rich sources of continuous data to improve patient care through early warning of the risk of clinical deterioration and improved situational awareness.Part of the success of predictive analytic implementation relies on integration of the analytic within complex clinical workflows. Pharmaceutical interventions have off-target uses where a drug indication has not been formally studied for a different indication but has potential for clinical benefit. An analog has not been described in the context of AI-based predictive analytics, that is, when a predictive analytic has been trained on one outcome of interest but is used for additional applications in clinical practice.

Methods

In this manuscript we present three clinical vignettes describing off-target use of AI-based predictive analytics that evolved organically through real-world practice.

Results

Off-target uses included:real-time feedback about treatment effectiveness, indication of readiness to discharge, and indication of the acuity of a hospital unit.

Conclusion

Such practice fits well with the learning health system goals to continuously integrate data and experience to provide.

Health Care Bias and Discrimination Experienced by Lesbian, Gay, Bisexual, Transgender, and Queer Parents of Children With Developmental Disabilities: A Qualitative Inquiry in the United States

INTRODUCTION

This study explored the impact of health care (HC) bias and discrimination on lesbian, gay, bisexual, transgender, and queer (LGBTQ) parents and their children with disabilities in the United States, including the timing of developmental screening and diagnosis.

METHOD

We conducted semistructured interviews with 16 LGBTQ parents of children with developmental concerns or disabilities recruited through a prior national survey. Interviews were transcribed and analyzed using a combined inductive and deductive approach.

RESULTS

Discrimination types reported included noninclusive forms, disclosure challenges, and providers dismissing nongestational parents and diverse families. Few parents reported screening and diagnosis delays. Parents' recommendations included: avoiding assumptions, honoring family diversity, increasing LGBTQ family support, improving HC forms, increasing antibias training, and convening a learning community.

DISCUSSION

Our study advances the knowledge around HC bias and discrimination among LGBTQ parents of children with disabilities. Findings highlight the need for increased LGBTQ-affirming family support and research representing LGBTQ family diversity in U.S. health care.

Improve-mentation for Faster Testing and Spread of Health Service Delivery Innovations

Can we speed the testing, implementation and spread of management innovations in a systematic way to also contribute to scientific knowledge? Researchers and implementers have developed an approach to test and revise a local version of an innovation during its implementation. The chapter starts with a case example of an application of this combination of implementation and quality improvement sciences and practices (improve-mentation). It then summarizes four examples of this approach so as to help understand what improve-mentation is and how it is different from traditional quality improvement and traditional implementation of evidence-based practices. It considers gaps in knowledge that are hindering both more use of improve-mentation to generate scientific knowledge about spread and implementation, as well as more use of improve-mentation by health care service organizations and researchers. It closes by proposing fruitful research and development that can address these knowledge gaps to speed the implementation, sustainment and spread of care and management innovations.

A conceptual framework to describe and evaluate a socially accountable learning health system: Development and application in a northern, rural, and remote setting

Health care and academic institutions are increasingly committing to social accountability, a strategic shift that requires priorities, activities, and evaluations to be co-determined with all relevant partners. Consequently, governments, accreditors, funders, and communities are calling for these institutions to communicate their progress towards social accountability. The purpose of this study was to develop a conceptual framework around a socially accountable learning health system. This article presents an integrated analysis of two studies: (i) a narrative review of 11 prominent social accountability and health services conceptual frameworks and (ii) a reflexive thematic analysis of 18 key informant interviews. Using a systematic conceptual framework development and integrated theory of change/realist evaluation methodologies, we describe a synthesis of these findings to develop a conceptual framework for describing and evaluating socially accountable health professional education. The resulting framework describes assessment phases of social accountability, transitions between phases, learning cycles, and the actors and systems that collectively mobilise social accountability at multiple levels in health and education systems. The framework can be used to evaluate interventions or characterise progress towards social accountability in different settings, as illustrated in the example at the end of the paper. The framework emphasises the significance of designing, mobilising, and evaluating social accountability as part of a contextualised learning health system.

Conceptualising learning healthcare systems and organisations in the context of rehabilitation: a scoping review protocol

Background: Transformative system wide action is needed for healthcare systems to meet the needs of an increasing aging population and changing health needs. One idea is that health systems can become “learning organisations” (LO) or “learning healthcare systems” (LHS) that continuously generate and apply evidence, innovation, quality, and value to provide better care. This is of value to non-acute healthcare settings such as rehabilitation, which are complex, multi-dimensional and multi-disciplinary in nature. Little is known about how these frameworks have been applied to rehabilitation settings. Objective and inclusion criteria: The aim of this scoping review is to systematically map and summarise the literature conceptualising and operationalising LHS and LO in rehabilitation settings. Studies will be included which define a LO or LHS; or describe an operating LHS/LO; or include the translation of research evidence generated from LHS/LO data into healthcare improvement within a rehabilitation context will be included. Study designs such as quantitative, qualitative, mixed method studies, and case studies will be included. Methods: The guidelines from the Joanna Briggs institute methodology for scoping reviews will be used for this review. The literature search will be performed using a three-step search strategy: an initial limited search of two databases has been performed to identify relevant key words and index terms. The developed search string will be adapted and applied across the following databases: OVID MEDLINE, EMBASE, CINAHL Plus, APA PsycINFO and COCHRANE Database of Systematic Reviews. This will be followed by search of the reference lists of selected sources and relevant data-hubs. A draft data extraction framework will be used and updated iteratively to extract data. Frequency counts and qualitative content analysis will be employed to address the research question of how LHS and LO have been conceptualised and operationalised in the context of rehabilitation.

Mapping continuous learning using social network research: a social network study of Australian Genomics as a Learning Health System

OBJECTIVES

To explore a macrolevel Learning Health System (LHS) and examine if an intentionally designed network can foster a collaborative learning community over time. The secondary aim was to demonstrate the application of social network research to the field of LHS.

DESIGN

Two longitudinal online questionnaires of the Australian Genomics learning community considering relationships between network members at three time points: 2016, 2018, 2019. The questionnaire included closed Likert response questions on collaborative learning patterns and open-response questions to capture general perceptions of the community. Social network data were analysed and visually constructed using Gephi V.0.9.2 software, Likert questions were analysed using SPSS, and open responses were analysed thematically using NVivo.

SETTING

Australian Genomic Health Alliance.

PARTICIPANTS

Clinicians, scientists, researchers and community representatives.

RESULTS

Australian Genomics members highlighted the collaborative benefits of the network as a learning community to foster continuous learning in the ever-evolving field of clinical genomics. The learning community grew from 186 members (2016), to 384 (2018), to 439 (2019). Network density increased (2016=0.023, 2018=0.043), then decreased (2019=0.036). Key players remained consistent with potential for new members to achieve focal positions in the network. Informal learning was identified as the most influential learning method for genomic practice.

CONCLUSIONS

This study shows that intentionally building a network provides a platform for continuous learning-a fundamental component for establishing an LHS. The Australian Genomics learning community shows evidence of maturity and sustainability in supporting the continuous learning culture of clinical genomics. The network provides a practical means to spread new knowledge and best practice across the entire field. We show that intentionally designed networks provide the opportunity and means for interdisciplinary learning between diverse agents over time and demonstrate the application of social network research to the LHS field.

Adapting an Interdisciplinary Learning Health System Framework for Academic Health Centers: A Scoping Review

PURPOSE

Learning health systems (LHSs), defined as a systematic process for aligning science, informatics, and clinical practice to integrate providers, researchers, and patients as active participants in an evidence-based care continuum, can provide an ideal environment for academic health centers to rapidly adopt evidence-based guidelines and translate research into practice. However, few LHS frameworks are specifically adapted for academic health centers. The authors wanted to identify the definitions, components, and other features of LHSs to develop an interdisciplinary LHS framework for use within academic health centers.

METHOD

The authors conducted a scoping review of the literature to identify definitions, components, and other features of LHSs that are useful to academic health centers. In January 2021, they searched PubMed, Academic Search Premier, and Scopus databases and identified English-language, peer-reviewed articles pertaining to LHS, LHS frameworks, organization, components, and models. Since the phrase learning health system is relatively new terminology, they conducted a supplemental review with alternative phrases, including embedded research and coordinated or collaborative research network . They used the Knowledge to Action (KTA) Framework to integrate the generation and flow of research into practice.

RESULTS

The primary review retrieved 719 articles and the supplemental review retrieved 209; of these, 49 articles were retained to synthesize common definitions, components, and other features of LHS frameworks. Seven structural components of LHSs were identified: organization and collaborations, performance, ethics and security, scientific approaches, data, information technology, and patient outcomes. An adapted interdisciplinary LHS framework was developed that incorporated research and learning engines derived from the KTA and adaptations of common components and other features within the reviewed articles to fit the interests of providers, researchers, and patients within academic health centers.

CONCLUSIONS

The adapted LHS framework can be used as a dynamic foundation for development and organization of interdisciplinary LHSs within academic health centers.

Fostering the use of Learning Health Systems through a fellowship program for interprofessional clinicians

Introduction

To address Australian workforce needs, we developed a Learning Healthcare System (LHS) Academy fellowship program for clinicians. In the Academy, fellows complete foundational coursework, an LHS project, and other professional development deliverables to foster their future as digital health champions within their organizations. In this paper, we describe the 11-month-long program, as well as our evaluation results from the first 2 months of the program.

Methods

In the first week of the program, we sent all fellows an open-ended survey asking fellows to describe their digital health professional identities and what they expected to achieve from the fellowship program. At 2 months, we sent a follow-up open-ended survey that captured identical measures, their perceived barriers to participation in the program, perceived use of topics in the workplace and to their projects, and recommendations for program improvement. We analyzed the open text responses using qualitative content analysis, to identify categories of responses.

Results

Overall, 2 months into the program, it was evident that participants were finding the teaching model engaging, useful, valuable, and applicable to their work and projects. Fellows perceived barriers to engagement in the program as balancing other commitments, lacking technical expertise, and having difficulty seeing themselves as leaders. Fellows expected that the program will allow them to implement new models of care, provide them with enough expertise to become leaders and champions in digital health, and become mentors for future generations. As far as changes in their professional identity, there was a notable increase in the number of fellows perceiving themselves as leaders.

Conclusion

Fellowship programs are one promising means of developing the healthcare workforce in LHS capabilities. Future studies should describe and evaluate LHS programs, to provide insights and recommendations for other educators interested in implementing similar programs of work within their own institutions.

Making Diabetes Electronic Medical Record Data Actionable: Promoting Benchmarking and Population Health Improvement Using the T1D Exchange Quality Improvement Portal

This article describes how the T1D Exchange Quality Improvement Collaborative leverages an innovative web platform, the QI Portal, to gather and store electronic medical record (EMR) data to promote benchmarking and population health improvement in a type 1 diabetes learning health system. The authors explain the value of the QI Portal, the process for mapping center-level data from EMRs using standardized data specifications, and the QI Portal's unique features for advancing population health.

Using a learning health system framework to examine COVID-19 pandemic planning and response at a Canadian Health Centre

Background

The COVID-19 pandemic has presented a unique opportunity to explore how health systems adapt under rapid and constant change and develop a better understanding of health system transformation. Learning health systems (LHS) have been proposed as an ideal structure to inform a data-driven response to a public health emergency like COVID-19. The aim of this study was to use a LHS framework to identify assets and gaps in health system pandemic planning and response during the initial stages of the COVID-19 pandemic at a single Canadian Health Centre.

Methods

This paper reports the data triangulation stage of a concurrent triangulation mixed methods study which aims to map study findings onto the LHS framework. We used a triangulation matrix to map quantitative (textual and administrative sources) and qualitative (semi-structured interviews) data onto the seven characteristics of a LHS and identify assets and gaps related to health-system receptors and research-system supports.

Results

We identified several health system assets within the LHS characteristics, including appropriate decision supports and aligned governance. Gaps were identified in the LHS characteristics of engaged patients and timely production and use of research evidence.

Conclusion

The LHS provided a useful framework to examine COVID-19 pandemic response measures. We highlighted opportunities to strengthen the LHS infrastructure for rapid integration of evidence and patient experience data into future practice and policy changes.

What implementation strategies and outcome measures are used to transform healthcare organizations into learning health systems? A mixed-methods review protocol

BACKGROUND

A learning health system (LHS) framework provides an opportunity for health system restructuring to provide value-based healthcare. However, there is little evidence showing how to effectively implement a LHS in practice.

OBJECTIVE

A mixed-methods review is proposed to identify and synthesize the existing evidence on effective implementation strategies and outcomes of LHS in an international context.

METHODS

A mixed-methods systematic review will be conducted following methodological guidance from Joanna Briggs Institute (JBI) and PRISMA reporting guidelines. Six databases (CINAHL, Embase, MEDLINE, PAIS, Scopus and Nursing & Allied Health Database) will be searched for terms related to LHS, implementation and evaluation measures. Three reviewers will independently screen the titles, abstracts and full texts of retrieved articles. Studies will be included if they report on the implementation of a LHS in any healthcare setting. Qualitative, quantitative or mixed-methods study designs will be considered for inclusion. No restrictions will be placed on language or date of publication. Grey literature will be considered for inclusion but reviews and protocol papers will be excluded. Data will be extracted from included studies using a standardized extraction form. One reviewer will extract all data and a second will verify. Critical appraisal of all included studies will be conducted by two reviewers. A convergent integration approach to data synthesis will be used, where qualitative and quantitative data will be synthesized separately and then integrated to present overarching findings. Data will be presented in tables and narratively.

CONCLUSION

This review will address a gap in the literature related to implementation of LHS. The findings from this review will provide researchers with a better understanding of how to design and implement LHS interventions. This systematic review was registered in PROSPERO (CRD42022293348).

Frameworks for value-based care in the nonoperating room setting

PURPOSE OF REVIEW

Nonoperating room anesthesia (NORA) presents a unique opportunity for the application of value-based care (VBC) principles to procedures performed in the office-based and nonoperating room inpatient settings. The purpose of this article is to review how value is defined in NORA and enabling principles by which anesthesiologists can maximize value in NORA.

RECENT FINDINGS

In order to drive value, NORA providers can target improvements in clinical outcomes where NORA lags behind operating room-based anesthesia (death, over-sedation, nerve injury), implement protocols focusing on intermediate outcomes/quality (postoperative nausea and vomiting, pain control, hypothermia, delirium), incorporate patient-reported outcomes (PROs) to assess the trajectory of a patient's perioperative care, and reduce costs (direct and indirect) through operational and supply-based efficiencies. Establishing a culture of patient and provider safety first, appropriate patient selection with targeted, perioperative optimization of comorbidities, and efficient deployment of staff, space, and resources are critical enablers for success.

SUMMARY

Value in NORA can be defined as clinical outcomes, quality, patient-reported outcomes, and efficiency divided by the direct and indirect costs for achieving those outcomes. We present a novel framework adapting current VBC practices in operating room anesthesia to the NORA environment.

Incentivizing appropriate prescribing in primary care: Development and first results of an electronic health record-based pay-for-performance scheme

OBJECTIVE

Part of the funding of Dutch General Practitioners (GPs) care is based on pay-for-performance, including an incentive for appropriate prescribing according to guidelines in national formularies. Aim of this paper is to describe the development of an indicator and an infrastructure based on prescription data from GP Electronic Health Records (EHR), to assess the level of adherence to formularies and the effects of the pay-for-performance scheme, thereby assessing the usefulness of the infrastructure and the indicator.

METHODS

Adherence to formularies was calculated as the percentage of first prescriptions by the GP for medications that were included in one of the national formularies used by the GP, based on prescription data from EHRs. Adherence scores were collected quarterly for 2018 and 2019 and subsequently sent to health insurance companies for the pay-for-performance scheme. Adherence scores were used to monitor the effect of the pay-for-performance scheme.

RESULTS

75% (2018) and 83% (2019) of all GP practicesparticipated. Adherence to formularies was around 85% or 95%, depending on the formulary used. Adherence improved significantly, especially for practices that scored lowest in 2018.

DISCUSSION

We found high levels of adherence to national formularies, with small improvements after one year. The infrastructure will be used to further stimulate formulary-based prescribing by implementing more actionable and relevant indicators on adherence scores for GPs.

Acceptability of an interstage home monitoring mobile application for caregivers of children with single ventricle physiology: Toward technology-integrated family management

PURPOSE

Infants with single ventricle physiology experience numerous vulnerable transitions, and the interstage period for shunt-dependent children represents the time of highest risk for morbidity and mortality. Data exchange, physiological monitoring, and communication between clinicians and caregivers through interstage home monitoring are critical. The purpose of this study is to report on the acceptability of a technology-enhanced home monitoring mobile application for interstage family management of children with single ventricle physiology.

DESIGN AND METHODS

This study employed a qualitative descriptive study design and recruited caregivers that were part of a broader quality improvement project where they were beta users of a mobile health application specifically developed for the interstage home monitoring time period.

RESULTS

Eleven caregivers were enrolled in this study that was a part of the early phases of beta testing the mobile application from a human-centered design perspective. In general, the participants had a favorable sentiment toward the technology-integrated family management aspects that the mobile application allowed for during the interstage process. The acceptability findings can be organized through the following themes: time needed for mobile application, family as integrated members of care team, connectedness and confidence, and resolving technical issues.

CONCLUSIONS

Evaluation of the feasibility and acceptability of this technology from the perspective of family/caregivers is a critical component of human-centered design. The integration of technology-facilitated communication shows immense promise for patient populations undergoing vulnerable transitions in care. Future study is needed to determine the role mobile applications have in improved clinical outcomes, enhanced provider clinical-decision support, and family engagement in care.

From "Invented here" to "Use it everywhere!": A Learning health system from bottom and/or top?

Introduction

Departing from a practical problem of how to use digitalization to improve care quality and efficiency, this paper investigates how the concept of Learning Health Systems (LHSs) can be applied to an existing organization. LHSs offer a vision for how healthcare can accelerate both scale-up of innovations and quality improvements at all levels. However, aligning stakeholders at different levels to convergent development is challenging and translation and adaptation of the LHS concept to fit with the existing organization is essential.

Methods

A one-year longitudinal action research (AR) study was conducted within five psychiatric departments at the Sahlgrenska University Hospital in Gothenburg, Sweden. Translation of the LHS concept to the local circumstances within the organization was set as the aim, to both improve practice and further scientific understanding. An AR group led the practical and scholarly work and holistic data were collected, including field notes, documents, recordings, and workshops. Data were analyzed by an insider-outsider approach.

Results

The one-year study is described to provide insights into the process of designing a locally adapted LHS using an AR approach. Practical needs were identified and iteratively matched with theory to form a local LHS model. A conflict between top-down and bottom-up views on development emerged, where higher-level management tended to prioritize uniform solutions and developers local learning. An adapted solution to balance these approaches was negotiated, consisting of a technical and an organizational part.

Conclusions

The conflict between top-down and bottom-up approaches for how to implement LHSs needs to be considered both in practical work to transform care organizations and in scientific studies of LHSs. The approach to translate, rather than instrumentally implement, LHSs to real-world settings is suggested as advantageous. Furthermore, designing such endeavors as AR projects can provide excellent conditions to create LHSs that work in practice.

The pursuit of health equity in digital transformation, health informatics, and the cardiovascular learning healthcare system

African Americans have a higher rate of cardiovascular morbidity and mortality and a lower rate of specialty consultation and treatment than Caucasians. These disparities also exist in the care and treatment of chemotherapy-related cardiovascular complications. African Americans suffer from cardiotoxicity at a higher rate than Caucasians and are underrepresented in clinical trials aimed at preventing cardiovascular injury associated with cancer therapies. To eliminate racial and ethnic disparities in the prevention of cardiotoxicity, an interdisciplinary and innovative approach will be required. Diverse forms of digital transformation leveraging health informatics have the potential to contribute to health equity if they are implemented carefully and thoughtfully in collaboration with minority communities. A learning healthcare system can serve as a model for developing, deploying, and disseminating interventions to minimize health inequities and maximize beneficial impact.

Toward a Learning Health Care System: A Systematic Review and Evidence-Based Conceptual Framework for Implementation of Clinical Analytics in a Digital Hospital

Objective  A learning health care system (LHS) uses routinely collected data to continuously monitor and improve health care outcomes. Little is reported on the challenges and methods used to implement the analytics underpinning an LHS. Our aim was to systematically review the literature for reports of real-time clinical analytics implementation in digital hospitals and to use these findings to synthesize a conceptual framework for LHS implementation.

Methods  Embase, PubMed, and Web of Science databases were searched for clinical analytics derived from electronic health records in adult inpatient and emergency department settings between 2015 and 2021. Evidence was coded from the final study selection that related to (1) dashboard implementation challenges, (2) methods to overcome implementation challenges, and (3) dashboard assessment and impact. The evidences obtained, together with evidence extracted from relevant prior reviews, were mapped to an existing digital health transformation model to derive a conceptual framework for LHS analytics implementation.

Results  A total of 238 candidate articles were reviewed and 14 met inclusion criteria. From the selected studies, we extracted 37 implementation challenges and 64 methods employed to overcome such challenges. We identified common approaches for evaluating the implementation of clinical dashboards. Six studies assessed clinical process outcomes and only four studies evaluated patient health outcomes. A conceptual framework for implementing the analytics of an LHS was developed.

Conclusion  Health care organizations face diverse challenges when trying to implement real-time data analytics. These challenges have shifted over the past decade. While prior reviews identified fundamental information problems, such as data size and complexity, our review uncovered more postpilot challenges, such as supporting diverse users, workflows, and user-interface screens. Our review identified practical methods to overcome these challenges which have been incorporated into a conceptual framework. It is hoped this framework will support health care organizations deploying near-real-time clinical dashboards and progress toward an LHS.

Machine learning models for diabetes management in acute care using electronic medical records: A systematic review

BACKGROUND

Machine learning (ML) is a subset of Artificial Intelligence (AI) that is used to predict and potentially prevent adverse patient outcomes. There is increasing interest in the application of these models in digital hospitals to improve clinical decision-making and chronic disease management, particularly for patients with diabetes. The potential of ML models using electronic medical records (EMR) to improve the clinical care of hospitalised patients with diabetes is currently unknown.

OBJECTIVE

The aim was to systematically identify and critically review the published literature examining the development and validation of ML models using EMR data for improving the care of hospitalised adult patients with diabetes.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) guidelines were followed. Four databases were searched (Embase, PubMed, IEEE and Web of Science) for studies published between January 2010 to January 2022. The reference lists of the eligible articles were manually searched. Articles that examined adults and both developed and validated ML models using EMR data were included. Studies conducted in primary care and community care settings were excluded. Studies were independently screened and data was extracted using Covidence® systematic review software. For data extraction and critical appraisal, the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was followed. Risk of bias was assessed using the Prediction model Risk Of Bias Assessment Tool (PROBAST). Quality of reporting was assessed by adherence to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guideline. The IJMEDI checklist was followed to assess quality of ML models and the reproducibility of their outcomes. The external validation methodology of the studies was appraised.

RESULTS

Of the 1317 studies screened, twelve met inclusion criteria. Eight studies developed ML models to predict disglycaemic episodes for hospitalized patients with diabetes, one study developed a ML model to predict total insulin dosage, two studies predicted risk of readmission, and one study improved the prediction of hospital readmission for inpatients with diabetes. All included studies were heterogeneous with regard to ML types, cohort, input predictors, sample size, performance and validation metrics and clinical outcomes. Two studies adhered to the TRIPOD guideline. The methodological reporting of all the studies was evaluated to be at high risk of bias. The quality of ML models in all studies was assessed as poor. Robust external validation was not performed on any of the studies. No models were implemented or evaluated in routine clinical care.

CONCLUSIONS

This review identified a limited number of ML models which were developed to improve inpatient management of diabetes. No ML models were implemented in real hospital settings. Future research needs to enhance the development, reporting and validation steps to enable ML models for integration into routine clinical care.

Electronic Health Record-Triggered Research Infrastructure Combining Real-world Electronic Health Record Data and Patient-Reported Outcomes to Detect Benefits, Risks, and Impact of Medication: Development Study

Background

Real-world data from electronic health records (EHRs) represent a wealth of information for studying the benefits and risks of medical treatment. However, they are limited in scope and should be complemented by information from the patient perspective.

Objective

The aim of this study is to develop an innovative research infrastructure that combines information from EHRs with patient experiences reported in questionnaires to monitor the risks and benefits of medical treatment.

Methods

We focused on the treatment of overactive bladder (OAB) in general practice as a use case. To develop the Benefit, Risk, and Impact of Medication Monitor (BRIMM) infrastructure, we first performed a requirement analysis. BRIMM’s starting point is routinely recorded general practice EHR data that are sent to the Dutch Nivel Primary Care Database weekly. Patients with OAB were flagged weekly on the basis of diagnoses and prescriptions. They were invited subsequently for participation by their general practitioner (GP), via a trusted third party. Patients received a series of questionnaires on disease status, pharmacological and nonpharmacological treatments, adverse drug reactions, drug adherence, and quality of life. The questionnaires and a dedicated feedback portal were developed in collaboration with a patient association for pelvic-related diseases, Bekkenbodem4All. Participating patients and GPs received feedback. An expert meeting was organized to assess the strengths, weaknesses, opportunities, and threats of the new research infrastructure.

Results

The BRIMM infrastructure was developed and implemented. In the Nivel Primary Care Database, 2933 patients with OAB from 27 general practices were flagged. GPs selected 1636 (55.78%) patients who were eligible for the study, of whom 295 (18.0% of eligible patients) completed the first questionnaire. A total of 288 (97.6%) patients consented to the linkage of their questionnaire data with their EHR data. According to experts, the strengths of the infrastructure were the linkage of patient-reported outcomes with EHR data, comparison of pharmacological and nonpharmacological treatments, flexibility of the infrastructure, and low registration burden for GPs. Methodological weaknesses, such as susceptibility to bias, patient selection, and low participation rates among GPs and patients, were seen as weaknesses and threats. Opportunities represent usefulness for policy makers and health professionals, conditional approval of medication, data linkage to other data sources, and feedback to patients.

Conclusions

The BRIMM research infrastructure has the potential to assess the benefits and safety of (medical) treatment in real-life situations using a unique combination of EHRs and patient-reported outcomes. As patient involvement is an important aspect of the treatment process, generating knowledge from clinical and patient perspectives is valuable for health care providers, patients, and policy makers. The developed methodology can easily be applied to other treatments and health problems.

Baseline Quality Improvement Capacity of 33 Endocrinology Centers Participating in the T1D Exchange Quality Improvement Collaborative

This article describes the evolution of the Type 1 Diabetes Exchange Quality Improvement Collaborative (T1DX-QI) and provides insight into the development and growth of a successful type 1 diabetes quality improvement (QI) program. Since its inception 8 years ago, the collaborative has expanded to include centers across the United States with varying levels of QI experience, while simultaneously achieving many tangible improvements in type 1 diabetes care. These successes underscore the importance of learning health systems, data-sharing, benchmarking, and peer collaboration as drivers for continuous QI. Future efforts will include recruiting additional small- to medium-sized centers focused on adult care and underserved communities to further the goal of improving care and outcomes for all people living with type 1 diabetes.

Learning health systems: A review of key topic areas and bibliometric trends

INTRODUCTION

The emergent field of learning health systems (LHSs) has been rapidly evolving as the concept continues to be embraced by researchers, managers, and clinicians. This paper reports on a scoping review and bibliometric analysis of the LHS literature to identify key topic areas and examine the influence and spread of recent research.

METHODS

We conducted a scoping review of LHS literature published between January 2016 and May 2020. The authors extracted publication data (eg, journal, country, authors, citation count, keywords) and reviewed full-texts to identify: type of study (empirical, non-empirical, or review), degree of focus (general or specific), and the reference used when defining LHSs.

RESULTS

A total of 272 publications were included in this review. Almost two thirds (65.1%) of the included articles were non-empirical and over two-thirds (68.4%) were from authors in the United States. More than half of the publications focused on specific areas, for example: oncology, cardiovascular care, and genomic medicine. Other key topic areas included: ethics, research, quality improvement, and electronic health records. We identified that definitions of the LHS concept are converging; however, many papers focused on data platforms and analytical processes rather than organisational and behavioural factors to support change and learning activities.

CONCLUSIONS

The literature on LHSs remains largely theoretical with definitions of LHSs focusing on technical processes to reuse data collected during the clinical process and embedding analysed data back into the system. A shift in the literature to empirical LHS studies with consideration of organisational and human factors is warranted.

Leveraging National Claims and Hospital Big Data: Cohort Study on a Statin-Drug Interaction Use Case

Background

Linking different sources of medical data is a promising approach to analyze care trajectories. The aim of the INSHARE (Integrating and Sharing Health Big Data for Research) project was to provide the blueprint for a technological platform that facilitates integration, sharing, and reuse of data from 2 sources: the clinical data warehouse (CDW) of the Rennes academic hospital, called eHOP (entrepôt Hôpital), and a data set extracted from the French national claim data warehouse (Système National des Données de Santé [SNDS]).

Objective

This study aims to demonstrate how the INSHARE platform can support big data analytic tasks in the health field using a pharmacovigilance use case based on statin consumption and statin-drug interactions.

Methods

A Spark distributed cluster-computing framework was used for the record linkage procedure and all analyses. A semideterministic record linkage method based on the common variables between the chosen data sources was developed to identify all patients discharged after at least one hospital stay at the Rennes academic hospital between 2015 and 2017. The use-case study focused on a cohort of patients treated with statins prescribed by their general practitioner or during their hospital stay.

Results

The whole process (record linkage procedure and use-case analyses) required 88 minutes. Of the 161,532 and 164,316 patients from the SNDS and eHOP CDW data sets, respectively, 159,495 patients were successfully linked (98.74% and 97.07% of patients from SNDS and eHOP CDW, respectively). Of the 16,806 patients with at least one statin delivery, 8293 patients started the consumption before and continued during the hospital stay, 6382 patients stopped statin consumption at hospital admission, and 2131 patients initiated statins in hospital. Statin-drug interactions occurred more frequently during hospitalization than in the community (3800/10,424, 36.45% and 3253/14,675, 22.17%, respectively; P<.001). Only 121 patients had the most severe level of statin-drug interaction. Hospital stay burden (length of stay and in-hospital mortality) was more severe in patients with statin-drug interactions during hospitalization.

Conclusions

This study demonstrates the added value of combining and reusing clinical and claim data to provide large-scale measures of drug-drug interaction prevalence and care pathways outside hospitals. It builds a path to move the current health care system toward a Learning Health System using knowledge generated from research on real-world health data.

A scoping review of whole-of-community interventions on six modifiable cancer prevention risk factors in youth: A systems typology

Whole-of-community interventions delivered across entire geospatial areas show promise for improving population health for youth cancer prevention. The aims of this scoping review were to synthesize the whole-of-community intervention literature on six modifiable risk factors in youth for cancer prevention (alcohol use, diet, obesity, physical activity, sun exposure, tobacco use) and to develop and apply a typology describing the inclusion of fundamental control system functional characteristics. A systematic search was conducted in PubMed, Cumulative Index to Nursing and Allied Health Literature, Embase, PsycINFO, and Scopus for studies published to the end of 2019. Eligible studies included a geospatially defined whole-of-community intervention; youth 0-18 years; and at least one of the six cancer risk factor outcomes. An iterative process was undertaken to create a typology describing the functions for whole-of-community interventions guided by systems theory, and the typology was used to code the included interventions. A total of 41 interventions were included. Most interventions (43.9%) assessed multiple cancer risk factors. Few interventions provided fundamental functions necessary for community system coordination: sensor, controller, effector. Although communities are a patchwork quilt of microsystems where individuals interact in geographically bounded places nested within larger whole systems of influence, a control systems approach has not been used to frame the literature. Whole-of-community interventions can be characterized by the fundamental system functions necessary for coordinating population health improvement. Future whole-of-community intervention efforts should draw on fundamental knowledge of how systems operate and test whether adoption of the key functions is necessary for whole-of-community population health improvement.

The role of co-production in Learning Health Systems

Background

Co-production of health is defined as ‘the interdependent work of users and professionals who are creating, designing, producing, delivering, assessing, and evaluating the relationships and actions that contribute to the health of individuals and populations’. It can assume many forms and include multiple stakeholders in pursuit of continuous improvement, as in Learning Health Systems (LHSs). There is increasing interest in how the LHS concept allows integration of different knowledge domains to support and achieve better health. Even if definitions of LHSs include engaging users and their family as active participants in aspects of enabling better health for individuals and populations, LHS descriptions emphasize technological solutions, such as the use of information systems. Fewer LHS texts address how interpersonal interactions contribute to the design and improvement of healthcare services.

Objective

We examined the literature on LHS to clarify the role and contributions of co-production in LHS conceptualizations and applications.

Method

First, we undertook a scoping review of LHS conceptualizations. Second, we compared those conceptualizations to the characteristics of LHSs first described by the US Institute of Medicine. Third, we examined the LHS conceptualizations to assess how they bring four types of value co-creation in public services into play: co-production, co-design, co-construction and co-innovation. These were used to describe core ideas, as principles, to guide development.

Result

Among 17 identified LHS conceptualizations, 3 qualified as most comprehensive regarding fidelity to LHS characteristics and their use in multiple settings: (i) the Cincinnati Collaborative LHS Model, (ii) the Dartmouth Coproduction LHS Model and (iii) the Michigan Learning Cycle Model. These conceptualizations exhibit all four types of value co-creation, provide examples of how LHSs can harness co-production and are used to identify principles that can enhance value co-creation: (i) use a shared aim, (ii) navigate towards improved outcomes, (iii) tailor feedback with and for users, (iv) distribute leadership, (v) facilitate interactions, (vi) co-design services and (vii) support self-organization.

Conclusions

The LHS conceptualizations have common features and harness co-production to generate value for individual patients as well as for health systems. They facilitate learning and improvement by integrating supportive technologies into the sociotechnical systems that make up healthcare. Further research on LHS applications in real-world complex settings is needed to unpack how LHSs are grown through coproduction and other types of value co-creation.