1
|
Keim-Malpass J, Moorman LP, Moorman JR, Hamil S, Yousevfand G, Monfredi OJ, Ratcliffe SJ, Krahn KN, Jones MK, Clark MT, Bourque JM. Prospective validation of clinical deterioration predictive models prior to intensive care unit transfer among patients admitted to acute care cardiology wards. Physiol Meas 2024; 45:065004. [PMID: 38772399 DOI: 10.1088/1361-6579/ad4e90] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Accepted: 05/21/2024] [Indexed: 05/23/2024]
Abstract
Objective. Very few predictive models have been externally validated in a prospective cohort following the implementation of an artificial intelligence analytic system. This type of real-world validation is critically important due to the risk of data drift, or changes in data definitions or clinical practices over time, that could impact model performance in contemporaneous real-world cohorts. In this work, we report the model performance of a predictive analytics tool developed before COVID-19 and demonstrate model performance during the COVID-19 pandemic.Approach. The analytic system (CoMETⓇ, Nihon Kohden Digital Health Solutions LLC, Irvine, CA) was implemented in a randomized controlled trial that enrolled 10 422 patient visits in a 1:1 display-on display-off design. The CoMET scores were calculated for all patients but only displayed in the display-on arm. Only the control/display-off group is reported here because the scores could not alter care patterns.Main results.Of the 5184 visits in the display-off arm, 311 experienced clinical deterioration and care escalation, resulting in transfer to the intensive care unit, primarily due to respiratory distress. The model performance of CoMET was assessed based on areas under the receiver operating characteristic curve, which ranged from 0.725 to 0.737.Significance.The models were well-calibrated, and there were dynamic increases in the model scores in the hours preceding the clinical deterioration events. A hypothetical alerting strategy based on a rise in score and duration of the rise would have had good performance, with a positive predictive value more than 10-fold the event rate. We conclude that predictive statistical models developed five years before study initiation had good model performance despite the passage of time and the impact of the COVID-19 pandemic.
Collapse
Affiliation(s)
- Jessica Keim-Malpass
- Center for Advanced Medical Analytics, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
- Department of Pediatrics, Hematology-Oncology Division, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
| | - Liza P Moorman
- Nihon Kohden Digital Health Solutions, Irvine, CA, United States of America
| | - J Randall Moorman
- Center for Advanced Medical Analytics, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
- Department of Internal Medicine, Cardiovascular Division, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
| | - Susan Hamil
- Center for Advanced Medical Analytics, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
- Department of Internal Medicine, Cardiovascular Division, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
| | - Gholamreza Yousevfand
- Center for Advanced Medical Analytics, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
- Department of Internal Medicine, Cardiovascular Division, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
| | - Oliver J Monfredi
- Center for Advanced Medical Analytics, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
- Department of Internal Medicine, Cardiovascular Division, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
| | - Sarah J Ratcliffe
- Department of Public Health Sciences, Biostatistics Division, University of Virginia, Charlottesville, VA, United States of America
| | - Katy N Krahn
- Center for Advanced Medical Analytics, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
- Department of Internal Medicine, Cardiovascular Division, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
| | - Marieke K Jones
- Department of Public Health Sciences, Biostatistics Division, University of Virginia, Charlottesville, VA, United States of America
| | - Matthew T Clark
- Nihon Kohden Digital Health Solutions, Irvine, CA, United States of America
| | - Jamieson M Bourque
- Center for Advanced Medical Analytics, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
- Department of Internal Medicine, Cardiovascular Division, School of Medicine, University of Virginia, Charlottesville, VA, United States of America
| |
Collapse
|
2
|
Hobensack M, Withall J, Douthit B, Cato K, Dykes P, Cho S, Lowenthal G, Ivory C, Yen PY, Rossetti S. Identifying Barriers to The Implementation of Communicating Narrative Concerns Entered by Registered Nurses, An Early Warning System SmartApp. Appl Clin Inform 2024; 15:295-305. [PMID: 38631380 PMCID: PMC11023711 DOI: 10.1055/s-0044-1785688] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Accepted: 02/06/2024] [Indexed: 04/19/2024] Open
Abstract
BACKGROUND Nurses are at the frontline of detecting patient deterioration. We developed Communicating Narrative Concerns Entered by Registered Nurses (CONCERN), an early warning system for clinical deterioration that generates a risk prediction score utilizing nursing data. CONCERN was implemented as a randomized clinical trial at two health systems in the Northeastern United States. Following the implementation of CONCERN, our team sought to develop the CONCERN Implementation Toolkit to enable other hospital systems to adopt CONCERN. OBJECTIVE The aim of this study was to identify the optimal resources needed to implement CONCERN and package these resources into the CONCERN Implementation Toolkit to enable the spread of CONCERN to other hospital sites. METHODS To accomplish this aim, we conducted qualitative interviews with nurses, prescribing providers, and information technology experts in two health systems. We recruited participants from July 2022 to January 2023. We conducted thematic analysis guided by the Donabedian model. Based on the results of the thematic analysis, we updated the α version of the CONCERN Implementation Toolkit. RESULTS There was a total of 32 participants included in our study. In total, 12 themes were identified, with four themes mapping to each domain in Donabedian's model (i.e., structure, process, and outcome). Eight new resources were added to the CONCERN Implementation Toolkit. CONCLUSIONS This study validated the α version of the CONCERN Implementation Toolkit. Future studies will focus on returning the results of the Toolkit to the hospital sites to validate the β version of the CONCERN Implementation Toolkit. As the development of early warning systems continues to increase and clinician workflows evolve, the results of this study will provide considerations for research teams interested in implementing early warning systems in the acute care setting.
Collapse
Affiliation(s)
- Mollie Hobensack
- Brookdale Department of Geriatrics and Palliative Medicine, Icahn School of Medicine at Mount Sinai, New York City, New York, United States
| | - Jennifer Withall
- Department of Biomedical Informatics, Columbia University, New York City, New York, United States
| | - Brian Douthit
- Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, United States
| | - Kenrick Cato
- Department of Informatics, University of Pennsylvania, Philadelphia, Pennsylvania, United States
| | - Patricia Dykes
- Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
| | - Sandy Cho
- Department of Clinical Informatics, Newton-Wellesley Hospital, Newton, Massachusetts, United States
| | - Graham Lowenthal
- Brigham and Women's Hospital, Boston, Massachusetts, United States
| | - Catherine Ivory
- Vanderbilt University Medical Center, Nashville, Tennessee, United States
| | - Po-Yin Yen
- Washington University School of Medicine in St. Louis, St. Louis, MO, United States
| | - Sarah Rossetti
- Department of Biomedical Informatics, Columbia University, New York City, New York, United States
| |
Collapse
|
3
|
Kumar D, Suthar N. Predictive analytics and early intervention in healthcare social work: a scoping review. SOCIAL WORK IN HEALTH CARE 2024; 63:208-229. [PMID: 38349783 DOI: 10.1080/00981389.2024.2316700] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 01/05/2024] [Indexed: 02/15/2024]
Abstract
This scoping review investigates the untapped potential of predictive analytics in healthcare social work, specifically targeting early intervention frameworks. Despite the escalating attention predictive analytics has garnered across multiple disciplines, its tailored application in social work remains notably sparse. This study endeavors to fill this lacuna by meticulously reviewing the extant literature and delineating the prospective advantages and inherent constraints of integrating predictive analytics into healthcare social work. The outcomes of this inquiry enrich the prevailing dialogue on the utility of predictive analytics in healthcare, offering indispensable perspectives for practitioners and policymakers in the social work domain.
Collapse
Affiliation(s)
- Dinesh Kumar
- Faculty of Business and Applied Arts, Lovely Professional University, Mittal School of Business, Phagwara, India
| | - Nidhi Suthar
- Administration, Pomento IT Services, Hisar, India
| |
Collapse
|
4
|
Wieben AM, Walden RL, Alreshidi BG, Brown SF, Cato K, Coviak CP, Cruz C, D'Agostino F, Douthit BJ, Forbes TH, Gao G, Johnson SG, Lee MA, Mullen-Fortino M, Park JI, Park S, Pruinelli L, Reger A, Role J, Sileo M, Schultz MA, Vyas P, Jeffery AD. Data Science Implementation Trends in Nursing Practice: A Review of the 2021 Literature. Appl Clin Inform 2023; 14:585-593. [PMID: 37150179 PMCID: PMC10411069 DOI: 10.1055/a-2088-2893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Accepted: 05/03/2023] [Indexed: 05/09/2023] Open
Abstract
OBJECTIVES The goal of this work was to provide a review of the implementation of data science-driven applications focused on structural or outcome-related nurse-sensitive indicators in the literature in 2021. By conducting this review, we aim to inform readers of trends in the nursing indicators being addressed, the patient populations and settings of focus, and lessons and challenges identified during the implementation of these tools. METHODS We conducted a rigorous descriptive review of the literature to identify relevant research published in 2021. We extracted data on model development, implementation-related strategies and measures, lessons learned, and challenges and stakeholder involvement. We also assessed whether reports of data science application implementations currently follow the guidelines of the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by AI (DECIDE-AI) framework. RESULTS Of 4,943 articles found in PubMed (NLM) and CINAHL (EBSCOhost), 11 were included in the final review and data extraction. Systems leveraging data science were developed for adult patient populations and were primarily deployed in hospital settings. The clinical domains targeted included mortality/deterioration, utilization/resource allocation, and hospital-acquired infections/COVID-19. The composition of development teams and types of stakeholders involved varied. Research teams more frequently reported on implementation methods than implementation results. Most studies provided lessons learned that could help inform future implementations of data science systems in health care. CONCLUSION In 2021, very few studies report on the implementation of data science-driven applications focused on structural- or outcome-related nurse-sensitive indicators. This gap in the sharing of implementation strategies needs to be addressed in order for these systems to be successfully adopted in health care settings.
Collapse
Affiliation(s)
- Ann M. Wieben
- University of Wisconsin-Madison School of Nursing, Madison, Wisconsin, United States
| | - Rachel Lane Walden
- Annette and Irwin Eskind Family Biomedical Library, Vanderbilt University, Nashville, Tennessee, United States
| | - Bader G. Alreshidi
- Medical-Surgical Nursing Department, College of Nursing, University of Hail, Hail, Saudi Arabia
| | | | - Kenrick Cato
- Department of Emergency Medicine, Columbia University School of Nursing, New York, New York, United States
| | - Cynthia Peltier Coviak
- Kirkhof College of Nursing, Grand Valley State University, Allendale, Michigan, United States
| | - Christopher Cruz
- Global Health Technology and Informatics, Chevron, San Ramon, California, United States
| | - Fabio D'Agostino
- Department of Medicine and Surgery, Saint Camillus International University of Health Sciences, Rome, Italy
| | - Brian J. Douthit
- Department of Biomedical Informatics, United States Department of Veterans Affairs, Vanderbilt University, Nashville, Tennessee, United States
| | - Thompson H. Forbes
- Department of Advanced Nursing Practice and Education, East Carolina University College of Nursing, Greenville, North Carolina, United States
| | - Grace Gao
- Atlanta VA Quality Scholars Program, Joseph Maxwell Cleland, Atlanta VA Medical Center, North Druid Hills, Georgia, United States
| | - Steve G. Johnson
- Institute for Health Informatics, University of Minnesota, Minneapolis, Minnesota, United States
| | | | | | - Jung In Park
- Sue and Bill Gross School of Nursing, University of California, Irvine, United States
| | - Suhyun Park
- College of Nursing and College of Medicine, University of Florida, Gainesville, Florida, United States
| | - Lisiane Pruinelli
- College of Nursing and College of Medicine, University of Florida, Gainesville, Florida, United States
| | | | - Jethrone Role
- Loma Linda University Health, Loma Linda, California, United States
| | - Marisa Sileo
- Boston Children's Hospital, Boston, Massachusetts, United States
| | | | - Pankaj Vyas
- University of Arizona College of Nursing, Tucson, Arizona, United States
| | - Alvin D. Jeffery
- U.S. Department of Veterans Affairs, Vanderbilt University School of Nursing, Tennessee Valley Healthcare System, Nashville, Tennessee, United States
| |
Collapse
|
5
|
Walker SB, Badke CM, Carroll MS, Honegger KS, Fawcett A, Weese-Mayer DE, Sanchez-Pinto LN. Novel approaches to capturing and using continuous cardiorespiratory physiological data in hospitalized children. Pediatr Res 2023; 93:396-404. [PMID: 36329224 DOI: 10.1038/s41390-022-02359-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/02/2022] [Revised: 08/16/2022] [Accepted: 10/11/2022] [Indexed: 11/06/2022]
Abstract
Continuous cardiorespiratory physiological monitoring is a cornerstone of care in hospitalized children. The data generated by monitoring devices coupled with machine learning could transform the way we provide care. This scoping review summarizes existing evidence on novel approaches to continuous cardiorespiratory monitoring in hospitalized children. We aimed to identify opportunities for the development of monitoring technology and the use of machine learning to analyze continuous physiological data to improve the outcomes of hospitalized children. We included original research articles published on or after January 1, 2001, involving novel approaches to collect and use continuous cardiorespiratory physiological data in hospitalized children. OVID Medline, PubMed, and Embase databases were searched. We screened 2909 articles and performed full-text extraction of 105 articles. We identified 58 articles describing novel devices or approaches, which were generally small and single-center. In addition, we identified 47 articles that described the use of continuous physiological data in prediction models, but only 7 integrated multidimensional data (e.g., demographics, laboratory results). We identified three areas for development: (1) further validation of promising novel devices; (2) more studies of models integrating multidimensional data with continuous cardiorespiratory data; and (3) further dissemination, implementation, and validation of prediction models using continuous cardiorespiratory data. IMPACT: We performed a comprehensive scoping review of novel approaches to capture and use continuous cardiorespiratory physiological data for monitoring, diagnosis, providing care, and predicting events in hospitalized infants and children, from novel devices to machine learning-based prediction models. We identified three key areas for future development: (1) further validation of promising novel devices; (2) more studies of models integrating multidimensional data with continuous cardiorespiratory data; and (3) further dissemination, implementation, and validation of prediction models using cardiorespiratory data.
Collapse
Affiliation(s)
- Sarah B Walker
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA. .,Stanley Manne Children's Research Institute, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA.
| | - Colleen M Badke
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.,Stanley Manne Children's Research Institute, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Michael S Carroll
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.,Stanley Manne Children's Research Institute, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Kyle S Honegger
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.,Stanley Manne Children's Research Institute, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Andrea Fawcett
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.,Stanley Manne Children's Research Institute, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Debra E Weese-Mayer
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.,Stanley Manne Children's Research Institute, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - L Nelson Sanchez-Pinto
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.,Stanley Manne Children's Research Institute, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| |
Collapse
|
6
|
Petersson L, Larsson I, Nygren JM, Nilsen P, Neher M, Reed JE, Tyskbo D, Svedberg P. Challenges to implementing artificial intelligence in healthcare: a qualitative interview study with healthcare leaders in Sweden. BMC Health Serv Res 2022; 22:850. [PMID: 35778736 PMCID: PMC9250210 DOI: 10.1186/s12913-022-08215-8] [Citation(s) in RCA: 32] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2022] [Accepted: 06/20/2022] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Artificial intelligence (AI) for healthcare presents potential solutions to some of the challenges faced by health systems around the world. However, it is well established in implementation and innovation research that novel technologies are often resisted by healthcare leaders, which contributes to their slow and variable uptake. Although research on various stakeholders' perspectives on AI implementation has been undertaken, very few studies have investigated leaders' perspectives on the issue of AI implementation in healthcare. It is essential to understand the perspectives of healthcare leaders, because they have a key role in the implementation process of new technologies in healthcare. The aim of this study was to explore challenges perceived by leaders in a regional Swedish healthcare setting concerning the implementation of AI in healthcare. METHODS The study takes an explorative qualitative approach. Individual, semi-structured interviews were conducted from October 2020 to May 2021 with 26 healthcare leaders. The analysis was performed using qualitative content analysis, with an inductive approach. RESULTS The analysis yielded three categories, representing three types of challenge perceived to be linked with the implementation of AI in healthcare: 1) Conditions external to the healthcare system; 2) Capacity for strategic change management; 3) Transformation of healthcare professions and healthcare practice. CONCLUSIONS In conclusion, healthcare leaders highlighted several implementation challenges in relation to AI within and beyond the healthcare system in general and their organisations in particular. The challenges comprised conditions external to the healthcare system, internal capacity for strategic change management, along with transformation of healthcare professions and healthcare practice. The results point to the need to develop implementation strategies across healthcare organisations to address challenges to AI-specific capacity building. Laws and policies are needed to regulate the design and execution of effective AI implementation strategies. There is a need to invest time and resources in implementation processes, with collaboration across healthcare, county councils, and industry partnerships.
Collapse
Affiliation(s)
- Lena Petersson
- School of Health and Welfare, Halmstad University, Box 823, 301 18, Halmstad, Sweden.
| | - Ingrid Larsson
- School of Health and Welfare, Halmstad University, Box 823, 301 18, Halmstad, Sweden
| | - Jens M Nygren
- School of Health and Welfare, Halmstad University, Box 823, 301 18, Halmstad, Sweden
| | - Per Nilsen
- School of Health and Welfare, Halmstad University, Box 823, 301 18, Halmstad, Sweden.,Department of Health, Medicine and Caring Sciences, Division of Public Health, Faculty of Health Sciences, Linköping University, Linköping, Sweden
| | - Margit Neher
- School of Health and Welfare, Halmstad University, Box 823, 301 18, Halmstad, Sweden.,Department of Rehabilitation, School of Health Sciences, Jönköping University, Jönköping, Sweden
| | - Julie E Reed
- School of Health and Welfare, Halmstad University, Box 823, 301 18, Halmstad, Sweden
| | - Daniel Tyskbo
- School of Health and Welfare, Halmstad University, Box 823, 301 18, Halmstad, Sweden
| | - Petra Svedberg
- School of Health and Welfare, Halmstad University, Box 823, 301 18, Halmstad, Sweden
| |
Collapse
|
7
|
Sharma M, Savage C, Nair M, Larsson I, Svedberg P, Nygren JM. Artificial Intelligence Applications in Health Care Practice: A Scoping Review (Preprint). J Med Internet Res 2022; 24:e40238. [PMID: 36197712 PMCID: PMC9582911 DOI: 10.2196/40238] [Citation(s) in RCA: 30] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2022] [Revised: 08/19/2022] [Accepted: 08/30/2022] [Indexed: 11/25/2022] Open
Abstract
Background Artificial intelligence (AI) is often heralded as a potential disruptor that will transform the practice of medicine. The amount of data collected and available in health care, coupled with advances in computational power, has contributed to advances in AI and an exponential growth of publications. However, the development of AI applications does not guarantee their adoption into routine practice. There is a risk that despite the resources invested, benefits for patients, staff, and society will not be realized if AI implementation is not better understood. Objective The aim of this study was to explore how the implementation of AI in health care practice has been described and researched in the literature by answering 3 questions: What are the characteristics of research on implementation of AI in practice? What types and applications of AI systems are described? What characteristics of the implementation process for AI systems are discernible? Methods A scoping review was conducted of MEDLINE (PubMed), Scopus, Web of Science, CINAHL, and PsycINFO databases to identify empirical studies of AI implementation in health care since 2011, in addition to snowball sampling of selected reference lists. Using Rayyan software, we screened titles and abstracts and selected full-text articles. Data from the included articles were charted and summarized. Results Of the 9218 records retrieved, 45 (0.49%) articles were included. The articles cover diverse clinical settings and disciplines; most (32/45, 71%) were published recently, were from high-income countries (33/45, 73%), and were intended for care providers (25/45, 56%). AI systems are predominantly intended for clinical care, particularly clinical care pertaining to patient-provider encounters. More than half (24/45, 53%) possess no action autonomy but rather support human decision-making. The focus of most research was on establishing the effectiveness of interventions (16/45, 35%) or related to technical and computational aspects of AI systems (11/45, 24%). Focus on the specifics of implementation processes does not yet seem to be a priority in research, and the use of frameworks to guide implementation is rare. Conclusions Our current empirical knowledge derives from implementations of AI systems with low action autonomy and approaches common to implementations of other types of information systems. To develop a specific and empirically based implementation framework, further research is needed on the more disruptive types of AI systems being implemented in routine care and on aspects unique to AI implementation in health care, such as building trust, addressing transparency issues, developing explainable and interpretable solutions, and addressing ethical concerns around privacy and data protection.
Collapse
Affiliation(s)
- Malvika Sharma
- Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Medical Management Centre, Stockholm, Sweden
| | - Carl Savage
- Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Medical Management Centre, Stockholm, Sweden
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Monika Nair
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Ingrid Larsson
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Petra Svedberg
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Jens M Nygren
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| |
Collapse
|
8
|
Lim HC, Austin JA, van der Vegt AH, Rahimi AK, Canfell OJ, Mifsud J, Pole JD, Barras MA, Hodgson T, Shrapnel S, Sullivan CM. Toward a Learning Health Care System: A Systematic Review and Evidence-Based Conceptual Framework for Implementation of Clinical Analytics in a Digital Hospital. Appl Clin Inform 2022; 13:339-354. [PMID: 35388447 PMCID: PMC8986462 DOI: 10.1055/s-0042-1743243] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022] Open
Abstract
Objective
A learning health care system (LHS) uses routinely collected data to continuously monitor and improve health care outcomes. Little is reported on the challenges and methods used to implement the analytics underpinning an LHS. Our aim was to systematically review the literature for reports of real-time clinical analytics implementation in digital hospitals and to use these findings to synthesize a conceptual framework for LHS implementation.
Methods
Embase, PubMed, and Web of Science databases were searched for clinical analytics derived from electronic health records in adult inpatient and emergency department settings between 2015 and 2021. Evidence was coded from the final study selection that related to (1) dashboard implementation challenges, (2) methods to overcome implementation challenges, and (3) dashboard assessment and impact. The evidences obtained, together with evidence extracted from relevant prior reviews, were mapped to an existing digital health transformation model to derive a conceptual framework for LHS analytics implementation.
Results
A total of 238 candidate articles were reviewed and 14 met inclusion criteria. From the selected studies, we extracted 37 implementation challenges and 64 methods employed to overcome such challenges. We identified common approaches for evaluating the implementation of clinical dashboards. Six studies assessed clinical process outcomes and only four studies evaluated patient health outcomes. A conceptual framework for implementing the analytics of an LHS was developed.
Conclusion
Health care organizations face diverse challenges when trying to implement real-time data analytics. These challenges have shifted over the past decade. While prior reviews identified fundamental information problems, such as data size and complexity, our review uncovered more postpilot challenges, such as supporting diverse users, workflows, and user-interface screens. Our review identified practical methods to overcome these challenges which have been incorporated into a conceptual framework. It is hoped this framework will support health care organizations deploying near-real-time clinical dashboards and progress toward an LHS.
Collapse
Affiliation(s)
- Han Chang Lim
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia.,Department of Health, eHealth Queensland, Queensland Government, Brisbane, Australia
| | - Jodie A Austin
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia.,Department of Health, eHealth Queensland, Queensland Government, Brisbane, Australia
| | - Anton H van der Vegt
- Information Engineering Lab, School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Brisbane, Australia
| | - Amir Kamel Rahimi
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia.,Digital Health Cooperative Research Centre, Australian Government, Sydney, New South Wales, Australia
| | - Oliver J Canfell
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia.,Digital Health Cooperative Research Centre, Australian Government, Sydney, New South Wales, Australia.,UQ Business School, Faculty of Business, Economics and Law, The University of Queensland, St. Lucia, Brisbane, Australia
| | - Jayden Mifsud
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
| | - Jason D Pole
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
| | - Michael A Barras
- School of Pharmacy, Faculty of Health and Behavioural Sciences, The University of Queensland, PACE Precinct, Woolloongabba, Brisbane, Australia.,Pharmacy Department, Princess Alexandra Hospital, Woolloongabba, Brisbane, Australia
| | - Tobias Hodgson
- UQ Business School, Faculty of Business, Economics and Law, The University of Queensland, St. Lucia, Brisbane, Australia
| | - Sally Shrapnel
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia.,School of Mathematics and Physics, Faculty of Science, The University of Queensland, St Lucia, Brisbane, Australia
| | - Clair M Sullivan
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia.,Department of Health, Metro North Hospital and Health Service, Queensland Government, Herston QLD, Australia
| |
Collapse
|
9
|
The principles of whole-hospital predictive analytics monitoring for clinical medicine originated in the neonatal ICU. NPJ Digit Med 2022; 5:41. [PMID: 35361861 PMCID: PMC8971442 DOI: 10.1038/s41746-022-00584-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Accepted: 02/23/2022] [Indexed: 11/17/2022] Open
Abstract
In 2011, a multicenter group spearheaded at the University of Virginia demonstrated reduced mortality from real-time continuous cardiorespiratory monitoring in the neonatal ICU using what we now call Artificial Intelligence, Big Data, and Machine Learning. The large, randomized heart rate characteristics trial made real, for the first time that we know of, the promise that early detection of illness would allow earlier and more effective intervention and improved patient outcomes. Currently, though, we hear as much of failures as we do of successes in the rapidly growing field of predictive analytics monitoring that has followed. This Perspective aims to describe the principles of how we developed heart rate characteristics monitoring for neonatal sepsis and then applied them throughout adult ICU and hospital medicine. It primarily reflects the work since the 1990s of the University of Virginia group: the theme is that sudden and catastrophic deteriorations can be preceded by subclinical but measurable physiological changes apparent in the continuous cardiorespiratory monitoring and electronic health record.
Collapse
|
10
|
Callcut RA, Xu Y, Moorman JR, Tsai C, Villaroman A, Robles AJ, Lake DE, Hu X, Clark MT. External validation of a novel signature of illness in continuous cardiorespiratory monitoring to detect early respiratory deterioration of ICU patients. Physiol Meas 2021; 42. [PMID: 34580242 PMCID: PMC9548299 DOI: 10.1088/1361-6579/ac2264] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2021] [Accepted: 08/31/2021] [Indexed: 12/23/2022]
Abstract
Objective: The goal of predictive analytics monitoring is the early detection of patients at high risk of subacute potentially catastrophic illnesses. An excellent example of a targeted illness is respiratory failure leading to urgent unplanned intubation, where early detection might lead to interventions that improve patient outcomes. Previously, we identified signatures of this illness in the continuous cardiorespiratory monitoring data of intensive care unit (ICU) patients and devised algorithms to identify patients at rising risk. Here, we externally validated three logistic regression models to estimate the risk of emergency intubation developed in Medical and Surgical ICUs at the University of Virginia. Approach: We calculated the model outputs for more than 8000 patients in the University of California—San Francisco ICUs, 240 of whom underwent emergency intubation as determined by individual chart review. Main results: We found that the AUC of the models exceeded 0.75 in this external population, and that the risk rose appreciably over the 12 h before the event. Significance: We conclude that there are generalizable physiological signatures of impending respiratory failure in the continuous cardiorespiratory monitoring data.
Collapse
Affiliation(s)
- Rachael A Callcut
- University of California, Davis, Department of Surgery, Davis, CA, United States of America
| | - Yuan Xu
- University of California, San Francisco, Department of Surgery, San Francisco, CA, United States of America
| | - J Randall Moorman
- University of Virginia, UVa Center for Advanced Medical Analytics, Charlottesville, VA, United States of America.,University of Virginia, Cardiovascular Division, Charlottesville, VA, United States of America
| | - Christina Tsai
- University of California, San Francisco, Department of Surgery, San Francisco, CA, United States of America
| | - Andrea Villaroman
- University of California, San Francisco, Department of Surgery, San Francisco, CA, United States of America
| | - Anamaria J Robles
- University of California, San Francisco, Department of Surgery, San Francisco, CA, United States of America
| | - Douglas E Lake
- University of Virginia, UVa Center for Advanced Medical Analytics, Charlottesville, VA, United States of America.,University of Virginia, Cardiovascular Division, Charlottesville, VA, United States of America
| | - Xiao Hu
- Duke University, School of Nursing, United States of America
| | - Matthew T Clark
- University of Virginia, UVa Center for Advanced Medical Analytics, Charlottesville, VA, United States of America.,Advanced Medical Predictive Devices, Diagnostics, and Displays, Charlottesville, VA, United States of America
| |
Collapse
|