1
|
Hu Y, Lui A, Goldstein M, Sudarshan M, Tinsay A, Tsui C, Maidman SD, Medamana J, Jethani N, Puli A, Nguy V, Aphinyanaphongs Y, Kiefer N, Smilowitz NR, Horowitz J, Ahuja T, Fishman GI, Hochman J, Katz S, Bernard S, Ranganath R. Development and external validation of a dynamic risk score for early prediction of cardiogenic shock in cardiac intensive care units using machine learning. EUROPEAN HEART JOURNAL. ACUTE CARDIOVASCULAR CARE 2024; 13:472-480. [PMID: 38518758 PMCID: PMC11214586 DOI: 10.1093/ehjacc/zuae037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 03/11/2024] [Accepted: 03/19/2024] [Indexed: 03/24/2024]
Abstract
AIMS Myocardial infarction and heart failure are major cardiovascular diseases that affect millions of people in the USA with morbidity and mortality being highest among patients who develop cardiogenic shock. Early recognition of cardiogenic shock allows prompt implementation of treatment measures. Our objective is to develop a new dynamic risk score, called CShock, to improve early detection of cardiogenic shock in the cardiac intensive care unit (ICU). METHODS AND RESULTS We developed and externally validated a deep learning-based risk stratification tool, called CShock, for patients admitted into the cardiac ICU with acute decompensated heart failure and/or myocardial infarction to predict the onset of cardiogenic shock. We prepared a cardiac ICU dataset using the Medical Information Mart for Intensive Care-III database by annotating with physician-adjudicated outcomes. This dataset which consisted of 1500 patients with 204 having cardiogenic/mixed shock was then used to train CShock. The features used to train the model for CShock included patient demographics, cardiac ICU admission diagnoses, routinely measured laboratory values and vital signs, and relevant features manually extracted from echocardiogram and left heart catheterization reports. We externally validated the risk model on the New York University (NYU) Langone Health cardiac ICU database which was also annotated with physician-adjudicated outcomes. The external validation cohort consisted of 131 patients with 25 patients experiencing cardiogenic/mixed shock. CShock achieved an area under the receiver operator characteristic curve (AUROC) of 0.821 (95% CI 0.792-0.850). CShock was externally validated in the more contemporary NYU cohort and achieved an AUROC of 0.800 (95% CI 0.717-0.884), demonstrating its generalizability in other cardiac ICUs. Having an elevated heart rate is most predictive of cardiogenic shock development based on Shapley values. The other top 10 predictors are having an admission diagnosis of myocardial infarction with ST-segment elevation, having an admission diagnosis of acute decompensated heart failure, Braden Scale, Glasgow Coma Scale, blood urea nitrogen, systolic blood pressure, serum chloride, serum sodium, and arterial blood pH. CONCLUSION The novel CShock score has the potential to provide automated detection and early warning for cardiogenic shock and improve the outcomes for millions of patients who suffer from myocardial infarction and heart failure.
Collapse
Affiliation(s)
- Yuxuan Hu
- Leon. H. Charney Division of Cardiology, NYU Langone Health, 550 1st Avenue, New York, NY 10016, USA
| | - Albert Lui
- NYU Grossman School of Medicine, New York, USA
| | - Mark Goldstein
- Courant Institute of Mathematics, New York University, New York, USA
| | - Mukund Sudarshan
- Courant Institute of Mathematics, New York University, New York, USA
| | - Andrea Tinsay
- Department of Medicine, NYU Langone Health, New York, USA
| | - Cindy Tsui
- Department of Medicine, NYU Langone Health, New York, USA
| | | | - John Medamana
- Department of Medicine, NYU Langone Health, New York, USA
| | - Neil Jethani
- NYU Grossman School of Medicine, New York, USA
- Courant Institute of Mathematics, New York University, New York, USA
| | - Aahlad Puli
- Courant Institute of Mathematics, New York University, New York, USA
| | - Vuthy Nguy
- Department of Population Health, NYU Langone Health, New York, USA
| | | | - Nicholas Kiefer
- Leon. H. Charney Division of Cardiology, NYU Langone Health, 550 1st Avenue, New York, NY 10016, USA
| | - Nathaniel R Smilowitz
- Leon. H. Charney Division of Cardiology, NYU Langone Health, 550 1st Avenue, New York, NY 10016, USA
| | - James Horowitz
- Leon. H. Charney Division of Cardiology, NYU Langone Health, 550 1st Avenue, New York, NY 10016, USA
| | - Tania Ahuja
- Department of Pharmacy, NYU Langone Health, New York, USA
| | - Glenn I Fishman
- Leon. H. Charney Division of Cardiology, NYU Langone Health, 550 1st Avenue, New York, NY 10016, USA
| | - Judith Hochman
- Leon. H. Charney Division of Cardiology, NYU Langone Health, 550 1st Avenue, New York, NY 10016, USA
| | - Stuart Katz
- Leon. H. Charney Division of Cardiology, NYU Langone Health, 550 1st Avenue, New York, NY 10016, USA
| | - Samuel Bernard
- Leon. H. Charney Division of Cardiology, NYU Langone Health, 550 1st Avenue, New York, NY 10016, USA
| | - Rajesh Ranganath
- Courant Institute of Mathematics, New York University, New York, USA
- Department of Population Health, NYU Langone Health, New York, USA
- Center for Data Science, New York University, New York, USA
| |
Collapse
|
2
|
Zhang H, Wang C, Yang N. Diagnostic performance of machine-learning algorithms for sepsis prediction: An updated meta-analysis. Technol Health Care 2024:THC240087. [PMID: 38968031 DOI: 10.3233/thc-240087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/07/2024]
Abstract
BACKGROUND Early identification of sepsis has been shown to significantly improve patient prognosis. OBJECTIVE Therefore, the aim of this meta-analysis is to systematically evaluate the diagnostic efficacy of machine-learning algorithms for sepsis prediction. METHODS Systematic searches were conducted in PubMed, Embase and Cochrane databases, covering literature up to December 2023. The keywords included machine learning, sepsis and prediction. After screening, data were extracted and analysed from studies meeting the inclusion criteria. Key evaluation metrics included sensitivity, specificity and the area under the curve (AUC) for diagnostic accuracy. RESULTS The meta-analysis included a total of 21 studies with a data sample size of 4,158,941. Overall, the pooled sensitivity was 0.82 (95% confidence interval [CI] = 0.70-0.90; P< 0.001; I2=99.7%), the specificity was 0.91 (95% CI = 0.86-0.94; P< 0.001; I2= 99.9%), and the AUC was 0.94 (95% CI = 0.91-0.96). The subgroup analysis revealed that in the emergency department setting (6 studies), the pooled sensitivity was 0.79 (95% CI = 0.68-0.87; P< 0.001; I2= 99.6%), the specificity was 0.94 (95% CI 0.90-0.97; P< 0.001; I2= 99.9%), and the AUC was 0.94 (95% CI = 0.92-0.96). In the Intensive Care Unit setting (11 studies), the sensitivity was 0.91 (95% CI = 0.75-0.97; P< 0.001; I2= 98.3%), the specificity was 0.85 (95% CI = 0.75-0.92; P< 0.001; I2= 99.9%), and the AUC was 0.93 (95% CI = 0.91-0.95). Due to the limited number of studies in the in-hospital and mixed settings (n< 3), no pooled analysis was performed. CONCLUSION Machine-learning algorithms have demonstrated excellent diagnostic accuracy in predicting the occurrence of sepsis, showing potential for clinical application.
Collapse
|
3
|
Dos Santos L, Silva LL, Pelloso FC, Maia V, Pujals C, Borghesan DH, Carvalho MD, Pedroso RB, Pelloso SM. Use of machine learning to identify protective factors for death from COVID-19 in the ICU: a retrospective study. PeerJ 2024; 12:e17428. [PMID: 38881861 PMCID: PMC11179634 DOI: 10.7717/peerj.17428] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Accepted: 04/29/2024] [Indexed: 06/18/2024] Open
Abstract
Background Patients in serious condition due to COVID-19 often require special care in intensive care units (ICUs). This disease has affected over 758 million people and resulted in 6.8 million deaths worldwide. Additionally, the progression of the disease may vary from individual to individual, that is, it is essential to identify the clinical parameters that indicate a good prognosis for the patient. Machine learning (ML) algorithms have been used for analyzing complex medical data and identifying prognostic indicators. However, there is still an urgent need for a model to elucidate the predictors related to patient outcomes. Therefore, this research aimed to verify, through ML, the variables involved in the discharge of patients admitted to the ICU due to COVID-19. Methods In this study, 126 variables were collected with information on demography, hospital length stay and outcome, chronic diseases and tumors, comorbidities and risk factors, complications and adverse events, health care, and vital indicators of patients admitted to an ICU in southern Brazil. These variables were filtered and then selected by a ML algorithm known as decision trees to identify the optimal set of variables for predicting patient discharge using logistic regression. Finally, a confusion matrix was performed to evaluate the model's performance for the selected variables. Results Of the 532 patients evaluated, 180 were discharged: female (16.92%), with a central venous catheter (23.68%), with a bladder catheter (26.13%), and with an average of 8.46- and 23.65-days using bladder catheter and submitted to mechanical ventilation, respectively. In addition, the chances of discharge increase by 14% for each additional day in the hospital, by 136% for female patients, 716% when there is no bladder catheter, and 737% when no central venous catheter is used. However, the chances of discharge decrease by 3% for each additional year of age and by 9% for each other day of mechanical ventilation. The performance of the training data presented a balanced accuracy of 0.81, sensitivity of 0.74, specificity of 0.88, and the kappa value was 0.64. The test performance had a balanced accuracy of 0.85, sensitivity 0.75, specificity 0.95, and kappa value of 0.73. The McNemar test found that there were no significant differences in the error rates in the training and test data, suggesting good classification. This work showed that female, the absence of a central venous catheter and bladder catheter, shorter mechanical ventilation, and bladder catheter duration were associated with a greater chance of hospital discharge. These results may help develop measures that lead to a good prognosis for the patient.
Collapse
Affiliation(s)
- Lander Dos Santos
- State University of Maringá, Graduate Program in Health Sciences, Maringá, Paraná, Brazil
| | - Lincoln Luis Silva
- Department of Emergency Medicine, Duke University School of Medicine, Durham, NC, United States of America
| | | | | | - Constanza Pujals
- State University of Maringá, Graduate Program in Health Sciences, Maringá, Paraná, Brazil
| | | | - Maria Dalva Carvalho
- State University of Maringá, Graduate Program in Health Sciences, Maringá, Paraná, Brazil
| | - Raíssa Bocchi Pedroso
- State University of Maringá, Graduate Program in Health Sciences, Maringá, Paraná, Brazil
| | - Sandra Marisa Pelloso
- State University of Maringá, Graduate Program in Health Sciences, Maringá, Paraná, Brazil
| |
Collapse
|
4
|
Padula WV, Armstrong DG, Pronovost PJ, Saria S. Predicting pressure injury risk in hospitalised patients using machine learning with electronic health records: a US multilevel cohort study. BMJ Open 2024; 14:e082540. [PMID: 38594078 PMCID: PMC11146395 DOI: 10.1136/bmjopen-2023-082540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/26/2023] [Accepted: 03/06/2024] [Indexed: 04/11/2024] Open
Abstract
OBJECTIVE To predict the risk of hospital-acquired pressure injury using machine learning compared with standard care. DESIGN We obtained electronic health records (EHRs) to structure a multilevel cohort of hospitalised patients at risk for pressure injury and then calibrate a machine learning model to predict future pressure injury risk. Optimisation methods combined with multilevel logistic regression were used to develop a predictive algorithm of patient-specific shifts in risk over time. Machine learning methods were tested, including random forests, to identify predictive features for the algorithm. We reported the results of the regression approach as well as the area under the receiver operating characteristics (ROC) curve for predictive models. SETTING Hospitalised inpatients. PARTICIPANTS EHRs of 35 001 hospitalisations over 5 years across 2 academic hospitals. MAIN OUTCOME MEASURE Longitudinal shifts in pressure injury risk. RESULTS The predictive algorithm with features generated by machine learning achieved significantly improved prediction of pressure injury risk (p<0.001) with an area under the ROC curve of 0.72; whereas standard care only achieved an area under the ROC curve of 0.52. At a specificity of 0.50, the predictive algorithm achieved a sensitivity of 0.75. CONCLUSIONS These data could help hospitals conserve resources within a critical period of patient vulnerability of hospital-acquired pressure injury which is not reimbursed by US Medicare; thus, conserving between 30 000 and 90 000 labour-hours per year in an average 500-bed hospital. Hospitals can use this predictive algorithm to initiate a quality improvement programme for pressure injury prevention and further customise the algorithm to patient-specific variation by facility.
Collapse
Affiliation(s)
- William V Padula
- Department of Pharmaceutical & Health Economics, University of Southern California Mann School of Pharmacy & Pharmaceutical Sciences, Los Angeles, CA, USA
- Stage Analytics, Suwanee, GA, USA
- The Leonard D. Schaeffer Center for Health Policy & Economics, University of Southern California, Los Angeles, CA, USA
| | - David G Armstrong
- The Leonard D. Schaeffer Center for Health Policy & Economics, University of Southern California, Los Angeles, CA, USA
- Department of Surgery, USC Keck School of Medicine, Los Angeles, California, USA
| | - Peter J Pronovost
- University Hospitals of Cleveland, Shaker Heights, Ohio, USA
- Anesthesiology and Critical Care Medicine, Case Western Reserve University School of Medicine, Cleveland, Ohio, USA
| | - Suchi Saria
- Department of Computer Science, Johns Hopkins University Whiting School of Engineering, Baltimore, Maryland, USA
- Department of Health Policy & Management, Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland, USA
| |
Collapse
|
5
|
Mares V, Nehemy MB, Bogunovic H, Frank S, Reiter GS, Schmidt-Erfurth U. AI-based support for optical coherence tomography in age-related macular degeneration. Int J Retina Vitreous 2024; 10:31. [PMID: 38589936 PMCID: PMC11000391 DOI: 10.1186/s40942-024-00549-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2024] [Accepted: 03/16/2024] [Indexed: 04/10/2024] Open
Abstract
Artificial intelligence (AI) has emerged as a transformative technology across various fields, and its applications in the medical domain, particularly in ophthalmology, has gained significant attention. The vast amount of high-resolution image data, such as optical coherence tomography (OCT) images, has been a driving force behind AI growth in this field. Age-related macular degeneration (AMD) is one of the leading causes for blindness in the world, affecting approximately 196 million people worldwide in 2020. Multimodal imaging has been for a long time the gold standard for diagnosing patients with AMD, however, currently treatment and follow-up in routine disease management are mainly driven by OCT imaging. AI-based algorithms have by their precision, reproducibility and speed, the potential to reliably quantify biomarkers, predict disease progression and assist treatment decisions in clinical routine as well as academic studies. This review paper aims to provide a summary of the current state of AI in AMD, focusing on its applications, challenges, and prospects.
Collapse
Affiliation(s)
- Virginia Mares
- Laboratory for Ophthalmic Image Analysis, Department of Ophthalmology and Optometry, Medical University of Vienna, Währinger Gürtel 18-20, 1090, Vienna, Austria
- Department of Ophthalmology, Federal University of Minas Gerais, Belo Horizonte, Brazil
| | - Marcio B Nehemy
- Department of Ophthalmology, Federal University of Minas Gerais, Belo Horizonte, Brazil
| | - Hrvoje Bogunovic
- Laboratory for Ophthalmic Image Analysis, Department of Ophthalmology and Optometry, Medical University of Vienna, Währinger Gürtel 18-20, 1090, Vienna, Austria
| | - Sophie Frank
- Laboratory for Ophthalmic Image Analysis, Department of Ophthalmology and Optometry, Medical University of Vienna, Währinger Gürtel 18-20, 1090, Vienna, Austria
| | - Gregor S Reiter
- Laboratory for Ophthalmic Image Analysis, Department of Ophthalmology and Optometry, Medical University of Vienna, Währinger Gürtel 18-20, 1090, Vienna, Austria
| | - Ursula Schmidt-Erfurth
- Laboratory for Ophthalmic Image Analysis, Department of Ophthalmology and Optometry, Medical University of Vienna, Währinger Gürtel 18-20, 1090, Vienna, Austria.
| |
Collapse
|
6
|
Persson I, Macura A, Becedas D, Sjövall F. Early prediction of sepsis in intensive care patients using the machine learning algorithm NAVOY® Sepsis, a prospective randomized clinical validation study. J Crit Care 2024; 80:154400. [PMID: 38245375 DOI: 10.1016/j.jcrc.2023.154400] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2023] [Revised: 08/09/2023] [Accepted: 08/11/2023] [Indexed: 01/22/2024]
Abstract
PURPOSE To prospectively validate, in an ICU setting, the prognostic accuracy of the sepsis prediction algorithm NAVOY® Sepsis which uses 4 h of input for routinely collected vital parameters, blood gas values, and lab values. MATERIALS AND METHODS Patients 18 years or older admitted to the ICU at Skåne University Hospital Malmö from December 2020 to September 2021 were recruited in the study. A total of 304 patients were randomized into one of two groups: Algorithm group with active sepsis alerts, or Standard of care. NAVOY® Sepsis made silent predictions in the Standard of care group, in order to evaluate its performance without disturbing the outcome. The study was blinded, i.e., study personnel did not know to which group patients were randomized. The healthcare provider followed standard practices in assessing possible development of sepsis and intervening accordingly. The patients were followed-up in the study until ICU discharge. RESULTS NAVOY® Sepsis could predict the development of sepsis, according to the Sepsis-3 criteria, three hours before sepsis onset with high performance: accuracy 0.79; sensitivity 0.80; and specificity 0.78. CONCLUSIONS The accuracy, sensitivity, and specificity were all high, validating the prognostic accuracy of NAVOY® Sepsis in an ICU setting, including Covid-19 patients.
Collapse
Affiliation(s)
- Inger Persson
- Department of Statistics, Uppsala University, Uppsala, Sweden, AlgoDx AB, Stockholm, Sweden.
| | | | | | - Fredrik Sjövall
- Department of Intensive- and Perioperative Medicine, Skåne University Hospital, Malmö, Sweden
| |
Collapse
|
7
|
Liu Y, Yo CH, Hu JR, Hsu WT, Hsiung JC, Chang YH, Chen SC, Lee CC. Sepsis increases the risk of in-hospital cardiac arrest: a population-based analysis. Intern Emerg Med 2024; 19:353-363. [PMID: 38141118 DOI: 10.1007/s11739-023-03475-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Accepted: 10/19/2023] [Indexed: 12/24/2023]
Abstract
Sepsis patients have a high risk of developing in-hospital cardiac arrest (IHCA), which portends poor survival. However, little is known about whether the increased incidence of IHCA is due to sepsis itself or to comorbidities harbored by sepsis patients. We conducted a retrospective population-based cohort study comprising 20,022 patients admitted with sepsis to hospitals in Taiwan using the National Health Insurance Research Database (NHIRD). We constructed three non-sepsis comparison cohorts using risk set sampling and propensity score (PS) matching. We used univariate conditional logistic regression to evaluate the risk of IHCA and associated mortality. We identified 12,790 inpatients without infection (matched cohort 1), 12,789 inpatients with infection but without sepsis (matched cohort 2), and 10,536 inpatients with end-organ dysfunction but without sepsis (matched cohort 3). In the three PS-matched cohorts, the odds ratios (OR) for developing ICHA were 21.17 (95% CI 17.19, 26.06), 18.96 (95% CI: 15.56, 23.10), and 1.23 (95% CI: 1.13, 1.33), respectively (p < 0.001 for all ORs). In conclusion, in our study of inpatients across Taiwan, sepsis was independently associated with an increased risk of IHCA. Further studies should focus on identifying the proxy causes of IHCA using real-time monitoring data to further reduce the incidence of cardiopulmonary insufficiency in patients with sepsis.
Collapse
Affiliation(s)
- Ye Liu
- Department of Health Policy and Organization, School of Public Health, University of Alabama at Birmingham, Birmingham, AL, USA
| | - Chia-Hung Yo
- Department of Emergency Medicine, Far Eastern Memorial Hospital, Taipei, Taiwan
| | - Jiun-Ruey Hu
- Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA
| | - Wan-Ting Hsu
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Jo-Ching Hsiung
- Department of Pediatrics, Jefferson Einstein Hospital, Philadelphia, PA, USA
| | - Yung-Han Chang
- Department of Biostatistics, University of California, Los Angeles, Los Angeles, USA
| | - Shyr-Chyr Chen
- Department of Emergency Medicine, National Taiwan University Hospital, Taipei, Taiwan
| | - Chien-Chang Lee
- Department of Emergency Medicine, National Taiwan University Hospital, Taipei, Taiwan.
- The Centre for Intelligent Healthcare, College of Medicine, National Taiwan University Hospital, National Taiwan University, No.7, Chung Shan S. Rd., Zhongzheng Dist., Taipei City, 100, Taiwan.
| |
Collapse
|
8
|
Briggs J, Kostakis I, Meredith P, Dall'ora C, Darbyshire J, Gerry S, Griffiths P, Hope J, Jones J, Kovacs C, Lawrence R, Prytherch D, Watkinson P, Redfern O. Safer and more efficient vital signs monitoring protocols to identify the deteriorating patients in the general hospital ward: an observational study. HEALTH AND SOCIAL CARE DELIVERY RESEARCH 2024; 12:1-143. [PMID: 38551079 DOI: 10.3310/hytr4612] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/02/2024]
Abstract
Background The frequency at which patients should have their vital signs (e.g. blood pressure, pulse, oxygen saturation) measured on hospital wards is currently unknown. Current National Health Service monitoring protocols are based on expert opinion but supported by little empirical evidence. The challenge is finding the balance between insufficient monitoring (risking missing early signs of deterioration and delays in treatment) and over-observation of stable patients (wasting resources needed in other aspects of care). Objective Provide an evidence-based approach to creating monitoring protocols based on a patient's risk of deterioration and link these to nursing workload and economic impact. Design Our study consisted of two parts: (1) an observational study of nursing staff to ascertain the time to perform vital sign observations; and (2) a retrospective study of historic data on patient admissions exploring the relationships between National Early Warning Score and risk of outcome over time. These were underpinned by opinions and experiences from stakeholders. Setting and participants Observational study: observed nursing staff on 16 randomly selected adult general wards at four acute National Health Service hospitals. Retrospective study: extracted, linked and analysed routinely collected data from two large National Health Service acute trusts; data from over 400,000 patient admissions and 9,000,000 vital sign observations. Results Observational study found a variety of practices, with two hospitals having registered nurses take the majority of vital sign observations and two favouring healthcare assistants or student nurses. However, whoever took the observations spent roughly the same length of time. The average was 5:01 minutes per observation over a 'round', including time to locate and prepare the equipment and travel to the patient area. Retrospective study created survival models predicting the risk of outcomes over time since the patient was last observed. For low-risk patients, there was little difference in risk between 4 hours and 24 hours post observation. Conclusions We explored several different scenarios with our stakeholders (clinicians and patients), based on how 'risk' could be managed in different ways. Vital sign observations are often done more frequently than necessary from a bald assessment of the patient's risk, and we show that a maximum threshold of risk could theoretically be achieved with less resource. Existing resources could therefore be redeployed within a changed protocol to achieve better outcomes for some patients without compromising the safety of the rest. Our work supports the approach of the current monitoring protocol, whereby patients' National Early Warning Score 2 guides observation frequency. Existing practice is to observe higher-risk patients more frequently and our findings have shown that this is objectively justified. It is worth noting that important nurse-patient interactions take place during vital sign monitoring and should not be eliminated under new monitoring processes. Our study contributes to the existing evidence on how vital sign observations should be scheduled. However, ultimately, it is for the relevant professionals to decide how our work should be used. Study registration This study is registered as ISRCTN10863045. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 17/05/03) and is published in full in Health and Social Care Delivery Research; Vol. 12, No. 6. See the NIHR Funding and Awards website for further award information.
Collapse
Affiliation(s)
- Jim Briggs
- Centre for Healthcare Modelling and Informatics, University of Portsmouth, Portsmouth, UK
| | - Ina Kostakis
- Centre for Healthcare Modelling and Informatics, University of Portsmouth, Portsmouth, UK
| | - Paul Meredith
- Research Department, Portsmouth Hospitals University NHS Trust, Portsmouth, UK
| | | | - Julie Darbyshire
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Stephen Gerry
- Centre for Statistics in Medicine, University of Oxford, Oxford, UK
| | | | - Jo Hope
- Health Sciences, University of Southampton, Southampton, UK
| | - Jeremy Jones
- Health Sciences, University of Southampton, Southampton, UK
| | - Caroline Kovacs
- Centre for Healthcare Modelling and Informatics, University of Portsmouth, Portsmouth, UK
| | | | - David Prytherch
- Centre for Healthcare Modelling and Informatics, University of Portsmouth, Portsmouth, UK
| | - Peter Watkinson
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Oliver Redfern
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| |
Collapse
|
9
|
Upadhyaya DP, Tarabichi Y, Prantzalos K, Ayub S, Kaelber DC, Sahoo SS. Machine Learning Interpretability Methods to Characterize the Importance of Hematologic Biomarkers in Prognosticating Patients with Suspected Infection. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2023.05.30.23290757. [PMID: 37398448 PMCID: PMC10312863 DOI: 10.1101/2023.05.30.23290757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/04/2023]
Abstract
Early detection of sepsis in patients admitted to the emergency department (ED) is an important clinical objective as early identification and treatment can help reduce morbidity and mortality rate of 20% or higher. Hematologic changes during sepsis-associated organ dysfunction are well established and a new biomarker called Monocyte Distribution Width (MDW) has been recently approved by the US Food and Drug Administration for sepsis. However, MDW, which quantifies monocyte activation in sepsis patients, is not a routinely reported parameter and it requires specialized proprietary laboratory equipment. Further, the relative importance of MDW as compared to other routinely available hematologic parameters and vital signs has not been studied, which makes it difficult for resource constrained hospital systems to make informed decisions in this regard. To address this issue, we analyzed data from a cohort of ED patients (n=10,229) admitted to a large regional safety-net hospital in Cleveland, Ohio with suspected infection who later developed poor outcomes associated with sepsis. We developed a new analytical framework consisting of seven data models and an ensemble of high accuracy machine learning (ML) algorithms (accuracy values ranging from 0.83 to 0.90) for the prediction of outcomes more common in sepsis than uncomplicated infection (3-day intensive care unit stay or death). To characterize the contributions of individual hematologic parameters, we applied the Local Interpretable Model-Agnostic Explanation (LIME) and Shapley Additive Value (SHAP) interpretability methods to the high accuracy ML algorithms. The ML interpretability results were consistent in their findings that the value of MDW is grossly attenuated in the presence of other routinely reported hematologic parameters and vital signs data. Further, this study for the first time shows that complete blood count with differential (CBC-DIFF) together with vital signs data can be used as a substitute for MDW in high accuracy ML algorithms to screen for poor outcomes associated with sepsis.
Collapse
|
10
|
Leisman DE, Deng H, Lee AH, Flynn MH, Rutkey H, Copenhaver MS, Gay EA, Dutta S, McEvoy DS, Dunham LN, Mort EA, Lucier DJ, Sonis JD, Aaronson EL, Hibbert KA, Safavi KC. Effect of Automated Real-Time Feedback on Early-Sepsis Care: A Pragmatic Clinical Trial. Crit Care Med 2024; 52:210-222. [PMID: 38088767 DOI: 10.1097/ccm.0000000000006057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2024]
Abstract
OBJECTIVES To determine if a real-time monitoring system with automated clinician alerts improves 3-hour sepsis bundle adherence. DESIGN Prospective, pragmatic clinical trial. Allocation alternated every 7 days. SETTING Quaternary hospital from December 1, 2020 to November 30, 2021. PATIENTS Adult emergency department or inpatients meeting objective sepsis criteria triggered an electronic medical record (EMR)-embedded best practice advisory. Enrollment occurred when clinicians acknowledged the advisory indicating they felt sepsis was likely. INTERVENTION Real-time automated EMR monitoring identified suspected sepsis patients with incomplete bundle measures within 1-hour of completion deadlines and generated reminder pages. Clinicians responsible for intervention group patients received reminder pages; no pages were sent for controls. The primary analysis cohort was the subset of enrolled patients at risk of bundle nonadherent care that had reminder pages generated. MEASUREMENTS AND MAIN RESULTS The primary outcome was orders for all 3-hour bundle elements within guideline time limits. Secondary outcomes included guideline-adherent delivery of all 3-hour bundle elements, 28-day mortality, antibiotic discontinuation within 48-hours, and pathogen recovery from any culture within 7 days of time-zero. Among 3,269 enrolled patients, 1,377 had reminder pages generated and were included in the primary analysis. There were 670 (48.7%) at-risk patients randomized to paging alerts and 707 (51.3%) to control. Bundle-adherent orders were placed for 198 intervention patients (29.6%) versus 149 (21.1%) controls (difference: 8.5%; 95% CI, 3.9-13.1%; p = 0.0003). Bundle-adherent care was delivered for 152 (22.7%) intervention versus 121 (17.1%) control patients (difference: 5.6%; 95% CI, 1.4-9.8%; p = 0.0095). Mortality was similar between groups (8.4% vs 8.3%), as were early antibiotic discontinuation (35.1% vs 33.4%) and pan-culture negativity (69.0% vs 68.2%). CONCLUSIONS Real-time monitoring and paging alerts significantly increased orders for and delivery of guideline-adherent care for suspected sepsis patients at risk of 3-hour bundle nonadherence. The trial was underpowered to determine whether adherence affected mortality. Despite enrolling patients with clinically suspected sepsis, early antibiotic discontinuation and pan-culture negativity were common, highlighting challenges in identifying appropriate patients for sepsis bundle application.
Collapse
Affiliation(s)
- Daniel E Leisman
- Department of Medicine, Massachusetts General Hospital, Boston, MA
- Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Boston, MA
| | - Hao Deng
- Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Boston, MA
| | - Andy H Lee
- Department of Emergency Medicine, Massachusetts General Hospital, Boston, MA
- Department of Emergency Medicine, Harvard Medical School, Boston, MA
| | - Micah H Flynn
- Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Boston, MA
| | - Hayley Rutkey
- Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Boston, MA
| | - Martin S Copenhaver
- Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Boston, MA
- Healthcare Systems Engineering, Massachusetts General Hospital, Boston, MA
| | - Elizabeth A Gay
- Department of Medicine, Massachusetts General Hospital, Boston, MA
| | - Sayon Dutta
- Department of Emergency Medicine, Massachusetts General Hospital, Boston, MA
- Department of Emergency Medicine, Harvard Medical School, Boston, MA
- Mass General Brigham Digital, Mass General Brigham Health System, Sommerville, MA
| | - Dustin S McEvoy
- Mass General Brigham Digital, Mass General Brigham Health System, Sommerville, MA
| | - Lisette N Dunham
- Mass General Brigham Digital, Mass General Brigham Health System, Sommerville, MA
| | - Elizabeth A Mort
- Department of Medicine, Massachusetts General Hospital, Boston, MA
| | - David J Lucier
- Department of Medicine, Massachusetts General Hospital, Boston, MA
| | - Jonathan D Sonis
- Department of Emergency Medicine, Massachusetts General Hospital, Boston, MA
- Department of Emergency Medicine, Harvard Medical School, Boston, MA
| | - Emily L Aaronson
- Department of Emergency Medicine, Massachusetts General Hospital, Boston, MA
- Department of Emergency Medicine, Harvard Medical School, Boston, MA
| | - Kathryn A Hibbert
- Division of Pulmonary and Critical Care Medicine, Department of Medicine, Massachusetts General Hospital, Boston, MA
- Department of Medicine, Harvard Medical School, Boston, MA
| | - Kyan C Safavi
- Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Boston, MA
- Healthcare Systems Engineering, Massachusetts General Hospital, Boston, MA
| |
Collapse
|
11
|
Boussina A, Shashikumar SP, Malhotra A, Owens RL, El-Kareh R, Longhurst CA, Quintero K, Donahue A, Chan TC, Nemati S, Wardi G. Impact of a deep learning sepsis prediction model on quality of care and survival. NPJ Digit Med 2024; 7:14. [PMID: 38263386 PMCID: PMC10805720 DOI: 10.1038/s41746-023-00986-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Accepted: 12/06/2023] [Indexed: 01/25/2024] Open
Abstract
Sepsis remains a major cause of mortality and morbidity worldwide. Algorithms that assist with the early recognition of sepsis may improve outcomes, but relatively few studies have examined their impact on real-world patient outcomes. Our objective was to assess the impact of a deep-learning model (COMPOSER) for the early prediction of sepsis on patient outcomes. We completed a before-and-after quasi-experimental study at two distinct Emergency Departments (EDs) within the UC San Diego Health System. We included 6217 adult septic patients from 1/1/2021 through 4/30/2023. The exposure tested was a nurse-facing Best Practice Advisory (BPA) triggered by COMPOSER. In-hospital mortality, sepsis bundle compliance, 72-h change in sequential organ failure assessment (SOFA) score following sepsis onset, ICU-free days, and the number of ICU encounters were evaluated in the pre-intervention period (705 days) and the post-intervention period (145 days). The causal impact analysis was performed using a Bayesian structural time-series approach with confounder adjustments to assess the significance of the exposure at the 95% confidence level. The deployment of COMPOSER was significantly associated with a 1.9% absolute reduction (17% relative decrease) in in-hospital sepsis mortality (95% CI, 0.3%-3.5%), a 5.0% absolute increase (10% relative increase) in sepsis bundle compliance (95% CI, 2.4%-8.0%), and a 4% (95% CI, 1.1%-7.1%) reduction in 72-h SOFA change after sepsis onset in causal inference analysis. This study suggests that the deployment of COMPOSER for early prediction of sepsis was associated with a significant reduction in mortality and a significant increase in sepsis bundle compliance.
Collapse
Affiliation(s)
- Aaron Boussina
- Department of Medicine, University of California San Diego, San Diego, CA, USA
| | | | - Atul Malhotra
- Department of Medicine, University of California San Diego, San Diego, CA, USA
| | - Robert L Owens
- Department of Medicine, University of California San Diego, San Diego, CA, USA
| | - Robert El-Kareh
- Department of Medicine, University of California San Diego, San Diego, CA, USA
- Department of Quality, University of California San Diego, San Diego, CA, USA
| | - Christopher A Longhurst
- Department of Medicine, University of California San Diego, San Diego, CA, USA
- Department of Quality, University of California San Diego, San Diego, CA, USA
| | - Kimberly Quintero
- Department of Quality, University of California San Diego, San Diego, CA, USA
| | - Allison Donahue
- Department of Emergency Medicine, University of California San Diego, San Diego, CA, USA
| | - Theodore C Chan
- Department of Emergency Medicine, University of California San Diego, San Diego, CA, USA
| | - Shamim Nemati
- Department of Medicine, University of California San Diego, San Diego, CA, USA
- Department of Emergency Medicine, University of California San Diego, San Diego, CA, USA
| | - Gabriel Wardi
- Department of Medicine, University of California San Diego, San Diego, CA, USA.
- Department of Emergency Medicine, University of California San Diego, San Diego, CA, USA.
| |
Collapse
|
12
|
Cohen SN, Foster J, Foster P, Lou H, Lyons T, Morley S, Morrill J, Ni H, Palmer E, Wang B, Wu Y, Yang L, Yang W. Subtle variation in sepsis-III definitions markedly influences predictive performance within and across methods. Sci Rep 2024; 14:1920. [PMID: 38253623 PMCID: PMC10803347 DOI: 10.1038/s41598-024-51989-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Accepted: 01/11/2024] [Indexed: 01/24/2024] Open
Abstract
Early detection of sepsis is key to ensure timely clinical intervention. Since very few end-to-end pipelines are publicly available, fair comparisons between methodologies are difficult if not impossible. Progress is further limited by discrepancies in the reconstruction of sepsis onset time. This retrospective cohort study highlights the variation in performance of predictive models under three subtly different interpretations of sepsis onset from the sepsis-III definition and compares this against inter-model differences. The models are chosen to cover tree-based, deep learning, and survival analysis methods. Using the MIMIC-III database, between 867 and 2178 intensive care unit admissions with sepsis were identified, depending on the onset definition. We show that model performance can be more sensitive to differences in the definition of sepsis onset than to the model itself. Given a fixed sepsis definition, the best performing method had a gain of 1-5% in the area under the receiver operating characteristic (AUROC). However, the choice of onset time can cause a greater effect, with variation of 0-6% in AUROC. We illustrate that misleading conclusions can be drawn if models are compared without consideration of the sepsis definition used which emphasizes the need for a standardized definition for sepsis onset.
Collapse
Affiliation(s)
- Samuel N Cohen
- Mathematical Institute, University of Oxford, Oxford, UK
- The Alan Turing Institute, London, UK
| | - James Foster
- Department of Mathematical Sciences, University of Bath, Bath, UK
| | | | - Hang Lou
- Department of Mathematics, University College London, Room 603, 25 Gordon St, London, WC1H 0AY, UK
| | - Terry Lyons
- Mathematical Institute, University of Oxford, Oxford, UK
| | - Sam Morley
- Mathematical Institute, University of Oxford, Oxford, UK
| | - James Morrill
- Mathematical Institute, University of Oxford, Oxford, UK
| | - Hao Ni
- Department of Mathematics, University College London, Room 603, 25 Gordon St, London, WC1H 0AY, UK.
| | - Edward Palmer
- Bloomsbury Institute of Intensive Care Medicine, University College London, London, UK
| | - Bo Wang
- The Alan Turing Institute, London, UK
- Center for Precision Psychiatry, Massachusetts General Hospital, Boston, MA, USA
| | - Yue Wu
- Department of Mathematics and Statistics, University of Strathclyde, Glasgow, UK
| | - Lingyi Yang
- Mathematical Institute, University of Oxford, Oxford, UK
- The Alan Turing Institute, London, UK
| | - Weixin Yang
- Mathematical Institute, University of Oxford, Oxford, UK
| |
Collapse
|
13
|
Samad M, Angel M, Rinehart J, Kanomata Y, Baldi P, Cannesson M. Medical Informatics Operating Room Vitals and Events Repository (MOVER): a public-access operating room database. JAMIA Open 2023; 6:ooad084. [PMID: 37860605 PMCID: PMC10582520 DOI: 10.1093/jamiaopen/ooad084] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Revised: 08/18/2023] [Accepted: 10/06/2023] [Indexed: 10/21/2023] Open
Abstract
Objectives Artificial intelligence (AI) holds great promise for transforming the healthcare industry. However, despite its potential, AI is yet to see widespread deployment in clinical settings in significant part due to the lack of publicly available clinical data and the lack of transparency in the published AI algorithms. There are few clinical data repositories publicly accessible to researchers to train and test AI algorithms, and even fewer that contain specialized data from the perioperative setting. To address this gap, we present and release the Medical Informatics Operating Room Vitals and Events Repository (MOVER). Materials and Methods This first release of MOVER includes adult patients who underwent surgery at the University of California, Irvine Medical Center from 2015 to 2022. Data for patients who underwent surgery were captured from 2 different sources: High-fidelity physiological waveforms from all of the operating rooms were captured in real time and matched with electronic medical record data. Results MOVER includes data from 58 799 unique patients and 83 468 surgeries. MOVER is available for download at https://doi.org/10.24432/C5VS5G, it can be downloaded by anyone who signs a data usage agreement (DUA), to restrict traffic to legitimate researchers. Discussion To the best of our knowledge MOVER is the only freely available public data repository that contains electronic health record and high-fidelity physiological waveforms data for patients undergoing surgery. Conclusion MOVER is freely available to all researchers who sign a DUA, and we hope that it will accelerate the integration of AI into healthcare settings, ultimately leading to improved patient outcomes.
Collapse
Affiliation(s)
- Muntaha Samad
- Department of Computer Science, University of California, Irvine, Irvine, CA 92697, United States
- Institute for Genomics and Bioinformatics, University of California, Irvine, Irvine, CA 92697, United States
| | - Mirana Angel
- Department of Computer Science, University of California, Irvine, Irvine, CA 92697, United States
- Institute for Genomics and Bioinformatics, University of California, Irvine, Irvine, CA 92697, United States
| | - Joseph Rinehart
- Department of Anesthesiology & Perioperative Care, University of California, Irvine, Irvine, CA 92697, United States
| | - Yuzo Kanomata
- Department of Computer Science, University of California, Irvine, Irvine, CA 92697, United States
- Institute for Genomics and Bioinformatics, University of California, Irvine, Irvine, CA 92697, United States
| | - Pierre Baldi
- Department of Computer Science, University of California, Irvine, Irvine, CA 92697, United States
- Institute for Genomics and Bioinformatics, University of California, Irvine, Irvine, CA 92697, United States
| | - Maxime Cannesson
- Department of Anesthesiology & Perioperative Medicine, University of California, Los Angeles, Los Angeles, CA 90095, United States
| |
Collapse
|
14
|
Kang HYJ, Batbaatar E, Choi DW, Choi KS, Ko M, Ryu KS. Synthetic Tabular Data Based on Generative Adversarial Networks in Health Care: Generation and Validation Using the Divide-and-Conquer Strategy. JMIR Med Inform 2023; 11:e47859. [PMID: 37999942 DOI: 10.2196/47859] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 08/02/2023] [Accepted: 10/28/2023] [Indexed: 11/25/2023] Open
Abstract
BACKGROUND Synthetic data generation (SDG) based on generative adversarial networks (GANs) is used in health care, but research on preserving data with logical relationships with synthetic tabular data (STD) remains challenging. Filtering methods for SDG can lead to the loss of important information. OBJECTIVE This study proposed a divide-and-conquer (DC) method to generate STD based on the GAN algorithm, while preserving data with logical relationships. METHODS The proposed method was evaluated on data from the Korea Association for Lung Cancer Registry (KALC-R) and 2 benchmark data sets (breast cancer and diabetes). The DC-based SDG strategy comprises 3 steps: (1) We used 2 different partitioning methods (the class-specific criterion distinguished between survival and death groups, while the Cramer V criterion identified the highest correlation between columns in the original data); (2) the entire data set was divided into a number of subsets, which were then used as input for the conditional tabular generative adversarial network and the copula generative adversarial network to generate synthetic data; and (3) the generated synthetic data were consolidated into a single entity. For validation, we compared DC-based SDG and conditional sampling (CS)-based SDG through the performances of machine learning models. In addition, we generated imbalanced and balanced synthetic data for each of the 3 data sets and compared their performance using 4 classifiers: decision tree (DT), random forest (RF), Extreme Gradient Boosting (XGBoost), and light gradient-boosting machine (LGBM) models. RESULTS The synthetic data of the 3 diseases (non-small cell lung cancer [NSCLC], breast cancer, and diabetes) generated by our proposed model outperformed the 4 classifiers (DT, RF, XGBoost, and LGBM). The CS- versus DC-based model performances were compared using the mean area under the curve (SD) values: 74.87 (SD 0.77) versus 63.87 (SD 2.02) for NSCLC, 73.31 (SD 1.11) versus 67.96 (SD 2.15) for breast cancer, and 61.57 (SD 0.09) versus 60.08 (SD 0.17) for diabetes (DT); 85.61 (SD 0.29) versus 79.01 (SD 1.20) for NSCLC, 78.05 (SD 1.59) versus 73.48 (SD 4.73) for breast cancer, and 59.98 (SD 0.24) versus 58.55 (SD 0.17) for diabetes (RF); 85.20 (SD 0.82) versus 76.42 (SD 0.93) for NSCLC, 77.86 (SD 2.27) versus 68.32 (SD 2.37) for breast cancer, and 60.18 (SD 0.20) versus 58.98 (SD 0.29) for diabetes (XGBoost); and 85.14 (SD 0.77) versus 77.62 (SD 1.85) for NSCLC, 78.16 (SD 1.52) versus 70.02 (SD 2.17) for breast cancer, and 61.75 (SD 0.13) versus 61.12 (SD 0.23) for diabetes (LGBM). In addition, we found that balanced synthetic data performed better. CONCLUSIONS This study is the first attempt to generate and validate STD based on a DC approach and shows improved performance using STD. The necessity for balanced SDG was also demonstrated.
Collapse
Affiliation(s)
- Ha Ye Jin Kang
- Department of Applied Artificial Intelligence, Hanyang University, Ansan, Republic of Korea
- Department of Cancer AI & Digital Health, Graduate School of Cancer Science and Policy, National Cancer Center, Gyeonggi-do, Republic of Korea
| | - Erdenebileg Batbaatar
- National Cancer Data Center, National Cancer Control Institute, National Cancer Center, Gyeonggi-do, Republic of Korea
| | - Dong-Woo Choi
- National Cancer Data Center, National Cancer Control Institute, National Cancer Center, Gyeonggi-do, Republic of Korea
| | - Kui Son Choi
- National Cancer Data Center, National Cancer Control Institute, National Cancer Center, Gyeonggi-do, Republic of Korea
- Department of Cancer Control and Policy, Graduate School of Cancer Science and Policy, National Cancer Center, Gyeonggi-do, Republic of Korea
| | - Minsam Ko
- Department of Applied Artificial Intelligence, Hanyang University, Ansan, Republic of Korea
- Department of Human-Computer Interaction, Hanyang University, Ansan, Republic of Korea
| | - Kwang Sun Ryu
- Department of Cancer AI & Digital Health, Graduate School of Cancer Science and Policy, National Cancer Center, Gyeonggi-do, Republic of Korea
- National Cancer Data Center, National Cancer Control Institute, National Cancer Center, Gyeonggi-do, Republic of Korea
| |
Collapse
|
15
|
Lashen H, St John TL, Almallah YZ, Sasidhar M, Shamout FE. Machine Learning Models Versus the National Early Warning Score System for Predicting Deterioration: Retrospective Cohort Study in the United Arab Emirates. JMIR AI 2023; 2:e45257. [PMID: 38875543 PMCID: PMC11041421 DOI: 10.2196/45257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Revised: 06/19/2023] [Accepted: 08/01/2023] [Indexed: 06/16/2024]
Abstract
BACKGROUND Early warning score systems are widely used for identifying patients who are at the highest risk of deterioration to assist clinical decision-making. This could facilitate early intervention and consequently improve patient outcomes; for example, the National Early Warning Score (NEWS) system, which is recommended by the Royal College of Physicians in the United Kingdom, uses predefined alerting thresholds to assign scores to patients based on their vital signs. However, there is limited evidence of the reliability of such scores across patient cohorts in the United Arab Emirates. OBJECTIVE Our aim in this study was to propose a data-driven model that accurately predicts in-hospital deterioration in an inpatient cohort in the United Arab Emirates. METHODS We conducted a retrospective cohort study using a real-world data set that consisted of 16,901 unique patients associated with 26,073 inpatient emergency encounters and 951,591 observation sets collected between April 2015 and August 2021 at a large multispecialty hospital in Abu Dhabi, United Arab Emirates. The observation sets included routine measurements of heart rate, respiratory rate, systolic blood pressure, level of consciousness, temperature, and oxygen saturation, as well as whether the patient was receiving supplementary oxygen. We divided the data set of 16,901 unique patients into training, validation, and test sets consisting of 11,830 (70%; 18,319/26,073, 70.26% emergency encounters), 3397 (20.1%; 5206/26,073, 19.97% emergency encounters), and 1674 (9.9%; 2548/26,073, 9.77% emergency encounters) patients, respectively. We defined an adverse event as the occurrence of admission to the intensive care unit, mortality, or both if the patient was admitted to the intensive care unit first. On the basis of 7 routine vital signs measurements, we assessed the performance of the NEWS system in detecting deterioration within 24 hours using the area under the receiver operating characteristic curve (AUROC). We also developed and evaluated several machine learning models, including logistic regression, a gradient-boosting model, and a feed-forward neural network. RESULTS In a holdout test set of 2548 encounters with 95,755 observation sets, the NEWS system achieved an overall AUROC value of 0.682 (95% CI 0.673-0.690). In comparison, the best-performing machine learning models, which were the gradient-boosting model and the neural network, achieved AUROC values of 0.778 (95% CI 0.770-0.785) and 0.756 (95% CI 0.749-0.764), respectively. Our interpretability results highlight the importance of temperature and respiratory rate in predicting patient deterioration. CONCLUSIONS Although traditional early warning score systems are the dominant form of deterioration prediction models in clinical practice today, we strongly recommend the development and use of cohort-specific machine learning models as an alternative. This is especially important in external patient cohorts that were unseen during model development.
Collapse
Affiliation(s)
- Hazem Lashen
- Engineering Division, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| | | | | | - Madhu Sasidhar
- Cleveland Clinic Tradition Hospital, Port St. Lucie, FL, United States
| | - Farah E Shamout
- Engineering Division, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| |
Collapse
|
16
|
Jiang J, Zou Y, Xie C, Yang M, Tong Q, Yuan M, Pei X, Deng S, Tian M, Xiao L, Gong Y. Oxytocin alleviates cognitive and memory impairments by decreasing hippocampal microglial activation and synaptic defects via OXTR/ERK/STAT3 pathway in a mouse model of sepsis-associated encephalopathy. Brain Behav Immun 2023; 114:195-213. [PMID: 37648002 DOI: 10.1016/j.bbi.2023.08.023] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 08/09/2023] [Accepted: 08/26/2023] [Indexed: 09/01/2023] Open
Abstract
BACKGROUND Sepsis-associated encephalopathy (SAE) is a diffuse brain dysfunction, characterized by cognitive and memory impairments closely linked to hippocampal dysfunction. Though it is well-known that SAE is a diffuse brain dysfunction with microglial activation, the pathological mechanisms of SAE are not well established and effective clinical interventions are lacking. Oxytocin (OXT) is reported to have anti-inflammatory and neuroprotective roles. However, the effects of OXT on SAE and the underlying mechanisms are not clear. METHODS SAE was induced in adult C57BL/6J male mice by cecal ligation and perforation (CLP) surgery. Exogenous OXT was intranasally applied after surgery. Clinical score, survivor rate, cognitive and memory behaviors, and hippocampal neuronal and non-neuronal functions were evaluated. Cultured microglia challenged with lipopolysaccharide (LPS) were used to investigate the effects of OXT on microglial functions, including inflammatory cytokines release and phagocytosis. The possible intracellular signal pathways involved in the OXT-induced neuroprotection were explored with RNA sequencing. RESULTS Hippocampal OXT level decreases, while the expression of OXT receptor (OXTR) increases around 24 h after CLP surgery. Intranasal OXT application at a proper dose increases mouse survival rate, alleviates cognitive and memory dysfunction, and restores hippocampal synaptic function and neuronal activity via OXTR in the SAE model. Intraperitoneal or local administration of the OXTR antagonist L-368,899 in hippocampal CA1 region inhibited the protective effects of OXT. Moreover, during the early stages of sepsis, hippocampal microglia are activated, while OXT application reduces microglial phagocytosis and the release of inflammatory cytokines, thereby exerting a neuroprotective effect. OXT may improve the SAE outcomes via the OXTR-ERK-STAT3 signaling pathway. CONCLUSION Our study uncovers the dysfunction of the OXT signal in SAE and shows that intranasal OXT application at a proper dose can alleviate SAE outcomes by reducing microglial overactivation, suggests that OXT may be a promising therapeutic approach in managing SAE patients.
Collapse
Affiliation(s)
- Junliang Jiang
- Department of Critical Care Medicine and Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, Shanghai, China; Department of Orthopedics & Traumatology, Affiliated Hospital of Yunnan University, Yunnan University, Kunming, China
| | - Yue Zou
- Yunnan Eye Institute & Key Laboratory of Yunnan Province, Yunnan Eye Disease Clinical Medical Center, Affiliated Hospital of Yunnan University, Yunnan University, Kunming, China
| | - Chuantong Xie
- Department of Critical Care Medicine and Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, Shanghai, China
| | - Miaoxian Yang
- Department of Critical Care Medicine and Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, Shanghai, China
| | - Qiuping Tong
- Department of Critical Care Medicine and Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, Shanghai, China
| | - Mimi Yuan
- Department of Critical Care Medicine and Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, Shanghai, China
| | - Xu Pei
- Department of Critical Care Medicine and Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, Shanghai, China
| | - Shuixiang Deng
- Department of Critical Care Medicine and Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, Shanghai, China
| | - Mi Tian
- Department of Critical Care Medicine and Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, Shanghai, China
| | - Lei Xiao
- Department of Critical Care Medicine and Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, Shanghai, China.
| | - Ye Gong
- Department of Critical Care Medicine and Neurosurgery of Huashan Hospital, State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, Shanghai, China.
| |
Collapse
|
17
|
Mushtaq AH, Shafqat A, Salah HT, Hashmi SK, Muhsen IN. Machine learning applications and challenges in graft-versus-host disease: a scoping review. Curr Opin Oncol 2023; 35:594-600. [PMID: 37820094 DOI: 10.1097/cco.0000000000000996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/13/2023]
Abstract
PURPOSE OF REVIEW This review delves into the potential of artificial intelligence (AI), particularly machine learning (ML), in enhancing graft-versus-host disease (GVHD) risk assessment, diagnosis, and personalized treatment. RECENT FINDINGS Recent studies have demonstrated the superiority of ML algorithms over traditional multivariate statistical models in donor selection for allogeneic hematopoietic stem cell transplantation. ML has recently enabled dynamic risk assessment by modeling time-series data, an upgrade from the static, "snapshot" assessment of patients that conventional statistical models and older ML algorithms offer. Regarding diagnosis, a deep learning model, a subset of ML, can accurately identify skin segments affected with chronic GVHD with satisfactory results. ML methods such as Q-learning and deep reinforcement learning have been utilized to develop adaptive treatment strategies (ATS) for the personalized prevention and treatment of acute and chronic GVHD. SUMMARY To capitalize on these promising advancements, there is a need for large-scale, multicenter collaborations to develop generalizable ML models. Furthermore, addressing pertinent issues such as the implementation of stringent ethical guidelines is crucial before the widespread introduction of AI into GVHD care.
Collapse
Affiliation(s)
- Ali Hassan Mushtaq
- Department of Internal Medicine, Cleveland Clinic Foundation, Cleveland, Ohio, USA
| | - Areez Shafqat
- College of Medicine, Alfaisal University, Riyadh, Saudi Arabia
| | - Haneen T Salah
- Department of Pathology and Genomic Medicine, Houston Methodist Hospital, Houston, Texas
| | - Shahrukh K Hashmi
- Division of Hematology, Department of Medicine, Mayo Clinic, Rochester, Minnesota, USA
- Department of Medicine, Sheikh Shakbout Medical City
- Medical Affairs, Khalifa University, Abu Dhabi, United Arab Emirates
| | - Ibrahim N Muhsen
- Section of Hematology and Oncology, Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| |
Collapse
|
18
|
Srivastava S, Rajan V. ExpertNet: A Deep Learning Approach to Combined Risk Modeling and Subtyping in Intensive Care Units. IEEE J Biomed Health Inform 2023; 27:5076-5086. [PMID: 37819834 DOI: 10.1109/jbhi.2023.3295751] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/13/2023]
Abstract
Risk models play a crucial role in disease prevention, particularly in intensive care units (ICUs). Diseases often have complex manifestations with heterogeneous subpopulations, or subtypes, that exhibit distinct clinical characteristics. Risk models that explicitly model subtypes have high predictive accuracy and facilitate subtype-specific personalization. Such models combine clustering and classification methods but do not effectively utilize the inferred subtypes in risk modeling. Their limitations include tendency to obtain degenerate clusters and cluster-specific data scarcity leading to insufficient training data for the corresponding classifier. In this article, we develop a new deep learning model for simultaneous clustering and classification, ExpertNet, with novel loss terms and network training strategies that address these limitations. The performance of ExpertNet is evaluated on the tasks of predicting risk of (i) sepsis and (ii) acute respiratory distress syndrome (ARDS), using two large electronic medical records datasets from ICUs. Our extensive experiments show that, in comparison to state-of-the-art baselines for combined clustering and classification, ExpertNet achieves superior accuracy in risk prediction for both ARDS and sepsis; and comparable clustering performance. Visual analysis of the clusters further demonstrates that the clusters obtained are clinically meaningful and a knowledge-distilled model shows significant differences in risk factors across the subtypes. By addressing technical challenges in training neural networks for simultaneous clustering and classification, ExpertNet lays the algorithmic foundation for the future development of subtype-aware risk models.
Collapse
|
19
|
Patton MJ, Liu VX. Predictive Modeling Using Artificial Intelligence and Machine Learning Algorithms on Electronic Health Record Data: Advantages and Challenges. Crit Care Clin 2023; 39:647-673. [PMID: 37704332 DOI: 10.1016/j.ccc.2023.02.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/15/2023]
Abstract
The rapid adoption of electronic health record (EHR) systems in US hospitals from 2008 to 2014 produced novel data elements for analysis. Concurrent innovations in computing architecture and machine learning (ML) algorithms have made rapid consumption of health data feasible and a powerful engine for clinical innovation. In critical care research, the net convergence of these trends has resulted in an exponential increase in outcome prediction research. In the following article, we explore the history of outcome prediction in the intensive care unit (ICU), the growing use of EHR data, and the rise of artificial intelligence and ML (AI) in critical care.
Collapse
Affiliation(s)
- Michael J Patton
- Medical Scientist Training Program, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA; Hugh Kaul Precision Medicine Institute at the University of Alabama at Birmingham, 720 20th Street South, Suite 202, Birmingham, Alabama, 35233, USA.
| | - Vincent X Liu
- Kaiser Permanente Division of Research, Oakland, CA, USA.
| |
Collapse
|
20
|
Islam KR, Prithula J, Kumar J, Tan TL, Reaz MBI, Sumon MSI, Chowdhury MEH. Machine Learning-Based Early Prediction of Sepsis Using Electronic Health Records: A Systematic Review. J Clin Med 2023; 12:5658. [PMID: 37685724 PMCID: PMC10488449 DOI: 10.3390/jcm12175658] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 08/13/2023] [Accepted: 08/23/2023] [Indexed: 09/10/2023] Open
Abstract
BACKGROUND Sepsis, a life-threatening infection-induced inflammatory condition, has significant global health impacts. Timely detection is crucial for improving patient outcomes as sepsis can rapidly progress to severe forms. The application of machine learning (ML) and deep learning (DL) to predict sepsis using electronic health records (EHRs) has gained considerable attention for timely intervention. METHODS PubMed, IEEE Xplore, Google Scholar, and Scopus were searched for relevant studies. All studies that used ML/DL to detect or early-predict the onset of sepsis in the adult population using EHRs were considered. Data were extracted and analyzed from all studies that met the criteria and were also evaluated for their quality. RESULTS This systematic review examined 1942 articles, selecting 42 studies while adhering to strict criteria. The chosen studies were predominantly retrospective (n = 38) and spanned diverse geographic settings, with a focus on the United States. Different datasets, sepsis definitions, and prevalence rates were employed, necessitating data augmentation. Heterogeneous parameter utilization, diverse model distribution, and varying quality assessments were observed. Longitudinal data enabled early sepsis prediction, and quality criteria fulfillment varied, with inconsistent funding-article quality correlation. CONCLUSIONS This systematic review underscores the significance of ML/DL methods for sepsis detection and early prediction through EHR data.
Collapse
Affiliation(s)
- Khandaker Reajul Islam
- Department of Physiology, Faculty of Medicine, University Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Johayra Prithula
- Department of Electrical and Electronics Engineering, University of Dhaka, Dhaka 1000, Bangladesh
| | - Jaya Kumar
- Department of Physiology, Faculty of Medicine, University Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Toh Leong Tan
- Department of Emergency Medicine, Faculty of Medicine, Universiti Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Mamun Bin Ibne Reaz
- Department of Electrical and Electronic Engineering, Independent University, Bangladesh Bashundhara, Dhaka 1229, Bangladesh
| | - Md. Shaheenur Islam Sumon
- Department of Biomedical Engineering, Military Institute of Science and Technology (MIST), Dhaka 1216, Bangladesh
| | | |
Collapse
|
21
|
Jones NW, Song SL, Thomasian N, Samuels EA, Ranney ML. Behavioral Health Decision Support Systems and User Interface Design in the Emergency Department. Appl Clin Inform 2023; 14:705-713. [PMID: 37673096 PMCID: PMC10482498 DOI: 10.1055/s-0043-1771395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Accepted: 06/06/2023] [Indexed: 09/08/2023] Open
Abstract
OBJECTIVE The objective of this qualitative study is to gauge physician sentiment about an emergency department (ED) clinical decision support (CDS) system implemented in multiple adult EDs within a university hospital system. This CDS system focuses on predicting patients' likelihood of ED recidivism and/or adverse opioid-related events. METHODS The study was conducted among adult emergency physicians working in three EDs of a single academic health system in Rhode Island. Qualitative, semistructured interviews were conducted with ED physicians. Interviews assessed physicians' prior experience with predictive analytics, thoughts on the alert's placement, design, and content, the alert's overall impact, and potential areas for improvement. Responses were aggregated and common themes identified. RESULTS Twenty-three interviews were conducted (11 preimplementation and 12 postimplementation). Themes were identified regarding each physician familiarity with predictive analytics, alert rollout, alert appearance and content, and on alert sentiments. Most physicians viewed these alerts as a neutral or positive EHR addition, with responses ranging from neutral to positive. The alert placement was noted to be largely intuitive and nonintrusive. The design of the alert was generally viewed positively. The alert's content was believed to be accurate, although the decision to respond to the alert's call-to-action was physician dependent. Those who tended to ignore the alert did so for a few reasons, including already knowing the information the alert contains, the alert offering information that is not relevant to this particular patient, and the alert not containing enough information to be useful. CONCLUSION Ultimately, this alert appears to have a marginally positive effect on ED physician workflow. At its most beneficial, the alert reminded physicians to deeply consider the care provided to high-risk populations and to potentially adjust their care and referrals. At its least beneficial, the alert did not affect physician decision-making but was not intrusive to the point of negatively impacting workflow.
Collapse
Affiliation(s)
- Nicholas W. Jones
- Department of Health Services, Policy, and Practice, Brown University School of Public Health, Providence, Rhode Island, United States
| | - Sophia L. Song
- Warren Alpert Medical School of Brown University, Providence, Rhode Island, United States
| | - Nicole Thomasian
- Department of Anesthesiology, New York Presbyterian-Weill Cornell Medical Center, New York, New York, United States
| | - Elizabeth A. Samuels
- Department of Emergency Medicine, Warren Alpert Medical School of Brown University, Providence, Rhode Island, United States
| | - Megan L. Ranney
- Department of Emergency Medicine, Warren Alpert Medical School of Brown University, Providence, Rhode Island, United States
| |
Collapse
|
22
|
Valik JK, Ward L, Tanushi H, Johansson AF, Färnert A, Mogensen ML, Pickering BW, Herasevich V, Dalianis H, Henriksson A, Nauclér P. Predicting sepsis onset using a machine learned causal probabilistic network algorithm based on electronic health records data. Sci Rep 2023; 13:11760. [PMID: 37474597 PMCID: PMC10359402 DOI: 10.1038/s41598-023-38858-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Accepted: 07/16/2023] [Indexed: 07/22/2023] Open
Abstract
Sepsis is a leading cause of mortality and early identification improves survival. With increasing digitalization of health care data automated sepsis prediction models hold promise to aid in prompt recognition. Most previous studies have focused on the intensive care unit (ICU) setting. Yet only a small proportion of sepsis develops in the ICU and there is an apparent clinical benefit to identify patients earlier in the disease trajectory. In this cohort of 82,852 hospital admissions and 8038 sepsis episodes classified according to the Sepsis-3 criteria, we demonstrate that a machine learned score can predict sepsis onset within 48 h using sparse routine electronic health record data outside the ICU. Our score was based on a causal probabilistic network model-SepsisFinder-which has similarities with clinical reasoning. A prediction was generated hourly on all admissions, providing a new variable was registered. Compared to the National Early Warning Score (NEWS2), which is an established method to identify sepsis, the SepsisFinder triggered earlier and had a higher area under receiver operating characteristic curve (AUROC) (0.950 vs. 0.872), as well as area under precision-recall curve (APR) (0.189 vs. 0.149). A machine learning comparator based on a gradient-boosting decision tree model had similar AUROC (0.949) and higher APR (0.239) than SepsisFinder but triggered later than both NEWS2 and SepsisFinder. The precision of SepsisFinder increased if screening was restricted to the earlier admission period and in episodes with bloodstream infection. Furthermore, the SepsisFinder signaled median 5.5 h prior to antibiotic administration. Identifying a high-risk population with this method could be used to tailor clinical interventions and improve patient care.
Collapse
Affiliation(s)
- John Karlsson Valik
- Division of Infectious Diseases, Department of Medicine, Karolinska Institutet, Solna, Stockholm, Sweden.
- Department of Infectious Diseases, Karolinska University Hospital, Stockholm, Sweden.
| | - Logan Ward
- Treat Systems ApS, Aalborg, Denmark
- Department of Health Science and Technology, Center for Model-Based Medical Decision Support, Aalborg University, Aalborg, Denmark
| | - Hideyuki Tanushi
- Division of Infectious Diseases, Department of Medicine, Karolinska Institutet, Solna, Stockholm, Sweden
| | - Anders F Johansson
- Department of Clinical Microbiology and the Laboratory for Molecular Infection Medicine (MIMS), Umeå University, Umeå, Sweden
| | - Anna Färnert
- Division of Infectious Diseases, Department of Medicine, Karolinska Institutet, Solna, Stockholm, Sweden
- Department of Infectious Diseases, Karolinska University Hospital, Stockholm, Sweden
| | | | - Brian W Pickering
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN, USA
| | - Vitaly Herasevich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN, USA
| | - Hercules Dalianis
- Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden
| | - Aron Henriksson
- Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden
| | - Pontus Nauclér
- Division of Infectious Diseases, Department of Medicine, Karolinska Institutet, Solna, Stockholm, Sweden
- Department of Infectious Diseases, Karolinska University Hospital, Stockholm, Sweden
| |
Collapse
|
23
|
Seth P, Hueppchen N, Miller SD, Rudzicz F, Ding J, Parakh K, Record JD. Data Science as a Core Competency in Undergraduate Medical Education in the Age of Artificial Intelligence in Health Care. JMIR MEDICAL EDUCATION 2023; 9:e46344. [PMID: 37432728 PMCID: PMC10369309 DOI: 10.2196/46344] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 05/07/2023] [Accepted: 06/26/2023] [Indexed: 07/12/2023]
Abstract
The increasingly sophisticated and rapidly evolving application of artificial intelligence in medicine is transforming how health care is delivered, highlighting a need for current and future physicians to develop basic competency in the data science that underlies this topic. Medical educators must consider how to incorporate central concepts in data science into their core curricula to train physicians of the future. Similar to how the advent of diagnostic imaging required the physician to understand, interpret, and explain the relevant results to patients, physicians of the future should be able to explain to patients the benefits and limitations of management plans guided by artificial intelligence. We outline major content domains and associated learning outcomes in data science applicable to medical student curricula, suggest ways to incorporate these themes into existing curricula, and note potential implementation barriers and solutions to optimize the integration of this content.
Collapse
Affiliation(s)
- Puneet Seth
- Department of Family Medicine, McMaster University, Hamilton, ON, Canada
| | - Nancy Hueppchen
- Department of Gynecology and Obstetrics, Johns Hopkins University School of Medicine, Baltimore, MD, United States
| | - Steven D Miller
- Division of Pediatric Gastroenterology, Hepatology, and Nutrition, Department of Pediatrics, Johns Hopkins University School of Medicine, Baltimore, MD, United States
| | - Frank Rudzicz
- Faculty of Computer Science, Dalhousie University, Halifax, NS, Canada
- Vector Institute for Artificial Intelligence, Toronto, ON, Canada
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
| | - Jerry Ding
- Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Kapil Parakh
- Department of Medicine, Georgetown University, Washington, DC, United States
| | - Janet D Record
- Department of Gynecology and Obstetrics, Johns Hopkins University School of Medicine, Baltimore, MD, United States
| |
Collapse
|
24
|
O'Sullivan C, Tsai DHT, Wu ICY, Boselli E, Hughes C, Padmanabhan D, Hsia Y. Machine learning applications on neonatal sepsis treatment: a scoping review. BMC Infect Dis 2023; 23:441. [PMID: 37386442 DOI: 10.1186/s12879-023-08409-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Accepted: 06/20/2023] [Indexed: 07/01/2023] Open
Abstract
INTRODUCTION Neonatal sepsis is a major cause of health loss and mortality worldwide. Without proper treatment, neonatal sepsis can quickly develop into multisystem organ failure. However, the signs of neonatal sepsis are non-specific, and treatment is labour-intensive and expensive. Moreover, antimicrobial resistance is a significant threat globally, and it has been reported that over 70% of neonatal bloodstream infections are resistant to first-line antibiotic treatment. Machine learning is a potential tool to aid clinicians in diagnosing infections and in determining the most appropriate empiric antibiotic treatment, as has been demonstrated for adult populations. This review aimed to present the application of machine learning on neonatal sepsis treatment. METHODS PubMed, Embase, and Scopus were searched for studies published in English focusing on neonatal sepsis, antibiotics, and machine learning. RESULTS There were 18 studies included in this scoping review. Three studies focused on using machine learning in antibiotic treatment for bloodstream infections, one focused on predicting in-hospital mortality associated with neonatal sepsis, and the remaining studies focused on developing machine learning prediction models to diagnose possible sepsis cases. Gestational age, C-reactive protein levels, and white blood cell count were important predictors to diagnose neonatal sepsis. Age, weight, and days from hospital admission to blood sample taken were important to predict antibiotic-resistant infections. The best-performing machine learning models were random forest and neural networks. CONCLUSION Despite the threat antimicrobial resistance poses, there was a lack of studies focusing on the use of machine learning for aiding empirical antibiotic treatment for neonatal sepsis.
Collapse
Affiliation(s)
| | - Daniel Hsiang-Te Tsai
- Centre for Neonatal and Paediatric Infection, St. George's, University of London, London, UK
- School of Pharmacy, Institute of Clinical Pharmacy and Pharmaceutical Sciences, College of Medicine, National Cheng Kung University, Tainan, Taiwan
| | - Ian Chang-Yen Wu
- Centre for Neonatal and Paediatric Infection, St. George's, University of London, London, UK
- School of Pharmacy, Institute of Clinical Pharmacy and Pharmaceutical Sciences, College of Medicine, National Cheng Kung University, Tainan, Taiwan
- Department of Pharmacy, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, Tainan, Taiwan
| | - Emanuela Boselli
- Department of Pediatrics, V. Buzzi Children's Hospital, University of Milan, Milan, Italy
| | - Carmel Hughes
- School of Pharmacy, Queen's University Belfast, Belfast, UK
| | - Deepak Padmanabhan
- School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, Belfast, UK
| | - Yingfen Hsia
- School of Pharmacy, Queen's University Belfast, Belfast, UK
- Centre for Neonatal and Paediatric Infection, St. George's, University of London, London, UK
| |
Collapse
|
25
|
Li Y, Wang Y. Temporal convolution attention model for sepsis clinical assistant diagnosis prediction. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:13356-13378. [PMID: 37501491 DOI: 10.3934/mbe.2023595] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/29/2023]
Abstract
Sepsis is an organ failure disease caused by an infection acquired in an intensive care unit (ICU), which leads to a high mortality rate. Developing intelligent monitoring and early warning systems for sepsis is a key research area in the field of smart healthcare. Early and accurate identification of patients at high risk of sepsis can help doctors make the best clinical decisions and reduce the mortality rate of patients with sepsis. However, the scientific understanding of sepsis remains inadequate, leading to slow progress in sepsis research. With the accumulation of electronic medical records (EMRs) in hospitals, data mining technologies that can identify patient risk patterns from the vast amount of sepsis-related EMRs and the development of smart surveillance and early warning models show promise in reducing mortality. Based on the Medical Information Mart for Intensive Care Ⅲ, a massive dataset of ICU EMRs published by MIT and Beth Israel Deaconess Medical Center, we propose a Temporal Convolution Attention Model for Sepsis Clinical Assistant Diagnosis Prediction (TCASP) to predict the incidence of sepsis infection in ICU patients. First, sepsis patient data is extracted from the EMRs. Then, the incidence of sepsis is predicted based on various physiological features of sepsis patients in the ICU. Finally, the TCASP model is utilized to predict the time of the first sepsis infection in ICU patients. The experiments show that the proposed model achieves an area under the receiver operating characteristic curve (AUROC) score of 86.9% (an improvement of 6.4% ) and an area under the precision-recall curve (AUPRC) score of 63.9% (an improvement of 3.9% ) compared to five state-of-the-art models.
Collapse
Affiliation(s)
- Yong Li
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| | - Yang Wang
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| |
Collapse
|
26
|
Yazdani A, Bigdeli SK, Zahmatkeshan M. Investigating the performance of machine learning algorithms in predicting the survival of COVID-19 patients: A cross section study of Iran. Health Sci Rep 2023; 6:e1212. [PMID: 37064314 PMCID: PMC10099201 DOI: 10.1002/hsr2.1212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2022] [Revised: 03/23/2023] [Accepted: 03/30/2023] [Indexed: 04/18/2023] Open
Abstract
Background and Aims Like early diagnosis, predicting the survival of patients with Coronavirus Disease 2019 (COVID-19) is of great importance. Survival prediction models help doctors be more cautious to treat the patients who are at high risk of dying because of medical conditions. This study aims to predict the survival of hospitalized patients with COVID-19 by comparing the accuracy of machine learning (ML) models. Methods It is a cross-sectional study which was performed in 2022 in Fasa city in Iran country. The research data set was extracted from the period February 18, 2020 to February 10, 2021, and contains 2442 hospitalized patients' records with 84 features. A comparison was made between the efficiency of five ML algorithms to predict survival, includes Naive Bayes (NB), K-nearest neighbors (KNN), random forest (RF), decision tree (DT), and multilayer perceptron (MLP). Modeling steps were done with Python language in the Anaconda Navigator 3 environment. Results Our findings show that NB algorithm had better performance than others with accuracy, precision, recall, F-score, and area under receiver operating characteristic curve of 97%, 96%, 96%, 96%, and 97%, respectively. Based on the analysis of factors affecting survival, heart disease, pulmonary diseases and blood related disease were the most important disease related to death. Conclusion The development of software systems based on NB will be effective to predict the survival of COVID-19 patients.
Collapse
Affiliation(s)
- Azita Yazdani
- Department of Health Information Management, School of Health Management and Information SciencesShiraz University of Medical SciencesShirazIran
- Clinical Education Research CenterShiraz University of Medical SciencesShirazIran
- Health Human Resources Research Center, School of Health Management and Information SciencesShiraz University of Medical SciencesShirazIran
| | - Somayeh Kianian Bigdeli
- Health Information Management Department, School of Allied Medical SciencesTehran University of Medical SciencesTehranIran
| | - Maryam Zahmatkeshan
- Noncommunicable Diseases Research CenterFasa University of Medical SciencesFasaIran
- School of Allied Medical SciencesFasa University of Medical SciencesFasaIran
| |
Collapse
|
27
|
Ouyang Y, Cheng M, He B, Zhang F, Ouyang W, Zhao J, Qu Y. Interpretable machine learning models for predicting in-hospital death in patients in the intensive care unit with cerebral infarction. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 231:107431. [PMID: 36827826 DOI: 10.1016/j.cmpb.2023.107431] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/26/2022] [Revised: 07/20/2022] [Accepted: 02/16/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND Research on patients with cerebral infarction in the Intensive Care Unit (ICU) is still lacking. Our study aims to develop and validate multiple machine-learning (ML) models using two large ICU databases-Medical Information Mart for Intensive Care version III (MIMIC-III) and eICU Research Institute Database (eRI)-to guide clinical practice. METHODS We collected clinical data from patients with cerebral infarction in the MIMIC-III and eRI databases within 24 h of admission. The opinion of neurologists and the Least Absolute Shrinkage and Selection Operator regression was used to screen for relevant clinical features. Using eRI as the training set and MIMIC-III as the test set, we developed and validated six ML models. Based on the results of the model validation, we select the best model and perform the interpretability analysis on it. RESULTS A total of 4,338 patients were included in the study (eRI:3002, MIMIC-III:1336), resulting in a total of 18 clinical characteristics through screening. Model validation results showed that random forest (RF) was the best model, with AUC and F1 scores of 0.799 and 0.417 in internal validation and 0.733 and 0.498 in external validation, respectively; moreover, its sensitivity and recall were the highest of the six algorithms for both the internal and external validation. The explanatory analysis of the model showed that the three most important variables in the RF model were Acute Physiology Score-III, Glasgow Coma Scale score, and heart rate, and that the influence of each variable on the judgement of the model was consistent with medical knowledge. CONCLUSION Based on a large sample of patients and advanced algorithms, our study bridges the limitations of studies on this area. With our model, physicians can use the admission information of cerebral infarction patients in the ICU to identify high-risk groups among them who are prone to in-hospital death, so that they could be more alert to this group of patients and upgrade medical measures early to minimize the mortality of patients.
Collapse
Affiliation(s)
- Yang Ouyang
- Department of Orthopedics, The Second Hospital of Jilin University, 218 Ziqiang Street, Changchun 130041, China
| | - Meng Cheng
- Department of Orthopedics, The Second Hospital of Jilin University, 218 Ziqiang Street, Changchun 130041, China
| | - Bingqing He
- Department of Neurology, The Second Hospital of Jilin University, 218 Ziqiang Street, Changchun 130041, China
| | - Fengjuan Zhang
- Department of Neurology, The Second Hospital of Jilin University, 218 Ziqiang Street, Changchun 130041, China
| | - Wen Ouyang
- Department of Endocrinology, First People's Hospital of Changde City, 818 renmin Street, Changde 415000, China
| | - Jianwu Zhao
- Department of Orthopedics, The Second Hospital of Jilin University, 218 Ziqiang Street, Changchun 130041, China.
| | - Yang Qu
- Department of Orthopedics, The Second Hospital of Jilin University, 218 Ziqiang Street, Changchun 130041, China.
| |
Collapse
|
28
|
Strickler EAT, Thomas J, Thomas JP, Benjamin B, Shamsuddin R. Exploring a global interpretation mechanism for deep learning networks when predicting sepsis. Sci Rep 2023; 13:3067. [PMID: 36810645 PMCID: PMC9945464 DOI: 10.1038/s41598-023-30091-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Accepted: 02/15/2023] [Indexed: 02/24/2023] Open
Abstract
The purpose of this study is to identify additional clinical features for sepsis detection through the use of a novel mechanism for interpreting black-box machine learning models trained and to provide a suitable evaluation for the mechanism. We use the publicly available dataset from the 2019 PhysioNet Challenge. It has around 40,000 Intensive Care Unit (ICU) patients with 40 physiological variables. Using Long Short-Term Memory (LSTM) as the representative black-box machine learning model, we adapted the Multi-set Classifier to globally interpret the black-box model for concepts it learned about sepsis. To identify relevant features, the result is compared against: (i) features used by a computational sepsis expert, (ii) clinical features from clinical collaborators, (iii) academic features from literature, and (iv) significant features from statistical hypothesis testing. Random Forest was found to be the computational sepsis expert because it had high accuracies for solving both the detection and early detection, and a high degree of overlap with clinical and literature features. Using the proposed interpretation mechanism and the dataset, we identified 17 features that the LSTM used for sepsis classification, 11 of which overlaps with the top 20 features from the Random Forest model, 10 with academic features and 5 with clinical features. Clinical opinion suggests, 3 LSTM features have strong correlation with some clinical features that were not identified by the mechanism. We also found that age, chloride ion concentration, pH and oxygen saturation should be investigated further for connection with developing sepsis. Interpretation mechanisms can bolster the incorporation of state-of-the-art machine learning models into clinical decision support systems, and might help clinicians to address the issue of early sepsis detection. The promising results from this study warrants further investigation into creation of new and improvement of existing interpretation mechanisms for black-box models, and into clinical features that are currently not used in clinical assessment of sepsis.
Collapse
Affiliation(s)
- Ethan A T Strickler
- Physics and Mathematics, East Central University, PO Box 385, Ada, OK, 74820, USA
| | - Joshua Thomas
- Department of Internal Medicine, Rush University Medical Center, 1700 W Van Buren St, 5th Floor, Chicago, IL, 60612, USA
| | - Johnson P Thomas
- Oklahoma State University, 201 Math and Science Building, Stillwater, OK, 74078, USA
| | - Bruce Benjamin
- School of Biomedical Sciences, Center for Health Sciences, 1111 W. 17th st., Tulsa, OK, 74107, USA
| | - Rittika Shamsuddin
- Oklahoma State University, 212 Math and Science Building, Stillwater, OK, 74078, USA.
| |
Collapse
|
29
|
Tang M, Mu F, Cui C, Zhao JY, Lin R, Sun KX, Guan Y, Wang JW. Research frontiers and trends in the application of artificial intelligence to sepsis: A bibliometric analysis. Front Med (Lausanne) 2023; 9:1043589. [PMID: 36714139 PMCID: PMC9878129 DOI: 10.3389/fmed.2022.1043589] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Accepted: 12/23/2022] [Indexed: 01/14/2023] Open
Abstract
Background With the increasing interest of academics in the application of artificial intelligence to sepsis, thousands of papers on this field had been published in the past few decades. It is difficult for researchers to understand the themes and latest research frontiers in this field from a multi-dimensional perspective. Consequently, the purpose of this study is to analyze the relevant literature in the application of artificial intelligence to sepsis through bibliometrics software, so as to better understand the development status, study the core hotspots and future development trends of this field. Methods We collected relevant publications in the application of artificial intelligence to sepsis from the Web of Science Core Collection in 2000 to 2021. The type of publication was limited to articles and reviews, and language was limited to English. Research cooperation network, journals, cited references, keywords in this field were visually analyzed by using CiteSpace, VOSviewer, and COOC software. Results A total of 8,481 publications in the application of artificial intelligence to sepsis between 2000 and 2021 were included, involving 8,132 articles and 349 reviews. Over the past 22 years, the annual number of publications had gradually increased exponentially. The USA was the most productive country, followed by China. Harvard University, Schuetz, Philipp, and Intensive Care Medicine were the most productive institution, author, and journal, respectively. Vincent, Jl and Critical Care Medicine were the most cited author and cited journal, respectively. Several conclusions can be drawn from the analysis of the cited references, including the following: screening and identification of sepsis biomarkers, treatment and related complications of sepsis, and precise treatment of sepsis. Moreover, there were a spike in searches relating to machine learning, antibiotic resistance and accuracy based on burst detection analysis. Conclusion This study conducted a comprehensive and objective analysis of the publications on the application of artificial intelligence in sepsis. It can be predicted that precise treatment of sepsis through machine learning technology is still research hotspot in this field.
Collapse
|
30
|
Mainali S, Park S. Artificial Intelligence and Big Data Science in Neurocritical Care. Crit Care Clin 2023; 39:235-242. [DOI: 10.1016/j.ccc.2022.07.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
31
|
Chen Q, Li R, Lin C, Lai C, Huang Y, Lu W, Li L. SEPRES: Intensive Care Unit Clinical Data Integration System to Predict Sepsis. Appl Clin Inform 2023; 14:65-75. [PMID: 36452980 PMCID: PMC9876660 DOI: 10.1055/a-1990-3037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Accepted: 11/28/2022] [Indexed: 12/05/2022] Open
Abstract
BACKGROUND The lack of information interoperability between different devices and systems in the intensive care unit (ICU) hinders further utilization of data, especially for early warning of specific diseases in the ICU. OBJECTIVES We aimed to establish a data integration system. Based on this system, the sepsis prediction module was added to compose the Sepsis PREdiction System (SEPRES), where real-time early warning of sepsis can be implemented at the bedside in the ICU. METHODS Data are collected from bedside devices through the integration hub and uploaded to the integration system through the local area network. The data integration system was designed to integrate vital signs data, laboratory data, ventilator data, demographic data, pharmacy data, nursing data, etc. from multiple medical devices and systems. It integrates, standardizes, and stores information, making the real-time inference of the early warning module possible. The built-in sepsis early warning module can detect the onset of sepsis within 5 hours preceding at most. RESULTS Our data integration system has already been deployed in Ruijin Hospital, confirming the feasibility of our system. CONCLUSION We highlight that SEPRES has the potential to improve ICU management by helping medical practitioners identify at-sepsis-risk patients and prepare for timely diagnosis and intervention.
Collapse
Affiliation(s)
- Qiyu Chen
- Division of Applied Mathematics, Fudan University, Shanghai, China
| | - Ranran Li
- Department of Critical Care Medicine, Shanghai Jiaotong University School of Medicine, Ruijin Hospital, Shanghai, China
| | - ChihChe Lin
- Department of Intelligent Medical Products, Shanghai Electric Group Co., Ltd. Central Academe, Shanghai, China
| | - Chiming Lai
- Department of Intelligent Medical Products, Shanghai Electric Group Co., Ltd. Central Academe, Shanghai, China
| | - Yaling Huang
- Department of Intelligent Medical Products, Shanghai Electric Group Co., Ltd. Central Academe, Shanghai, China
| | - Wenlian Lu
- Division of Applied Mathematics, Fudan University, Shanghai, China
| | - Lei Li
- Department of Critical Care Medicine, Shanghai Jiaotong University School of Medicine, Ruijin Hospital, Shanghai, China
| |
Collapse
|
32
|
Chen Q, Li R, Lin C, Lai C, Chen D, Qu H, Huang Y, Lu W, Tang Y, Li L. Transferability and interpretability of the sepsis prediction models in the intensive care unit. BMC Med Inform Decis Mak 2022; 22:343. [PMID: 36581881 PMCID: PMC9798724 DOI: 10.1186/s12911-022-02090-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2022] [Accepted: 12/16/2022] [Indexed: 12/31/2022] Open
Abstract
BACKGROUND We aimed to develop an early warning system for real-time sepsis prediction in the ICU by machine learning methods, with tools for interpretative analysis of the predictions. In particular, we focus on the deployment of the system in a target medical center with small historical samples. METHODS Light Gradient Boosting Machine (LightGBM) and multilayer perceptron (MLP) were trained on Medical Information Mart for Intensive Care (MIMIC-III) dataset and then finetuned on the private Historical Database of local Ruijin Hospital (HDRJH) using transfer learning technique. The Shapley Additive Explanations (SHAP) analysis was employed to characterize the feature importance in the prediction inference. Ultimately, the performance of the sepsis prediction system was further evaluated in the real-world study in the ICU of the target Ruijin Hospital. RESULTS The datasets comprised 6891 patients from MIMIC-III, 453 from HDRJH, and 67 from Ruijin real-world data. The area under the receiver operating characteristic curves (AUCs) for LightGBM and MLP models derived from MIMIC-III were 0.98 - 0.98 and 0.95 - 0.96 respectively on MIMIC-III dataset, and, in comparison, 0.82 - 0.86 and 0.84 - 0.87 respectively on HDRJH, from 1 to 5 h preceding. After transfer learning and ensemble learning, the AUCs of the final ensemble model were enhanced to 0.94 - 0.94 on HDRJH and to 0.86 - 0.9 in the real-world study in the ICU of the target Ruijin Hospital. In addition, the SHAP analysis illustrated the importance of age, antibiotics, net balance, and ventilation for sepsis prediction, making the model interpretable. CONCLUSIONS Our machine learning model allows accurate real-time prediction of sepsis within 5-h preceding. Transfer learning can effectively improve the feasibility to deploy the prediction model in the target cohort, and ameliorate the model performance for external validation. SHAP analysis indicates that the role of antibiotic usage and fluid management needs further investigation. We argue that our system and methodology have the potential to improve ICU management by helping medical practitioners identify at-sepsis-risk patients and prepare for timely diagnosis and intervention. TRIAL REGISTRATION NCT05088850 (retrospectively registered).
Collapse
Affiliation(s)
- Qiyu Chen
- grid.8547.e0000 0001 0125 2443Department of Applied Mathematics, School of Mathematical Sciences, Fudan University, Shanghai, 200433 China
| | - Ranran Li
- grid.16821.3c0000 0004 0368 8293Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025 China
| | - ChihChe Lin
- grid.495525.a0000 0004 0552 4356Shanghai Electric Group Co., Ltd., Central Academe, Shanghai, China
| | - Chiming Lai
- grid.495525.a0000 0004 0552 4356Shanghai Electric Group Co., Ltd., Central Academe, Shanghai, China
| | - Dechang Chen
- grid.16821.3c0000 0004 0368 8293Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025 China
| | - Hongping Qu
- grid.16821.3c0000 0004 0368 8293Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025 China
| | - Yaling Huang
- grid.495525.a0000 0004 0552 4356Shanghai Electric Group Co., Ltd., Central Academe, Shanghai, China
| | - Wenlian Lu
- grid.8547.e0000 0001 0125 2443Department of Applied Mathematics, School of Mathematical Sciences, Fudan University, Shanghai, 200433 China
| | - Yaoqing Tang
- grid.16821.3c0000 0004 0368 8293Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025 China
| | - Lei Li
- grid.16821.3c0000 0004 0368 8293Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025 China
| |
Collapse
|
33
|
DeShon B, Dummitt B, Allen J, Yount B. Prediction of sepsis onset in hospital admissions using survival analysis. J Clin Monit Comput 2022; 36:1611-1619. [PMID: 35076834 DOI: 10.1007/s10877-022-00804-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Accepted: 01/03/2022] [Indexed: 11/25/2022]
Abstract
To determine the efficacy of modern survival analysis methods for predicting sepsis onset in ICU, emergency, medical/surgical, and TCU departments. We performed a retrospective analysis on ICU, med/surg, ED, and TCU cases from multiple Mercy Health hospitals from August 2018 to March 2020. Patients in these departments were monitored by the Mercy Virtual vSepsis team and sepsis cases were determined and documented in the Mercy EHR via a rule-based engine utilizing clinical data. We used survival-based modeling methods to predict sepsis onset in these cases. The three survival methods that were used to predict the onset of severe sepsis and septic shock produced AUC values > 0.85 and each provided a median lead time of > 20 h prior to disease onset. This methodology improves upon previous work by demonstrating excellent model performance when generalizing survival-based prediction methods to both severe sepsis and septic shock as well as non-ICU departments.IRB InformationTrial Registration ID: 1,532,327-1.Trial Effective Date: 12/02/2019.
Collapse
Affiliation(s)
- Brandon DeShon
- Mercy Health, 14528 S. Outer Forty, Chesterfield, MO, 63017, USA.
| | - Benjamin Dummitt
- Mercy Health, 14528 S. Outer Forty, Chesterfield, MO, 63017, USA
| | - Joshua Allen
- Mercy Health, 14528 S. Outer Forty, Chesterfield, MO, 63017, USA
| | - Byron Yount
- Mercy Health, 14528 S. Outer Forty, Chesterfield, MO, 63017, USA
| |
Collapse
|
34
|
Zhang A, Xing L, Zou J, Wu JC. Shifting machine learning for healthcare from development to deployment and from models to data. Nat Biomed Eng 2022; 6:1330-1345. [PMID: 35788685 DOI: 10.1038/s41551-022-00898-y] [Citation(s) in RCA: 58] [Impact Index Per Article: 29.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2021] [Accepted: 05/03/2022] [Indexed: 01/14/2023]
Abstract
In the past decade, the application of machine learning (ML) to healthcare has helped drive the automation of physician tasks as well as enhancements in clinical capabilities and access to care. This progress has emphasized that, from model development to model deployment, data play central roles. In this Review, we provide a data-centric view of the innovations and challenges that are defining ML for healthcare. We discuss deep generative models and federated learning as strategies to augment datasets for improved model performance, as well as the use of the more recent transformer models for handling larger datasets and enhancing the modelling of clinical text. We also discuss data-focused problems in the deployment of ML, emphasizing the need to efficiently deliver data to ML models for timely clinical predictions and to account for natural data shifts that can deteriorate model performance.
Collapse
Affiliation(s)
- Angela Zhang
- Stanford Cardiovascular Institute, School of Medicine, Stanford University, Stanford, CA, USA. .,Department of Genetics, School of Medicine, Stanford University, Stanford, CA, USA. .,Greenstone Biosciences, Palo Alto, CA, USA. .,Department of Computer Science, Stanford University, Stanford, CA, USA.
| | - Lei Xing
- Department of Radiation Oncology, School of Medicine, Stanford University, Stanford, CA, USA
| | - James Zou
- Department of Computer Science, Stanford University, Stanford, CA, USA.,Department of Biomedical Informatics, School of Medicine, Stanford University, Stanford, CA, USA
| | - Joseph C Wu
- Stanford Cardiovascular Institute, School of Medicine, Stanford University, Stanford, CA, USA. .,Greenstone Biosciences, Palo Alto, CA, USA. .,Departments of Medicine, Division of Cardiovascular Medicine Stanford University, Stanford, CA, USA. .,Department of Radiology, School of Medicine, Stanford University, Stanford, CA, USA.
| |
Collapse
|
35
|
Mehl SC, Portuondo JI, Pettit RW, Fallon SC, Wesson DE, Shah SR, Vogel AM, Lopez ME, Massarweh NN. Association of prematurity with complications and failure to rescue in neonatal surgery. J Pediatr Surg 2022; 57:268-276. [PMID: 34857374 PMCID: PMC9125744 DOI: 10.1016/j.jpedsurg.2021.10.050] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/22/2021] [Revised: 10/15/2021] [Accepted: 10/25/2021] [Indexed: 10/19/2022]
Abstract
BACKGROUND The majority of failure to rescue (FTR), or death after a postoperative complication, in pediatric surgery occurs among infants and neonates. The purpose of this study is to evaluate the association between gestational age (GA) and FTR in infants and neonates. METHODS National cohort study of 46,452 patients < 1 year old within the National Surgical Quality Improvement Program-Pediatric database who underwent inpatient surgery. Patients were categorized as preterm neonates, term neonates, or infants. Neonates were stratified based on GA. Surgical procedures were classified as low- (< 1% mortality) or high-risk (≥ 1%). Multivariable logistic regression and cubic splines were used to evaluate the association between GA and FTR. RESULTS Preterm neonates had the highest FTR (28%) rates. Among neonates, FTR increased with decreasing GA (≥ 37 weeks, 12%; 33-36 weeks, 15%; 29-32 weeks, 30%; 25-28 weeks 41%; ≤ 24 weeks, 57%). For both low- and high-risk procedures, FTR significantly (trend test, p < 0.01) increased with decreasing GA. When stratifying preterm neonates by GA, all GAs ≤ 28 weeks were associated with significantly higher odds of FTR for low- (OR 2.47, 95% CI [1.38-4.41]) and high-risk (OR 2.27, 95% CI [1.33-3.87]) procedures. A lone inflection point for FTR was identified at 31-32 weeks with cubic spline analysis. CONCLUSIONS The dose-dependent relationship between decreasing GA and FTR as well as the FTR inflection point noted at GA 31-32 weeks can be used by stakeholders in designing quality improvement initiatives and directing perioperative care. LEVEL OF EVIDENCE Level IV, Retrospective cohort study.
Collapse
Affiliation(s)
- Steven C. Mehl
- Michael E. DeBakey Department of Surgery, Baylor College of Medicine, 1 Baylor Plaza, MS390, Houston, TX 77030, United States,Department of Surgery, Division of Pediatric Surgery, Texas Children’s Hospital, Houston, TX, United States,Corresponding author at: Michael E. DeBakey Department of Surgery, Baylor College of Medicine, 1 Baylor Plaza, MS390, Houston, TX 77030, United States. (S.C. Mehl)
| | - Jorge I. Portuondo
- Michael E. DeBakey Department of Surgery, Baylor College of Medicine, 1 Baylor Plaza, MS390, Houston, TX 77030, United States
| | - Rowland W. Pettit
- Michael E. DeBakey Department of Surgery, Baylor College of Medicine, 1 Baylor Plaza, MS390, Houston, TX 77030, United States
| | - Sara C. Fallon
- Michael E. DeBakey Department of Surgery, Baylor College of Medicine, 1 Baylor Plaza, MS390, Houston, TX 77030, United States,Department of Surgery, Division of Pediatric Surgery, Texas Children’s Hospital, Houston, TX, United States
| | - David E. Wesson
- Michael E. DeBakey Department of Surgery, Baylor College of Medicine, 1 Baylor Plaza, MS390, Houston, TX 77030, United States,Department of Surgery, Division of Pediatric Surgery, Texas Children’s Hospital, Houston, TX, United States
| | - Sohail R. Shah
- Michael E. DeBakey Department of Surgery, Baylor College of Medicine, 1 Baylor Plaza, MS390, Houston, TX 77030, United States,Department of Surgery, Division of Pediatric Surgery, Texas Children’s Hospital, Houston, TX, United States
| | - Adam M. Vogel
- Michael E. DeBakey Department of Surgery, Baylor College of Medicine, 1 Baylor Plaza, MS390, Houston, TX 77030, United States,Department of Surgery, Division of Pediatric Surgery, Texas Children’s Hospital, Houston, TX, United States
| | - Monica E. Lopez
- Michael E. DeBakey Department of Surgery, Baylor College of Medicine, 1 Baylor Plaza, MS390, Houston, TX 77030, United States,Department of Surgery, Division of Pediatric Surgery, Texas Children’s Hospital, Houston, TX, United States
| | - Nader N. Massarweh
- Atlanta VA Health Care System, Decatur, GA, United States,Department of Surgery, Division of Surgical Oncology, Emory University School of Medicine, Atlanta, GA, United States,Department of Surgery, Morehouse School of Medicine, Atlanta, GA, United States
| |
Collapse
|
36
|
Machine learning and artificial intelligence: applications in healthcare epidemiology. ANTIMICROBIAL STEWARDSHIP & HEALTHCARE EPIDEMIOLOGY : ASHE 2022; 1:e28. [PMID: 36168500 PMCID: PMC9495400 DOI: 10.1017/ash.2021.192] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Accepted: 08/11/2021] [Indexed: 12/21/2022]
Abstract
Artificial intelligence (AI) refers to the performance of tasks by machines ordinarily associated with human intelligence. Machine learning (ML) is a subtype of AI; it refers to the ability of computers to draw conclusions (ie, learn) from data without being directly programmed. ML builds from traditional statistical methods and has drawn significant interest in healthcare epidemiology due to its potential for improving disease prediction and patient care. This review provides an overview of ML in healthcare epidemiology and practical examples of ML tools used to support healthcare decision making at 4 stages of hospital-based care: triage, diagnosis, treatment, and discharge. Examples include model-building efforts to assist emergency department triage, predicting time before septic shock onset, detecting community-acquired pneumonia, and classifying COVID-19 disposition risk level. Increasing availability and quality of electronic health record (EHR) data as well as computing power provides opportunities for ML to increase patient safety, improve the efficiency of clinical management, and reduce healthcare costs.
Collapse
|
37
|
Xu H, Li T, Zhang X, Li H, Lv D, Wang Y, Huo F, Bai J, Wang C. Impaired Circulating Antibody-Secreting Cells Generation Predicts the Dismal Outcome in the Elderly Septic Shock Patients. J Inflamm Res 2022; 15:5293-5308. [PMID: 36124208 PMCID: PMC9482413 DOI: 10.2147/jir.s376962] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Accepted: 08/13/2022] [Indexed: 11/23/2022] Open
Abstract
Purpose Sepsis is a condition that derives from a dysregulated host response to infection. Although B lymphocytes play a pivotal role in immune response, little is known about status of their terminally differentiated cells, antibody-secreting cells (ASCs) during immunosuppressive phase of sepsis, especially in elderly patients. Our aim was to extensively characterize the immune functions of ASCs in elderly septic patients. Patients and Methods Clinical and laboratory data were collected on days 1, 3, and 7 of hospitalization. Circulating ASCs were evaluated by flow cytometry from fresh whole blood in elderly septic patients at the onset of disease. RNA sequencing analyzed ASCs gene expression profile. Receiver operating characteristic (ROC) curve analysis and logistic regression predicted the survival rate of 28-day mortality. Results A total of 103 septic patients were enrolled. The number and proportion of ASCs among total lymphocytes dramatically increased in septic patients, and RNA sequencing analysis showed that ASCs from septic patients exhibited a different gene expression profile. Furthermore, we found these ASCs could promote the function of T cells. Logistic regression analysis showed ASCs population was an independent outcome predictor in septic shock patients. Conclusion Our study revealed the complex nature of immune disorders in sepsis and identified circulating ASCs population as a useful biomarker for predicting mortality in elderly septic patients, which provided a novel clue to combat this severe disease.
Collapse
Affiliation(s)
- Huihui Xu
- Key Laboratory of Molecular Virology and Immunology, Institut Pasteur of Shanghai, Chinese Academy of Sciences, Shanghai, 200031, People's Republic of China.,University of Chinese Academy of Sciences, Beijing, 100000, People's Republic of China
| | - Teng Li
- Key Laboratory of Molecular Virology and Immunology, Institut Pasteur of Shanghai, Chinese Academy of Sciences, Shanghai, 200031, People's Republic of China.,University of Chinese Academy of Sciences, Beijing, 100000, People's Republic of China
| | - Xiaoming Zhang
- Key Laboratory of Molecular Virology and Immunology, Institut Pasteur of Shanghai, Chinese Academy of Sciences, Shanghai, 200031, People's Republic of China.,Shanghai Huashen Institute of Microbes and Infections, Shanghai, 200052, People's Republic of China
| | - Hongqiang Li
- Department of Emergency Medicine and Critical Care, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, People's Republic of China
| | - Diyu Lv
- Department of Emergency Medicine and Critical Care, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, People's Republic of China
| | - Yiyuan Wang
- Department of Emergency Medicine and Critical Care, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, People's Republic of China
| | - Fangjie Huo
- Department of Respiratory Medicine, Xi'an No. 4 hospital, Xi'an, 710004, People's Republic of China
| | - Jianwen Bai
- Department of Emergency Medicine and Critical Care, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, People's Republic of China.,Department of Emergency Medicine and Critical Care, Shanghai East Hospital, Nanjing Medical University, Nanjing, Jiangsu Province, 211166, People's Republic of China
| | - Chunmei Wang
- Department of Emergency Medicine and Critical Care, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, People's Republic of China.,Department of Emergency Medicine and Critical Care, Shanghai East Hospital, Nanjing Medical University, Nanjing, Jiangsu Province, 211166, People's Republic of China
| |
Collapse
|
38
|
Singh P, Nagori A, Lodha R, Sethi T. Early prediction of hypothermia in pediatric intensive care units using machine learning. Front Physiol 2022; 13:921884. [PMID: 36171970 PMCID: PMC9511412 DOI: 10.3389/fphys.2022.921884] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2022] [Accepted: 07/28/2022] [Indexed: 11/13/2022] Open
Abstract
Hypothermia is a life-threatening condition where the temperature of the body drops below 35°C and is a key source of concern in Intensive Care Units (ICUs). Early identification can help to nudge clinical management to initiate early interventions. Despite its importance, very few studies have focused on the early prediction of hypothermia. In this study, we aim to monitor and predict Hypothermia (30 min-4 h) ahead of its onset using machine learning (ML) models developed on physiological vitals and to prospectively validate the best performing model in the pediatric ICU. We developed and evaluated ML algorithms for the early prediction of hypothermia in a pediatric ICU. Sepsis advanced forecasting engine ICU Database (SafeICU) data resource is an in-house ICU source of data built in the Pediatric ICU at the All-India Institute of Medical Science (AIIMS), New Delhi. Each time-stamp at 1-min resolution was labeled for the presence of hypothermia to construct a retrospective cohort of pediatric patients in the SafeICU data resource. The training set consisted of windows of the length of 4.2 h with a lead time of 30 min-4 h from the onset of hypothermia. A set of 3,835 hand-engineered time-series features were calculated to capture physiological features from the time series. Features selection using the Boruta algorithm was performed to select the most important predictors of hypothermia. A battery of models such as gradient boosting machine, random forest, AdaBoost, and support vector machine (SVM) was evaluated utilizing five-fold test sets. The best-performing model was prospectively validated. A total of 148 patients with 193 ICU stays were eligible for the model development cohort. Of 3,939 features, 726 were statistically significant in the Boruta analysis for the prediction of Hypothermia. The gradient boosting model performed best with an Area Under the Receiver Operating Characteristic curve (AUROC) of 85% (SD = 1.6) and a precision of 59.2% (SD = 8.8) for a 30-min lead time before the onset of Hypothermia onset. As expected, the model showed a decline in model performance at higher lead times, such as AUROC of 77.2% (SD = 2.3) and precision of 41.34% (SD = 4.8) for 4 h ahead of Hypothermia onset. Our GBM(gradient boosting machine) model produced equal and superior results for the prospective validation, where an AUROC of 79.8% and a precision of 53% for a 30-min lead time before the onset of Hypothermia whereas an AUROC of 69.6% and a precision of 38.52% for a (30 min-4 h) lead time prospective validation of Hypothermia. Therefore, this work establishes a pipeline termed ThermoGnose for predicting hypothermia, a major complication in pediatric ICUs.
Collapse
Affiliation(s)
- Pradeep Singh
- Indraprastha Institute of Information Technology, Delhi, India
| | - Aditya Nagori
- Indraprastha Institute of Information Technology, Delhi, India
- CSIR-Institute of Genomics and Integrative Biology, New Delhi, India
- Academy of Scientific and Innovative Research (AcSIR), Ghaziabad, India
| | - Rakesh Lodha
- All India Institute of Medical Sciences, Department of Pediatrics, New Delhi, India
| | - Tavpritesh Sethi
- Indraprastha Institute of Information Technology, Delhi, India
- All India Institute of Medical Sciences, Department of Pediatrics, New Delhi, India
- *Correspondence: Tavpritesh Sethi,
| |
Collapse
|
39
|
Kim D, Jin BT. Development and Comparative Performance of Physiologic Monitoring Strategies in the Emergency Department. JAMA Netw Open 2022; 5:e2233712. [PMID: 36169956 PMCID: PMC9520367 DOI: 10.1001/jamanetworkopen.2022.33712] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
IMPORTANCE Accurate and timely documentation of vital signs affects all aspects of triage, diagnosis, and management. The adequacy of current patient monitoring practices and the potential to improve on them are poorly understood. OBJECTIVE To develop measures of fit between documented and actual patient vital signs throughout the visit, as determined from continuous physiologic monitoring, and to compare the performance of actual practice with alternative patient monitoring strategies. DESIGN, SETTING, AND PARTICIPANTS This cross-sectional study evaluated 25 751 adult visits to continuously monitored emergency department (ED) beds between August 1, 2020, and December 31, 2021. A series of monitoring strategies for the documentation of vital signs (heart rate [HR], respiratory rate [RR], oxygen saturation by pulse oximetry [Spo2], mean arterial pressure [MAP]) was developed and the strategies' ability to capture physiologic trends and vital sign abnormalities simulated. Strategies included equal spacing of charting events, charting at variable intervals depending on the last observed values, and discrete optimization of charting events. MAIN OUTCOMES AND MEASURES Coverage was defined as the proportion of monitor-derived vital sign measurements (at 1-minute resolution) that fall within the bounds of nursing-charted values over the course of an ED visit (HR ± 5 beats/min, RR ± 3 breaths/min, Spo2 ± 2%, and MAP ± 6 mm Hg). Capture was defined as the documentation of a vital sign abnormality detected by bedside monitor (tachycardia [HR >100 beats/min], bradycardia [HR <60 beats/min], hypotension [MAP <65 mm Hg], and hypoxia [Spo2 <95%]). RESULTS Median patient age was 60 years (IQR, 43-75 years), and 13 329 visits (51.8%) were by women. Monitored visits had a median of 4 (IQR, 2-5) vital sign charting events per visit. Compared with actual practice, a simple rule, which observes vital signs more frequently if the last observation fell outside the bounds of the previous values, and using the same number of observations as actual practice, produced relative coverage improvements of 31.5% (95% CI, 30.5%-32.5%) for HR, 31.0% (95% CI, 30.0%-32.0%) for MAP, 16.8% (95% CI, 16.0%-17.6%) for RR, and 7.8% (95% CI, 7.3%-8.3%) for Spo2. The same strategy improved capture of abnormalities by 38.9% (95% CI, 26.8%-52.2%) for tachycardia, 38.1% (95% CI, 29.0%-47.9%) for bradycardia, 39.0% (95% CI, 24.2%-55.7%) for hypotension, and 123.1% (95% CI, 110.7%-136.3%) for hypoxia. Analysis of optimal coverage suggested an additional scope for improvement through more sophisticated strategies. CONCLUSIONS AND RELEVANCE In this cross-sectional study, actual documentation of ED vital signs was variable and incomplete, missing important trends and abnormalities. Alternative monitoring strategies may improve on current practice without increasing the overall frequency of patient monitoring.
Collapse
Affiliation(s)
- David Kim
- Department of Emergency Medicine, Stanford University, Palo Alto, California
| | - Boyang Tom Jin
- Department of Computer Science, Stanford University, Palo Alto, California
| |
Collapse
|
40
|
Kanakaraj P, Ramadass K, Bao S, Basford M, Jones LM, Lee HH, Xu K, Schilling KG, Carr JJ, Terry JG, Huo Y, Sandler KL, Netwon AT, Landman BA. Workflow Integration of Research AI Tools into a Hospital Radiology Rapid Prototyping Environment. J Digit Imaging 2022; 35:1023-1033. [PMID: 35266088 PMCID: PMC9485498 DOI: 10.1007/s10278-022-00601-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Revised: 01/14/2022] [Accepted: 01/23/2022] [Indexed: 11/25/2022] Open
Abstract
The field of artificial intelligence (AI) in medical imaging is undergoing explosive growth, and Radiology is a prime target for innovation. The American College of Radiology Data Science Institute has identified more than 240 specific use cases where AI could be used to improve clinical practice. In this context, thousands of potential methods are developed by research labs and industry innovators. Deploying AI tools within a clinical enterprise, even on limited retrospective evaluation, is complicated by security and privacy concerns. Thus, innovation must be weighed against the substantive resources required for local clinical evaluation. To reduce barriers to AI validation while maintaining rigorous security and privacy standards, we developed the AI Imaging Incubator. The AI Imaging Incubator serves as a DICOM storage destination within a clinical enterprise where images can be directed for novel research evaluation under Institutional Review Board approval. AI Imaging Incubator is controlled by a secure HIPAA-compliant front end and provides access to a menu of AI procedures captured within network-isolated containers. Results are served via a secure website that supports research and clinical data formats. Deployment of new AI approaches within this system is streamlined through a standardized application programming interface. This manuscript presents case studies of the AI Imaging Incubator applied to randomizing lung biopsies on chest CT, liver fat assessment on abdomen CT, and brain volumetry on head MRI.
Collapse
Affiliation(s)
| | | | - Shunxing Bao
- Computer Science, Vanderbilt University, Nashville, TN USA
| | - Melissa Basford
- Vanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN USA
| | - Laura M. Jones
- Vanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN USA
| | - Ho Hin Lee
- Computer Science, Vanderbilt University, Nashville, TN USA
| | - Kaiwen Xu
- Computer Science, Vanderbilt University, Nashville, TN USA
| | - Kurt G. Schilling
- Vanderbilt University Institute of Imaging Science, Vanderbilt University Medical Center, Nashville, TN USA ,Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN USA
| | - John Jeffrey Carr
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN USA
| | - James Gregory Terry
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN USA
| | - Yuankai Huo
- Computer Science, Vanderbilt University, Nashville, TN USA ,Data Science Institute, Vanderbilt University, Nashville, TN USA
| | - Kim Lori Sandler
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN USA
| | - Allen T. Netwon
- Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN USA
| | - Bennett A. Landman
- Computer Science, Vanderbilt University, Nashville, TN USA ,Vanderbilt University Institute of Imaging Science, Vanderbilt University Medical Center, Nashville, TN USA ,Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN USA ,Electrical Engineering, Vanderbilt University, Nashville, TN USA ,Biomedical Engineering, Vanderbilt University, Nashville, TN USA ,Data Science Institute, Vanderbilt University, Nashville, TN USA
| |
Collapse
|
41
|
Henry KE, Kornfield R, Sridharan A, Linton RC, Groh C, Wang T, Wu A, Mutlu B, Saria S. Human-machine teaming is key to AI adoption: clinicians' experiences with a deployed machine learning system. NPJ Digit Med 2022; 5:97. [PMID: 35864312 PMCID: PMC9304371 DOI: 10.1038/s41746-022-00597-7] [Citation(s) in RCA: 50] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Accepted: 03/09/2022] [Indexed: 12/23/2022] Open
Abstract
While a growing number of machine learning (ML) systems have been deployed in clinical settings with the promise of improving patient care, many have struggled to gain adoption and realize this promise. Based on a qualitative analysis of coded interviews with clinicians who use an ML-based system for sepsis, we found that, rather than viewing the system as a surrogate for their clinical judgment, clinicians perceived themselves as partnering with the technology. Our findings suggest that, even without a deep understanding of machine learning, clinicians can build trust with an ML system through experience, expert endorsement and validation, and systems designed to accommodate clinicians’ autonomy and support them across their entire workflow.
Collapse
Affiliation(s)
- Katharine E Henry
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - Rachel Kornfield
- Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.,Center for Behavioral Intervention Technologies, Northwestern University, Chicago, IL, USA
| | | | | | - Catherine Groh
- Department of Industrial Engineering, University of Wisconsin-Madison, Madison, WI, USA
| | - Tony Wang
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - Albert Wu
- Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Bilge Mutlu
- Department of Industrial Engineering, University of Wisconsin-Madison, Madison, WI, USA. .,Department of Computer Sciences, University of Wisconsin-Madison, Madison, WI, USA.
| | - Suchi Saria
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA. .,Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. .,Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA. .,Bayesian Health, New York, NY, 10005, USA.
| |
Collapse
|
42
|
Conditional generation of medical time series for extrapolation to underrepresented populations. PLOS DIGITAL HEALTH 2022; 1:e0000074. [PMID: 36812549 PMCID: PMC9931259 DOI: 10.1371/journal.pdig.0000074] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/15/2022] [Accepted: 06/10/2022] [Indexed: 11/19/2022]
Abstract
The widespread adoption of electronic health records (EHRs) and subsequent increased availability of longitudinal healthcare data has led to significant advances in our understanding of health and disease with direct and immediate impact on the development of new diagnostics and therapeutic treatment options. However, access to EHRs is often restricted due to their perceived sensitive nature and associated legal concerns, and the cohorts therein typically are those seen at a specific hospital or network of hospitals and therefore not representative of the wider population of patients. Here, we present HealthGen, a new approach for the conditional generation of synthetic EHRs that maintains an accurate representation of real patient characteristics, temporal information and missingness patterns. We demonstrate experimentally that HealthGen generates synthetic cohorts that are significantly more faithful to real patient EHRs than the current state-of-the-art, and that augmenting real data sets with conditionally generated cohorts of underrepresented subpopulations of patients can significantly enhance the generalisability of models derived from these data sets to different patient populations. Synthetic conditionally generated EHRs could help increase the accessibility of longitudinal healthcare data sets and improve the generalisability of inferences made from these data sets to underrepresented populations.
Collapse
|
43
|
Momenzadeh A, Shamsa A, Meyer JG. Bias or biology? Importance of model interpretation in machine learning studies from electronic health records. JAMIA Open 2022; 5:ooac063. [PMID: 35958671 PMCID: PMC9360778 DOI: 10.1093/jamiaopen/ooac063] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2022] [Revised: 06/27/2022] [Accepted: 07/04/2022] [Indexed: 11/13/2022] Open
Abstract
Objective The rate of diabetic complication progression varies across individuals and understanding factors that alter the rate of complication progression may uncover new clinical interventions for personalized diabetes management. Materials and Methods We explore how various machine learning (ML) models and types of electronic health records (EHRs) can predict fast versus slow onset of neuropathy, nephropathy, ocular disease, or cardiovascular disease using only patient data collected prior to diabetes diagnosis. Results We find that optimized random forest models performed best to accurately predict the diagnosis of a diabetic complication, with the most effective model distinguishing between fast versus slow nephropathy (AUROC = 0.75). Using all data sets combined allowed for the highest model predictive performance, and social history or laboratory alone were most predictive. SHapley Additive exPlanations (SHAP) model interpretation allowed for exploration of predictors of fast and slow complication diagnosis, including underlying biases present in the EHR. Patients in the fast group had more medical visits, incurring a potential informed decision bias. Discussion Our study is unique in the realm of ML studies as it leverages SHAP as a starting point to explore patient markers not routinely used in diabetes monitoring. A mix of both bias and biological processes is likely present in influencing a model’s ability to distinguish between groups. Conclusion Overall, model interpretation is a critical step in evaluating validity of a user-intended endpoint for a model when using EHR data, and predictors affected by bias and those driven by biologic processes should be equally recognized.
Collapse
Affiliation(s)
- Amanda Momenzadeh
- Department of Biochemistry, Medical College of Wisconsin , Milwaukee, Wisconsin, USA
- Department of Computational Biomedicine, Cedars-Sinai , Beverly Hills, California, USA
| | - Ali Shamsa
- Department of Biochemistry, Medical College of Wisconsin , Milwaukee, Wisconsin, USA
| | - Jesse G Meyer
- Department of Biochemistry, Medical College of Wisconsin , Milwaukee, Wisconsin, USA
- Department of Computational Biomedicine, Cedars-Sinai , Beverly Hills, California, USA
| |
Collapse
|
44
|
Padula WV, Kreif N, Vanness DJ, Adamson B, Rueda JD, Felizzi F, Jonsson P, IJzerman MJ, Butte A, Crown W. Machine Learning Methods in Health Economics and Outcomes Research-The PALISADE Checklist: A Good Practices Report of an ISPOR Task Force. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2022; 25:1063-1080. [PMID: 35779937 DOI: 10.1016/j.jval.2022.03.022] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Accepted: 03/25/2022] [Indexed: 06/15/2023]
Abstract
Advances in machine learning (ML) and artificial intelligence offer tremendous potential benefits to patients. Predictive analytics using ML are already widely used in healthcare operations and care delivery, but how can ML be used for health economics and outcomes research (HEOR)? To answer this question, ISPOR established an emerging good practices task force for the application of ML in HEOR. The task force identified 5 methodological areas where ML could enhance HEOR: (1) cohort selection, identifying samples with greater specificity with respect to inclusion criteria; (2) identification of independent predictors and covariates of health outcomes; (3) predictive analytics of health outcomes, including those that are high cost or life threatening; (4) causal inference through methods, such as targeted maximum likelihood estimation or double-debiased estimation-helping to produce reliable evidence more quickly; and (5) application of ML to the development of economic models to reduce structural, parameter, and sampling uncertainty in cost-effectiveness analysis. Overall, ML facilitates HEOR through the meaningful and efficient analysis of big data. Nevertheless, a lack of transparency on how ML methods deliver solutions to feature selection and predictive analytics, especially in unsupervised circumstances, increases risk to providers and other decision makers in using ML results. To examine whether ML offers a useful and transparent solution to healthcare analytics, the task force developed the PALISADE Checklist. It is a guide for balancing the many potential applications of ML with the need for transparency in methods development and findings.
Collapse
Affiliation(s)
- William V Padula
- Department of Pharmaceutical and Health Economics, School of Pharmacy, University of Southern California, Los Angeles, CA, USA; The Leonard D. Schaeffer Center for Health Policy & Economics, University of Southern California, Los Angeles, CA, USA.
| | - Noemi Kreif
- Centre for Health Economics, University of York, York, England, UK
| | - David J Vanness
- Department of Health Policy and Administration, College of Health and Human Development, Pennsylvania State University, Hershey, PA, USA
| | | | | | | | - Pall Jonsson
- National Institute for Health and Care Excellence, Manchester, England, UK
| | - Maarten J IJzerman
- Centre for Health Policy, School of Population and Global Health, University of Melbourne, Melbourne, Australia
| | - Atul Butte
- School of Medicine, University of California, San Francisco, San Francisco, CA, USA
| | - William Crown
- The Heller School for Social Policy and Management, Brandeis University, Waltham, MA, USA.
| |
Collapse
|
45
|
Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing. Nat Med 2022; 28:1447-1454. [PMID: 35864251 DOI: 10.1038/s41591-022-01895-z] [Citation(s) in RCA: 25] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2021] [Accepted: 06/08/2022] [Indexed: 01/04/2023]
Abstract
Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments at early time points, which is critical for improving sepsis outcomes. In view of the increasing use of such systems, better understanding of how they are adopted and used by healthcare providers is needed. Here, we analyzed provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System), which was deployed at five hospitals over a 2-year period. Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. Adjusting for patient presentation and severity, patients with sepsis whose alert was confirmed by a provider within 3 h had a 1.85-h (95% CI 1.66-2.00) reduction in median time to first antibiotic order compared to patients with sepsis whose alert was either dismissed, confirmed more than 3 h after the alert or never addressed in the system. Finally, we found that emergency department providers and providers who had previous interactions with an alert were more likely to interact with alerts, as well as to confirm alerts on retrospectively identified patients with sepsis. Beyond efforts to improve the performance of early warning systems, efforts to improve adoption are essential to their clinical impact and should focus on understanding providers' knowledge of, experience with and attitudes toward such systems.
Collapse
|
46
|
Adams R, Henry KE, Sridharan A, Soleimani H, Zhan A, Rawat N, Johnson L, Hager DN, Cosgrove SE, Markowski A, Klein EY, Chen ES, Saheed MO, Henley M, Miranda S, Houston K, Linton RC, Ahluwalia AR, Wu AW, Saria S. Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis. Nat Med 2022; 28:1455-1460. [PMID: 35864252 DOI: 10.1038/s41591-022-01894-0] [Citation(s) in RCA: 82] [Impact Index Per Article: 41.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2021] [Accepted: 06/08/2022] [Indexed: 12/20/2022]
Abstract
Early recognition and treatment of sepsis are linked to improved patient outcomes. Machine learning-based early warning systems may reduce the time to recognition, but few systems have undergone clinical evaluation. In this prospective, multi-site cohort study, we examined the association between patient outcomes and provider interaction with a deployed sepsis alert system called the Targeted Real-time Early Warning System (TREWS). During the study, 590,736 patients were monitored by TREWS across five hospitals. We focused our analysis on 6,877 patients with sepsis who were identified by the alert before initiation of antibiotic therapy. Adjusting for patient presentation and severity, patients in this group whose alert was confirmed by a provider within 3 h of the alert had a reduced in-hospital mortality rate (3.3%, confidence interval (CI) 1.7, 5.1%, adjusted absolute reduction, and 18.7%, CI 9.4, 27.0%, adjusted relative reduction), organ failure and length of stay compared with patients whose alert was not confirmed by a provider within 3 h. Improvements in mortality rate (4.5%, CI 0.8, 8.3%, adjusted absolute reduction) and organ failure were larger among those patients who were additionally flagged as high risk. Our findings indicate that early warning systems have the potential to identify sepsis patients early and improve patient outcomes and that sepsis patients who would benefit the most from early treatment can be identified and prioritized at the time of the alert.
Collapse
Affiliation(s)
- Roy Adams
- Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, MD, USA.,Department of Psychiatry and Behavioral Science, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Katharine E Henry
- Department of Psychiatry and Behavioral Science, Johns Hopkins School of Medicine, Baltimore, MD, USA.,Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | | | - Hossein Soleimani
- Health Informatics, University of California, San Francisco, CA, USA
| | - Andong Zhan
- Department of Psychiatry and Behavioral Science, Johns Hopkins School of Medicine, Baltimore, MD, USA.,Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - Nishi Rawat
- Armstrong Institute for Patient Safety and Quality, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Lauren Johnson
- Department of Quality Improvement, Johns Hopkins Hospital, Baltimore, MD, USA
| | - David N Hager
- Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Sara E Cosgrove
- Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | | | - Eili Y Klein
- Department of Emergency Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Edward S Chen
- Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Mustapha O Saheed
- Department of Emergency Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Maureen Henley
- Department of Quality Improvement, Johns Hopkins Hospital, Baltimore, MD, USA
| | - Sheila Miranda
- Department of Medicine, Johns Hopkins Hospital, Baltimore, MD, USA
| | - Katrina Houston
- Department of Quality Improvement, Johns Hopkins Hospital, Baltimore, MD, USA
| | | | | | - Albert W Wu
- Armstrong Institute for Patient Safety and Quality, Johns Hopkins School of Medicine, Baltimore, MD, USA. .,Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA. .,Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. .,Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. .,Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
| | - Suchi Saria
- Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, MD, USA. .,Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA. .,Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA. .,Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. .,Bayesian Health, New York, NY, USA.
| |
Collapse
|
47
|
Tell Me Something Interesting: Clinical Utility of Machine Learning Prediction Models in the ICU. J Biomed Inform 2022; 132:104107. [DOI: 10.1016/j.jbi.2022.104107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 05/26/2022] [Accepted: 05/28/2022] [Indexed: 11/18/2022]
|
48
|
The principles of whole-hospital predictive analytics monitoring for clinical medicine originated in the neonatal ICU. NPJ Digit Med 2022; 5:41. [PMID: 35361861 PMCID: PMC8971442 DOI: 10.1038/s41746-022-00584-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Accepted: 02/23/2022] [Indexed: 11/17/2022] Open
Abstract
In 2011, a multicenter group spearheaded at the University of Virginia demonstrated reduced mortality from real-time continuous cardiorespiratory monitoring in the neonatal ICU using what we now call Artificial Intelligence, Big Data, and Machine Learning. The large, randomized heart rate characteristics trial made real, for the first time that we know of, the promise that early detection of illness would allow earlier and more effective intervention and improved patient outcomes. Currently, though, we hear as much of failures as we do of successes in the rapidly growing field of predictive analytics monitoring that has followed. This Perspective aims to describe the principles of how we developed heart rate characteristics monitoring for neonatal sepsis and then applied them throughout adult ICU and hospital medicine. It primarily reflects the work since the 1990s of the University of Virginia group: the theme is that sudden and catastrophic deteriorations can be preceded by subclinical but measurable physiological changes apparent in the continuous cardiorespiratory monitoring and electronic health record.
Collapse
|
49
|
Liu X, Cao Y, Guo Y, Gong X, Feng Y, Wang Y, Wang M, Cui M, Guo W, Zhang L, Zhao N, Song X, Zheng X, Chen X, Shen Q, Zhang S, Song Z, Li L, Feng S, Han M, Zhu X, Jiang E, Chen J. Dynamic forecasting of severe acute graft-versus-host disease after transplantation. NATURE COMPUTATIONAL SCIENCE 2022; 2:153-159. [PMID: 38177449 PMCID: PMC10766514 DOI: 10.1038/s43588-022-00213-4] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/31/2021] [Accepted: 02/14/2022] [Indexed: 01/06/2024]
Abstract
Forecasting of severe acute graft-versus-host disease (aGVHD) after transplantation is a challenging 'large p, small n' problem that suffers from nonuniform data sampling. We propose a dynamic probabilistic algorithm, daGOAT, that accommodates sampling heterogeneity, integrates multidimensional clinical data and continuously updates the daily risk score for severe aGVHD onset within a two-week moving window. In the studied cohorts, the cross-validated area under the receiver operator characteristic curve (AUROC) of daGOAT rose steadily after transplantation and peaked at ≥0.78 in both the adult and pediatric cohorts, outperforming the two-biomarker MAGIC score, three-biomarker Ann Arbor score, peri-transplantation features-based models and XGBoost. Simulation experiments indicated that the daGOAT algorithm is well suited for short time-series scenarios where the underlying process for event generation is smooth, multidimensional and where there are frequent and irregular data missing. daGOAT's broader utility was demonstrated by performance testing on a remotely different task, that is, prediction of imminent human postural change based on smartphone inertial sensor time-series data.
Collapse
Affiliation(s)
- Xueou Liu
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Yigeng Cao
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Ye Guo
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Xiaowen Gong
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Yahui Feng
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Yao Wang
- Yidu Cloud Technology Inc., Beijing, China
| | - Mingyang Wang
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | | | - Wenwen Guo
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Luyang Zhang
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Ningning Zhao
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Xiaoqiang Song
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Xuetong Zheng
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Xia Chen
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Qiujin Shen
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Song Zhang
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Zhen Song
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Linfeng Li
- Yidu Cloud Technology Inc., Beijing, China
| | - Sizhou Feng
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Mingzhe Han
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China
| | - Xiaofan Zhu
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China.
| | - Erlie Jiang
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China.
| | - Junren Chen
- State Key Laboratory of Experimental Hematology, National Clinical Research Center for Blood Diseases, Institute of Hematology & Blood Diseases Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Tianjin, China.
| |
Collapse
|
50
|
Niemantsverdriet MSA, Varkila MRJ, Vromen-Wijsman JLP, Hoefer IE, Bellomo D, van Vliet MH, van Solinge WW, Cremer OL, Haitjema S. Transportability and Implementation Challenges of Early Warning Scores for Septic Shock in the ICU: A Perspective on the TREWScore. Front Med (Lausanne) 2022; 8:793815. [PMID: 35211485 PMCID: PMC8860834 DOI: 10.3389/fmed.2021.793815] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2021] [Accepted: 12/22/2021] [Indexed: 11/13/2022] Open
Abstract
The increased use of electronic health records (EHRs) has improved the availability of routine care data for medical research. Combined with machine learning techniques this has spurred the development of early warning scores (EWSs) in hospitals worldwide. EWSs are commonly used in the hospital where they have been developed, yet few have been transported to external settings and/or internationally. In this perspective, we describe our experiences in implementing the TREWScore, a septic shock EWS, and the transportability challenges regarding domain, predictors, and clinical outcome we faced. We used data of 53,330 ICU stays from Medical Information Mart for Intensive Care-III (MIMIC-III) and 18,013 ICU stays from the University Medical Center (UMC) Utrecht, including 17,023 (31.9%) and 2,557 (14.2%) cases of sepsis, respectively. The MIMIC-III and UMC populations differed significantly regarding the length of stay (6.9 vs. 9.0 days) and hospital mortality (11.6% vs. 13.6%). We mapped all 54 TREWScore predictors to the UMC database: 31 were readily available, seven required unit conversion, 14 had to be engineered, one predictor required text mining, and one predictor could not be mapped. Lastly, we classified sepsis cases for septic shock using the sepsis-2 criteria. Septic shock populations (UMC 31.3% and MIMIC-III 23.3%) and time to shock events showed significant differences between the two cohorts. In conclusion, we identified challenges to transportability and implementation regarding domain, predictors, and clinical outcome when transporting EWS between hospitals across two continents. These challenges need to be systematically addressed to improve model transportability between centers and unlock the potential clinical utility of EWS.
Collapse
Affiliation(s)
- Michael S A Niemantsverdriet
- Central Diagnostic Laboratory, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands.,SkylineDx, Rotterdam, Netherlands
| | - Meri R J Varkila
- Department of Intensive Care Medicine, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | | | - Imo E Hoefer
- Central Diagnostic Laboratory, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | | | | | - Wouter W van Solinge
- Central Diagnostic Laboratory, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Olaf L Cremer
- Department of Intensive Care Medicine, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Saskia Haitjema
- Central Diagnostic Laboratory, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| |
Collapse
|