1
|
Pérez-Tome JC, Parrón-Carreño T, Castaño-Fernández AB, Nievas-Soriano BJ, Castro-Luna G. Sepsis mortality prediction with Machine Learning Tecniques. Med Intensiva 2024:S2173-5727(24)00131-0. [PMID: 38876921 DOI: 10.1016/j.medine.2024.05.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Accepted: 04/30/2024] [Indexed: 06/16/2024]
Abstract
OBJECTIVE To develop a sepsis death classification model based on machine learning techniques for patients admitted to the Intensive Care Unit (ICU). DESIGN Cross-sectional descriptive study. SETTING The Intensive Care Units (ICUs) of three Hospitals from Murcia (Spain) and patients from the MIMIC III open-access database. PATIENTS 180 patients diagnosed with sepsis in the ICUs of three hospitals and a total of 4559 patients from the MIMIC III database. MAIN VARIABLES OF INTEREST Age, weight, heart rate, respiratory rate, temperature, lactate levels, partial oxygen saturation, systolic and diastolic blood pressure, pH, urine, and potassium levels. RESULTS A random forest classification model was calculated using the local and MIMIC III databases. The sensitivity of the model of our database, considering all the variables classified as important by the random forest, was 95.45%, the specificity was 100%, the accuracy was 96.77%, and an AUC of 95%. . In the case of the model based on the MIMIC III database, the sensitivity was 97.55%, the specificity was 100%, and the precision was 98.28%, with an AUC of 97.3%. CONCLUSIONS According to random forest classification in both databases, lactate levels, urine output and variables related to acid.base equilibrium were the most important variable in mortality due to sepsis in the ICU. The potassium levels were more critical in the MIMIC III database than the local database.
Collapse
Affiliation(s)
| | - Tesifón Parrón-Carreño
- Department of Nursing: Physiotherapy and Medicine, University of Almeria, 04120 Almeria, Spain
| | | | | | - Gracia Castro-Luna
- Department of Nursing: Physiotherapy and Medicine, University of Almeria, 04120 Almeria, Spain.
| |
Collapse
|
2
|
Peterson KS, Chapman AB, Widanagamaachchi W, Sutton J, Ochoa B, Jones BE, Stevens V, Classen DC, Jones MM. Automating detection of diagnostic error of infectious diseases using machine learning. PLOS DIGITAL HEALTH 2024; 3:e0000528. [PMID: 38848317 PMCID: PMC11161023 DOI: 10.1371/journal.pdig.0000528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Accepted: 05/07/2024] [Indexed: 06/09/2024]
Abstract
Diagnostic error, a cause of substantial morbidity and mortality, is largely discovered and evaluated through self-report and manual review, which is costly and not suitable to real-time intervention. Opportunities exist to leverage electronic health record data for automated detection of potential misdiagnosis, executed at scale and generalized across diseases. We propose a novel automated approach to identifying diagnostic divergence considering both diagnosis and risk of mortality. Our objective was to identify cases of emergency department infectious disease misdiagnoses by measuring the deviation between predicted diagnosis and documented diagnosis, weighted by mortality. Two machine learning models were trained for prediction of infectious disease and mortality using the first 24h of data. Charts were manually reviewed by clinicians to determine whether there could have been a more correct or timely diagnosis. The proposed approach was validated against manual reviews and compared using the Spearman rank correlation. We analyzed 6.5 million ED visits and over 700 million associated clinical features from over one hundred emergency departments. The testing set performances of the infectious disease (Macro F1 = 86.7, AUROC 90.6 to 94.7) and mortality model (Macro F1 = 97.6, AUROC 89.1 to 89.1) were in expected ranges. Human reviews and the proposed automated metric demonstrated positive correlations ranging from 0.231 to 0.358. The proposed approach for diagnostic deviation shows promise as a potential tool for clinicians to find diagnostic errors. Given the vast number of clinical features used in this analysis, further improvements likely need to either take greater account of data structure (what occurs before when) or involve natural language processing. Further work is needed to explain the potential reasons for divergence and to refine and validate the approach for implementation in real-world settings.
Collapse
Affiliation(s)
- Kelly S. Peterson
- Veterans Health Administration, Office of Analytics and Performance Integration, Washington D.C., District of Columbia, United States of America
- Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, Utah, United States of America
| | - Alec B. Chapman
- Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, Utah, United States of America
- Veterans Affairs Health Care System, Salt Lake City, Utah, United States of America
| | - Wathsala Widanagamaachchi
- Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, Utah, United States of America
- Veterans Affairs Health Care System, Salt Lake City, Utah, United States of America
| | - Jesse Sutton
- Veterans Affairs Health Care System, Minneapolis, Minnesota, United States of America
| | - Brennan Ochoa
- Rocky Mountain Infectious Diseases Specialists, Aurora, Colorado, United States of America
| | - Barbara E. Jones
- Veterans Affairs Health Care System, Salt Lake City, Utah, United States of America
- Division of Pulmonary & Critical Care Medicine, University of Utah, Salt Lake City, Utah, United States of America
| | - Vanessa Stevens
- Veterans Health Administration, Office of Analytics and Performance Integration, Washington D.C., District of Columbia, United States of America
- Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, Utah, United States of America
| | - David C. Classen
- Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, Utah, United States of America
| | - Makoto M. Jones
- Veterans Health Administration, Office of Analytics and Performance Integration, Washington D.C., District of Columbia, United States of America
- Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, Utah, United States of America
- Veterans Affairs Health Care System, Salt Lake City, Utah, United States of America
| |
Collapse
|
3
|
Soliman MM, Marshall C, Kimball JP, Choudhary T, Clermont G, Pinsky MR, Buchman TG, Coopersmith CM, Inan OT, Kamaleswaran R. Parsimonious Waveform-derived Features consisting of Pulse Arrival Time and Heart Rate Variability Predicts the Onset of Septic Shock. Biomed Signal Process Control 2024; 92:105974. [PMID: 38559667 PMCID: PMC10977921 DOI: 10.1016/j.bspc.2024.105974] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Sepsis is a major public health emergency and one of the leading causes of morbidity and mortality in critically ill patients. For each hour treatment is delayed, shock-related mortality increases, so early diagnosis and intervention is of utmost importance. However, earlier recognition of shock requires active monitoring, which may be delayed due to subclinical manifestations of the disease at the early phase of onset. Machine learning systems can increase timely detection of shock onset by exploiting complex interactions among continuous physiological waveforms. We use a dataset consisting of high-resolution physiological waveforms from intensive care unit (ICU) of a tertiary hospital system. We investigate the use of mean arterial blood pressure (MAP), pulse arrival time (PAT), heart rate variability (HRV), and heart rate (HR) for the early prediction of shock onset. Using only five minutes of the aforementioned vital signals from 239 ICU patients, our developed models can accurately predict septic shock onset 6 to 36 hours prior to clinical recognition with area under the receiver operating characteristic (AUROC) of 0.84 and 0.8 respectively. This work lays foundations for a robust, efficient, accurate and early prediction of septic shock onset which may help clinicians in their decision-making processes. This study introduces machine learning models that provide fast and accurate predictions of septic shock onset times up to 36 hours in advance. BP, PAT and HR dynamics can independently predict septic shock onset with a look-back period of only 5 mins.
Collapse
Affiliation(s)
- Moamen M. Soliman
- School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, 30332, GA, USA
| | - Curtis Marshall
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, 30322, GA, USA
| | - Jacob P. Kimball
- School of Biomedical and Electrical Engineering, University of Portland, Portland, 97203, OR, USA
| | - Tilendra Choudhary
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, 30322, GA, USA
| | - Gilles Clermont
- School of Medicine, University of Pittsburgh, Pittsburgh, 15213, PA, USA
| | - Michael R. Pinsky
- School of Medicine, University of Pittsburgh, Pittsburgh, 15213, PA, USA
| | - Timothy G. Buchman
- Department of Surgery and Emory Critical Care Center, Emory University School of Medicine, Atlanta, 30322, GA, USA
| | - Craig M. Coopersmith
- Department of Surgery and Emory Critical Care Center, Emory University School of Medicine, Atlanta, 30322, GA, USA
| | - Omer T. Inan
- School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, 30332, GA, USA
- The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, 30332, GA, USA
| | - Rishikesan Kamaleswaran
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, 30322, GA, USA
- The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, 30332, GA, USA
| |
Collapse
|
4
|
Böhnke J, Zapf A, Kramer K, Weber P, Karch A, Rübsamen N. Diagnostic test accuracy in longitudinal study settings: theoretical approaches with use cases from clinical practice. J Clin Epidemiol 2024; 169:111314. [PMID: 38432525 DOI: 10.1016/j.jclinepi.2024.111314] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 02/21/2024] [Accepted: 02/26/2024] [Indexed: 03/05/2024]
Abstract
OBJECTIVES In this study, we evaluate how to estimate diagnostic test accuracy (DTA) correctly in the presence of longitudinal patient data (ie, repeated test applications per patient). STUDY DESIGN AND SETTING We used a nonparametric approach to estimate the sensitivity and specificity of three tests for different target conditions with varying characteristics (ie, episode length and disease-free intervals between episodes): 1) systemic inflammatory response syndrome (n = 36), 2) depression (n = 33), and 3) epilepsy (n = 30). DTA was estimated on the levels 'time', 'block', and 'patient-time' for each diagnosis, representing different research questions. The estimation was conducted for the time units per minute, per hour, and per day. RESULTS A comparison of DTA per and across use cases showed variations in the estimates, which resulted from the used level, the time unit, the resulting number of observations per patient, and the diagnosis-specific characteristics. Intra- and inter-use-case comparisons showed that the time-level had the highest DTA, particularly the larger the time unit, and that the patient-time-level approximated 50% sensitivity and specificity. CONCLUSION Researchers need to predefine their choices (ie, estimation levels and time units) based on their individual research aims, estimands, and diagnosis-specific characteristics of the target outcomes to make sure that unbiased and clinically relevant measures are communicated. In cases of uncertainty, researchers could report the DTA of the test using more than one estimation level and/or time unit.
Collapse
Affiliation(s)
- Julia Böhnke
- Institute of Epidemiology and Social Medicine, University of Münster, Albert-Schweitzer-Campus 1, 48149 Münster, Germany.
| | - Antonia Zapf
- Department of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Katharina Kramer
- Mathematical Statistics and Artificial Intelligence in Medicine, University of Augsburg, Augsburg, Germany
| | - Philipp Weber
- Department of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - André Karch
- Institute of Epidemiology and Social Medicine, University of Münster, Albert-Schweitzer-Campus 1, 48149 Münster, Germany
| | - Nicole Rübsamen
- Institute of Epidemiology and Social Medicine, University of Münster, Albert-Schweitzer-Campus 1, 48149 Münster, Germany
| |
Collapse
|
5
|
Pinto-de-Sá R, Sousa-Pinto B, Costa-de-Oliveira S. Brave New World of Artificial Intelligence: Its Use in Antimicrobial Stewardship-A Systematic Review. Antibiotics (Basel) 2024; 13:307. [PMID: 38666983 PMCID: PMC11047419 DOI: 10.3390/antibiotics13040307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2024] [Revised: 03/24/2024] [Accepted: 03/25/2024] [Indexed: 04/29/2024] Open
Abstract
Antimicrobial resistance (AMR) is a growing public health problem in the One Health dimension. Artificial intelligence (AI) is emerging in healthcare, since it is helpful to deal with large amounts of data and as a prediction tool. This systematic review explores the use of AI in antimicrobial stewardship programs (ASPs) and summarizes the predictive performance of machine learning (ML) algorithms, compared with clinical decisions, in inpatients and outpatients who need antimicrobial prescriptions. This review includes eighteen observational studies from PubMed, Scopus, and Web of Science. The exclusion criteria comprised studies conducted only in vitro, not addressing infectious diseases, or not referencing the use of AI models as predictors. Data such as study type, year of publication, number of patients, study objective, ML algorithms used, features, and predictors were extracted from the included publications. All studies concluded that ML algorithms were useful to assist antimicrobial stewardship teams in multiple tasks such as identifying inappropriate prescribing practices, choosing the appropriate antibiotic therapy, or predicting AMR. The most extracted performance metric was AUC, which ranged from 0.64 to 0.992. Despite the risks and ethical concerns that AI raises, it can play a positive and promising role in ASP.
Collapse
Affiliation(s)
- Rafaela Pinto-de-Sá
- Division of Microbiology, Department of Pathology, Faculty of Medicine, University of Porto, Alameda Prof. Hernâni Monteiro, 4200-319 Porto, Portugal;
| | - Bernardo Sousa-Pinto
- Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, 4200-319 Porto, Portugal;
- Center for Health Technology and Services Research—CINTESIS@RISE, Faculty of Medicine, University of Porto, 4200-319 Porto, Portugal
| | - Sofia Costa-de-Oliveira
- Division of Microbiology, Department of Pathology, Faculty of Medicine, University of Porto, Alameda Prof. Hernâni Monteiro, 4200-319 Porto, Portugal;
- Center for Health Technology and Services Research—CINTESIS@RISE, Faculty of Medicine, University of Porto, 4200-319 Porto, Portugal
| |
Collapse
|
6
|
Tang Y, Zhang Y, Li J. A time series driven model for early sepsis prediction based on transformer module. BMC Med Res Methodol 2024; 24:23. [PMID: 38273257 PMCID: PMC10809699 DOI: 10.1186/s12874-023-02138-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2023] [Accepted: 12/27/2023] [Indexed: 01/27/2024] Open
Abstract
Sepsis remains a critical concern in intensive care units due to its high mortality rate. Early identification and intervention are paramount to improving patient outcomes. In this study, we have proposed predictive models for early sepsis prediction based on time-series data, utilizing both CNN-Transformer and LSTM-Transformer architectures. By collecting time-series data from patients at 4, 8, and 12 h prior to sepsis diagnosis and subjecting it to various network models for analysis and comparison. In contrast to traditional recurrent neural networks, our model exhibited a substantial improvement of approximately 20%. On average, our model demonstrated an accuracy of 0.964 (± 0.018), a precision of 0.956 (± 0.012), a recall of 0.967 (± 0.012), and an F1 score of 0.959 (± 0.014). Furthermore, by adjusting the time window, it was observed that the Transformer-based model demonstrated exceptional predictive capabilities, particularly within the earlier time window (i.e., 12 h before onset), thus holding significant promise for early clinical diagnosis and intervention. Besides, we employed the SHAP algorithm to visualize the weight distribution of different features, enhancing the interpretability of our model and facilitating early clinical diagnosis and intervention.
Collapse
Affiliation(s)
- Yan Tang
- Department of Clinical Laboratory Medicine, Jinniu Maternity and Child Health Hospital of Chengdu, Chengdu, China
| | - Yu Zhang
- Information Center, West China Hospital, Sichuan University, Chengdu, China
| | - Jiaxi Li
- Department of Clinical Laboratory Medicine, Jinniu Maternity and Child Health Hospital of Chengdu, Chengdu, China.
| |
Collapse
|
7
|
Cheng TY, Yu-Chieh Ho S, Chien TW, Chou W. Global research trends in artificial intelligence for critical care with a focus on chord network charts: Bibliometric analysis. Medicine (Baltimore) 2023; 102:e35082. [PMID: 37746962 PMCID: PMC10519472 DOI: 10.1097/md.0000000000035082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/05/2023] [Revised: 08/14/2023] [Accepted: 08/15/2023] [Indexed: 09/26/2023] Open
Abstract
BACKGROUND The field of critical care-related artificial intelligence (AI) research is rapidly gaining interest. However, there is still a lack of comprehensive bibliometric studies that measure and analyze scientific publications on a global scale. Network charts have traditionally been used to highlight author collaborations and coword phenomena (ACCP). It is necessary to determine whether chord network charts (CNCs) can provide a better understanding of ACCP, thus requiring clarification. This study aimed to achieve 2 objectives: evaluate global research trends in AI in intensive care medicine on publication outputs, coauthorships between nations, citations, and co-occurrences of keywords; and demonstrate the use of CNCs for ACCP in bibliometric analysis. METHODS The web of science database was searched for a total of 1992 documents published between 2013 and 2022. The document type was limited to articles and article reviews, and titles and abstracts were screened for eligibility. The characteristics of the publications, including preferred journals, leading research countries, international collaborations, top institutions, and major keywords, were analyzed using the category-journal rank-authorship-L-index score and trend analysis. The 100 most highly cited articles are also listed in detail. RESULTS Between 2018 and 2022, there was a sharp increase in publications, which accounted for 92.8% (1849/1992) of all papers included in the study. The United States and China were responsible for nearly 50% (936/1992) of the total publications. The leading countries, institutes, departments, authors, and journals in terms of publications were the US, Massachusetts Gen Hosp (US), Medical School, Zhongheng Zhang (China), and Science Reports. The top 3 primary keywords denoting research hotspots for AI in critically ill patients were mortality, model, and intensive care unit, with mortality having the highest burst strength (4.49). The keywords risk and system showed the highest growth trend (0.98) in counts over the past 4 years. CONCLUSIONS This study provides valuable insights into the potential for ACCP and future research opportunities. For AI-based clinical research to become widely accepted in critical care practice, collaborative research efforts are necessary to strengthen the maturity and robustness of AI-driven models using CNCs for display.
Collapse
Affiliation(s)
- Teng-Yun Cheng
- Department of Emergency Medicine, Chi-Mei Medical Center, Liouying, Tainan, Taiwan
| | - Sam Yu-Chieh Ho
- Department of Emergency Medicine, Chi-Mei Medical Center, Tainan, Taiwan
| | - Tsair-Wei Chien
- Department of Medical Research, Chi-Mei Medical Center, Tainan, Taiwan
| | - Willy Chou
- Department of Physical Medicine and Rehabilitation, Chi Mei Medical Center, Jiali, Tainan, Taiwan
- Department of Physical Medicine and Rehabilitation, Chung San Medical University Hospital, Taichung, Taiwan
| |
Collapse
|
8
|
Jazayeri A, Yang CC, Capan M. Frequent temporal patterns of physiological and biological biomarkers and their evolution in sepsis. Artif Intell Med 2023; 143:102576. [PMID: 37673556 PMCID: PMC10483113 DOI: 10.1016/j.artmed.2023.102576] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2021] [Revised: 02/01/2023] [Accepted: 05/09/2023] [Indexed: 09/08/2023]
Abstract
Sepsis is one of the most challenging health conditions worldwide, with relatively high incidence and mortality rates. It is shown that preventing sepsis is the key to avoid potentially irreversible organ dysfunction. However, data-driven early identification of sepsis is challenging as sepsis shares signs and symptoms with other health conditions. This paper adopts a temporal pattern mining approach to identify frequent temporal and evolving patterns of physiological and biological biomarkers in sepsis patients. We show that using these frequent patterns as features for classifying sepsis and non-sepsis patients can improve the prediction accuracy and performance up to 7%. Most of the temporal modeling approaches adopted in the sepsis literature are based on deep learning methods. Although these approaches produce high accuracy, they generally have limited model explainability and interpretability. Using the adopted methods in this study, we could identify the most important features contributing to the patients' sepsis incidence, such as fluctuations in platelet, lactate, and creatinine, or evolution of patterns including renal and metabolic organ systems, and consequently, enhance the findings' clinical interpretability.
Collapse
Affiliation(s)
- Ali Jazayeri
- College of Computing & Informatics, Drexel University, Philadelphia, PA, United States of America.
| | - Christopher C Yang
- College of Computing & Informatics, Drexel University, Philadelphia, PA, United States of America
| | - Muge Capan
- Mechanical and Industrial Engineering, College of Engineering, University of Massachusetts Amherst, Amherst, MA, United States of America
| |
Collapse
|
9
|
van den Berg MAM, Medina OOAG, Loohuis IIP, van der Flier MM, Dudink JJ, Benders MMJNL, Bartels RRT, Vijlbrief DDC. Development and clinical impact assessment of a machine-learning model for early prediction of late-onset sepsis. Comput Biol Med 2023; 163:107156. [PMID: 37369173 DOI: 10.1016/j.compbiomed.2023.107156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Revised: 05/24/2023] [Accepted: 06/07/2023] [Indexed: 06/29/2023]
Abstract
BACKGROUND AND AIM Preterm infants are prone to neonatal infections such as late-onset sepsis (LOS). The consequences of LOS can be severe and potentially life-threatening. Unfortunately, LOS often presents with unspecific symptoms, and early screening laboratory tests have limited diagnostic value and are often late. This study aimed to build a predictive algorithm to aid doctors in the early detection of LOS in very preterm infants. METHODS In a retrospective cohort study, all consecutively admitted preterm infants (GA ≤ 32 weeks) from 2008 until 2019 were included. They were classified as LOS or control according to blood culture results, currently the gold standard. To generate features, routine and continuously measured oxygen saturation and heart rate data with a minute-by-minute sampling rate were extracted from electronic medical records. Care was taken not to include variables indicative of existing LOS suspicion. The timing of a positive blood culture served as a proxy for LOS-onset. An equivalent timestamp was generated in gestational-age-matched control patients without a positive blood culture. Three machine learning (ML) techniques (generalized additive models, logistic regression, and XGBoost) were used to build a classification algorithm. To simulate the performance of the algorithm in clinical practice, a simulation using multiple alarm thresholds was performed on hourly predictions for the total hospitalization period. RESULTS 292 infants with LOS were matched to 1497 controls. The median gestational age before matching was 28.1 and 30.3 weeks, respectively. Evaluation of the overall discriminative power of the LR algorithm yielded an AUC of 0.73 (p < 0.05) at the moment of clinical suspicion (t = 0). In the longitudinal simulation, our algorithm detects LOS in at least 47% of the patients before clinical suspicion without exceeding the alarm fatigue threshold of 3 alarms per day. Furthermore, medical experts evaluated the algorithm as clinically relevant regarding the feature contributions in the model explanations. CONCLUSIONS An ML algorithm was trained for the early detection of LOS. Performance was evaluated on both prediction horizons and in a clinical impact simulation. To the best of our knowledge, our assessment of clinical impact with a retrospective simulation on longitudinal data is the most extensive in the literature on LOS prediction to date. The clinically relevant algorithm, based on routinely collected data, can potentially accelerate clinical decisions in the early detection of LOS, even with limited inputs.
Collapse
Affiliation(s)
- Merel A M van den Berg
- Department of Neonatology, Wilhelmina Children's Hospital, UMC Utrecht, Utrecht, the Netherlands
| | | | | | - Michiel M van der Flier
- Department of Pediatric Infectious Disease, Wilhelmina Children's Hospital, UMC Utrecht, Utrecht, the Netherlands
| | - Jeroen J Dudink
- Department of Neonatology, Wilhelmina Children's Hospital, UMC Utrecht, Utrecht, the Netherlands
| | - Manon M J N L Benders
- Department of Neonatology, Wilhelmina Children's Hospital, UMC Utrecht, Utrecht, the Netherlands
| | | | - Daniel D C Vijlbrief
- Department of Neonatology, Wilhelmina Children's Hospital, UMC Utrecht, Utrecht, the Netherlands.
| |
Collapse
|
10
|
Islam KR, Prithula J, Kumar J, Tan TL, Reaz MBI, Sumon MSI, Chowdhury MEH. Machine Learning-Based Early Prediction of Sepsis Using Electronic Health Records: A Systematic Review. J Clin Med 2023; 12:5658. [PMID: 37685724 PMCID: PMC10488449 DOI: 10.3390/jcm12175658] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 08/13/2023] [Accepted: 08/23/2023] [Indexed: 09/10/2023] Open
Abstract
BACKGROUND Sepsis, a life-threatening infection-induced inflammatory condition, has significant global health impacts. Timely detection is crucial for improving patient outcomes as sepsis can rapidly progress to severe forms. The application of machine learning (ML) and deep learning (DL) to predict sepsis using electronic health records (EHRs) has gained considerable attention for timely intervention. METHODS PubMed, IEEE Xplore, Google Scholar, and Scopus were searched for relevant studies. All studies that used ML/DL to detect or early-predict the onset of sepsis in the adult population using EHRs were considered. Data were extracted and analyzed from all studies that met the criteria and were also evaluated for their quality. RESULTS This systematic review examined 1942 articles, selecting 42 studies while adhering to strict criteria. The chosen studies were predominantly retrospective (n = 38) and spanned diverse geographic settings, with a focus on the United States. Different datasets, sepsis definitions, and prevalence rates were employed, necessitating data augmentation. Heterogeneous parameter utilization, diverse model distribution, and varying quality assessments were observed. Longitudinal data enabled early sepsis prediction, and quality criteria fulfillment varied, with inconsistent funding-article quality correlation. CONCLUSIONS This systematic review underscores the significance of ML/DL methods for sepsis detection and early prediction through EHR data.
Collapse
Affiliation(s)
- Khandaker Reajul Islam
- Department of Physiology, Faculty of Medicine, University Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Johayra Prithula
- Department of Electrical and Electronics Engineering, University of Dhaka, Dhaka 1000, Bangladesh
| | - Jaya Kumar
- Department of Physiology, Faculty of Medicine, University Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Toh Leong Tan
- Department of Emergency Medicine, Faculty of Medicine, Universiti Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Mamun Bin Ibne Reaz
- Department of Electrical and Electronic Engineering, Independent University, Bangladesh Bashundhara, Dhaka 1229, Bangladesh
| | - Md. Shaheenur Islam Sumon
- Department of Biomedical Engineering, Military Institute of Science and Technology (MIST), Dhaka 1216, Bangladesh
| | | |
Collapse
|
11
|
Alanazi A, Aldakhil L, Aldhoayan M, Aldosari B. Machine Learning for Early Prediction of Sepsis in Intensive Care Unit (ICU) Patients. MEDICINA (KAUNAS, LITHUANIA) 2023; 59:1276. [PMID: 37512087 PMCID: PMC10385427 DOI: 10.3390/medicina59071276] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Revised: 06/26/2023] [Accepted: 07/07/2023] [Indexed: 07/30/2023]
Abstract
Background and Objectives: Early detection of sepsis is crucial and can save lives. However, identifying sepsis early and accurately remains a difficult task in the medical field. This study aims to investigate a new machine-learning approach. By analyzing the clinical laboratory results and vital signs of adult patients in the ICU, this approach can predict and detect the initial signs of sepsis. Materials and Methods: To examine survival rates and predict outcomes, the study utilized several models, including the proportional hazards model and data mining algorithms. We analyzed data from the BESTCare database at KAMC, with a focus on patients aged 14 and older who were admitted to the ICU between April and October 2018. We conducted a thorough analysis of the medical records of a total of 1182 patients who were diagnosed with sepsis. Results: We studied two approaches to predict sepsis in ICU patients. The regression model utilizing survival analysis showed moderate predictive ability, emphasizing the importance of only three factors-time (from sepsis to an outcome; discharge or death), lactic acid, and temperature-had a significant p-value (p = 0.000568, p = 0.01, p = 0.02, respectively). Other data mining algorithms may have limitations due to their assumptions of variable independence and linear classification nature. Conclusions: To achieve progress and accuracy in the field of sepsis prediction, it is important to continuously strive for improvement. By meticulously cleaning and selecting data attributes, we can create a strong foundation for future advancements in this area.
Collapse
Affiliation(s)
- Abdullah Alanazi
- Department of Health Informatics, College of Public Health and Health Informatics, King Saud Ibn Abdulaziz University for Health Sciences, Riyadh 11481, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh 14611, Saudi Arabia
| | - Lujain Aldakhil
- Department of Health Informatics, College of Public Health and Health Informatics, King Saud Ibn Abdulaziz University for Health Sciences, Riyadh 11481, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh 14611, Saudi Arabia
| | - Mohammed Aldhoayan
- Department of Health Informatics, College of Public Health and Health Informatics, King Saud Ibn Abdulaziz University for Health Sciences, Riyadh 11481, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh 14611, Saudi Arabia
| | - Bakheet Aldosari
- Department of Health Informatics, College of Public Health and Health Informatics, King Saud Ibn Abdulaziz University for Health Sciences, Riyadh 11481, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh 14611, Saudi Arabia
| |
Collapse
|
12
|
Li Y, Wang Y. Temporal convolution attention model for sepsis clinical assistant diagnosis prediction. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:13356-13378. [PMID: 37501491 DOI: 10.3934/mbe.2023595] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/29/2023]
Abstract
Sepsis is an organ failure disease caused by an infection acquired in an intensive care unit (ICU), which leads to a high mortality rate. Developing intelligent monitoring and early warning systems for sepsis is a key research area in the field of smart healthcare. Early and accurate identification of patients at high risk of sepsis can help doctors make the best clinical decisions and reduce the mortality rate of patients with sepsis. However, the scientific understanding of sepsis remains inadequate, leading to slow progress in sepsis research. With the accumulation of electronic medical records (EMRs) in hospitals, data mining technologies that can identify patient risk patterns from the vast amount of sepsis-related EMRs and the development of smart surveillance and early warning models show promise in reducing mortality. Based on the Medical Information Mart for Intensive Care Ⅲ, a massive dataset of ICU EMRs published by MIT and Beth Israel Deaconess Medical Center, we propose a Temporal Convolution Attention Model for Sepsis Clinical Assistant Diagnosis Prediction (TCASP) to predict the incidence of sepsis infection in ICU patients. First, sepsis patient data is extracted from the EMRs. Then, the incidence of sepsis is predicted based on various physiological features of sepsis patients in the ICU. Finally, the TCASP model is utilized to predict the time of the first sepsis infection in ICU patients. The experiments show that the proposed model achieves an area under the receiver operating characteristic curve (AUROC) score of 86.9% (an improvement of 6.4% ) and an area under the precision-recall curve (AUPRC) score of 63.9% (an improvement of 3.9% ) compared to five state-of-the-art models.
Collapse
Affiliation(s)
- Yong Li
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| | - Yang Wang
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| |
Collapse
|
13
|
van der Vegt AH, Scott IA, Dermawan K, Schnetler RJ, Kalke VR, Lane PJ. Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework. J Am Med Inform Assoc 2023:7161075. [PMID: 37172264 DOI: 10.1093/jamia/ocad075] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 04/04/2023] [Accepted: 04/23/2023] [Indexed: 05/14/2023] Open
Abstract
OBJECTIVE To retrieve and appraise studies of deployed artificial intelligence (AI)-based sepsis prediction algorithms using systematic methods, identify implementation barriers, enablers, and key decisions and then map these to a novel end-to-end clinical AI implementation framework. MATERIALS AND METHODS Systematically review studies of clinically applied AI-based sepsis prediction algorithms in regard to methodological quality, deployment and evaluation methods, and outcomes. Identify contextual factors that influence implementation and map these factors to the SALIENT implementation framework. RESULTS The review identified 30 articles of algorithms applied in adult hospital settings, with 5 studies reporting significantly decreased mortality post-implementation. Eight groups of algorithms were identified, each sharing a common algorithm. We identified 14 barriers, 26 enablers, and 22 decision points which were able to be mapped to the 5 stages of the SALIENT implementation framework. DISCUSSION Empirical studies of deployed sepsis prediction algorithms demonstrate their potential for improving care and reducing mortality but reveal persisting gaps in existing implementation guidance. In the examined publications, key decision points reflecting real-word implementation experience could be mapped to the SALIENT framework and, as these decision points appear to be AI-task agnostic, this framework may also be applicable to non-sepsis algorithms. The mapping clarified where and when barriers, enablers, and key decisions arise within the end-to-end AI implementation process. CONCLUSIONS A systematic review of real-world implementation studies of sepsis prediction algorithms was used to validate an end-to-end staged implementation framework that has the ability to account for key factors that warrant attention in ensuring successful deployment, and which extends on previous AI implementation frameworks.
Collapse
Affiliation(s)
- Anton H van der Vegt
- Queensland Digital Health Centre, The University of Queensland, Brisbane, Queensland, Australia
| | - Ian A Scott
- Department of Internal Medicine and Clinical Epidemiology, Princess Alexandra Hospital, Brisbane, Australia
| | - Krishna Dermawan
- Centre for Information Resilience, The University of Queensland, St Lucia, Australia
| | - Rudolf J Schnetler
- School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Australia
| | - Vikrant R Kalke
- Patient Safety and Quality, Clinical Excellence Queensland, Queensland Health, Brisbane, Australia
| | - Paul J Lane
- Safety Quality & Innovation, The Prince Charles Hospital, Queensland Health, Brisbane, Australia
| |
Collapse
|
14
|
Duan Y, Huo J, Chen M, Hou F, Yan G, Li S, Wang H. Early prediction of sepsis using double fusion of deep features and handcrafted features. APPL INTELL 2023; 53:1-17. [PMID: 36685641 PMCID: PMC9843111 DOI: 10.1007/s10489-022-04425-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/21/2022] [Indexed: 01/19/2023]
Abstract
Sepsis is a life-threatening medical condition that is characterized by the dysregulated immune system response to infections, having both high morbidity and mortality rates. Early prediction of sepsis is critical to the decrease of mortality. This paper presents a novel early warning model called Double Fusion Sepsis Predictor (DFSP) for sepsis onset. DFSP is a double fusion framework that combines the benefits of early and late fusion strategies. First, a hybrid deep learning model that combines both the convolutional and recurrent neural networks to extract deep features is proposed. Second, deep features and handcrafted features, such as clinical scores, are concatenated to build the joint feature representation (early fusion). Third, several tree-based models based on joint feature representation are developed to generate the risk scores of sepsis onset that are combined with an End-to-End neural network for final sepsis detection (late fusion). To evaluate DFSP, a retrospective study was conducted, which included patients admitted to the ICUs of a hospital in Shanghai China. The results demonstrate that the DFSP outperforms state-of-the-art approaches in early sepsis prediction.
Collapse
Affiliation(s)
- Yongrui Duan
- School of Economics & Management, Tongji University, Shanghai, China
| | - Jiazhen Huo
- School of Economics & Management, Tongji University, Shanghai, China
| | - Mingzhou Chen
- School of Economics & Management, Tongji University, Shanghai, China
| | - Fenggang Hou
- Department of Oncology, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai, China
| | - Guoliang Yan
- Department of Geriatrics, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai, China
| | - Shufang Li
- Emergency Department, Shuguang Hospital Affiliated to Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Haihui Wang
- Department of Geriatrics, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai, China
| |
Collapse
|
15
|
Tan Y, Young M, Girish A, Hu B, Kurian Z, Greenstein JL, Kim H, Winslow RL, Fackler J, Bergmann J. Predicting respiratory decompensation in mechanically ventilated adult ICU patients. Front Physiol 2023; 14:1125991. [PMID: 37123253 PMCID: PMC10140580 DOI: 10.3389/fphys.2023.1125991] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Accepted: 04/06/2023] [Indexed: 05/02/2023] Open
Abstract
Introduction: Mechanical ventilation is a life-saving treatment in the Intensive Care Unit (ICU), but often causes patients to be at risk of further respiratory complication. We created a statistical model utilizing electronic health record and physiologic vitals data to predict the Center for Disease Control and Prevention (CDC) defined Ventilator Associated Complications (VACs). Further, we evaluated the effect of data temporal resolution and feature generation method choice on the accuracy of such a constructed model. Methods: We constructed a random forest model to predict occurrence of VACs using health records and chart events from adult patients in the Medical Information Mart for Intensive Care III (MIMIC-III) database. We trained the machine learning models on two patient populations of 1921 and 464 based on low and high frequency data availability. Model features were generated using both basic statistical summaries and tsfresh, a python library that generates a large number of derived time-series features. Classification to determine whether a patient will experience VAC one hour after 35 h of ventilation was performed using a random forest classifier. Two different sample spaces conditioned on five varying feature extraction techniques were evaluated to identify the most optimal selection of features resulting in the best VAC discrimination. Each dataset was assessed using K-folds cross-validation (k = 10), giving average area under the receiver operating characteristic curves (AUROCs) and accuracies. Results: After feature selection, hyperparameter tuning, and feature extraction, the best performing model used automatically generated features on high frequency data and achieved an average AUROC of 0.83 ± 0.11 and an average accuracy of 0.69 ± 0.10. Discussion: Results show the potential viability of predicting VACs using machine learning, and indicate that higher-resolution data and the larger feature set generated by tsfresh yield better AUROCs compared to lower-resolution data and manual statistical features.
Collapse
Affiliation(s)
- Yvette Tan
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Michael Young
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Akanksha Girish
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Beini Hu
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Zina Kurian
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Joseph L. Greenstein
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
- Institute for Computational Medicine, Johns Hopkins University, Baltimore, MD, United States
| | - Han Kim
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Raimond L Winslow
- Institute for Computational Medicine, Johns Hopkins University, Baltimore, MD, United States
| | - James Fackler
- Department of Anesthesiology and Critical Care Medicine, School of Medicine, Johns Hopkins University, Baltimore, MD, United States
| | - Jules Bergmann
- Department of Anesthesiology and Critical Care Medicine, School of Medicine, Johns Hopkins University, Baltimore, MD, United States
- *Correspondence: Jules Bergmann,
| |
Collapse
|
16
|
Petersson L, Vincent K, Svedberg P, Nygren JM, Larsson I. Ethical considerations in implementing AI for mortality prediction in the emergency department: Linking theory and practice. Digit Health 2023; 9:20552076231206588. [PMID: 37829612 PMCID: PMC10566278 DOI: 10.1177/20552076231206588] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/21/2023] [Indexed: 10/14/2023] Open
Abstract
Background Artificial intelligence (AI) is predicted to be a solution for improving healthcare, increasing efficiency, and saving time and recourses. A lack of ethical principles for the use of AI in practice has been highlighted by several stakeholders due to the recent attention given to it. Research has shown an urgent need for more knowledge regarding the ethical implications of AI applications in healthcare. However, fundamental ethical principles may not be sufficient to describe ethical concerns associated with implementing AI applications. Objective The aim of this study is twofold, (1) to use the implementation of AI applications to predict patient mortality in emergency departments as a setting to explore healthcare professionals' perspectives on ethical issues in relation to ethical principles and (2) to develop a model to guide ethical considerations in AI implementation in healthcare based on ethical theory. Methods Semi-structured interviews were conducted with 18 participants. The abductive approach used to analyze the empirical data consisted of four steps alternating between inductive and deductive analyses. Results Our findings provide an ethical model demonstrating the need to address six ethical principles (autonomy, beneficence, non-maleficence, justice, explicability, and professional governance) in relation to ethical theories defined as virtue, deontology, and consequentialism when AI applications are to be implemented in clinical practice. Conclusions Ethical aspects of AI applications are broader than the prima facie principles of medical ethics and the principle of explicability. Ethical aspects thus need to be viewed from a broader perspective to cover different situations that healthcare professionals, in general, and physicians, in particular, may face when using AI applications in clinical practice.
Collapse
Affiliation(s)
- Lena Petersson
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Kalista Vincent
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Petra Svedberg
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Jens M Nygren
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| | - Ingrid Larsson
- School of Health and Welfare, Halmstad University, Halmstad, Sweden
| |
Collapse
|
17
|
Chen Q, Li R, Lin C, Lai C, Huang Y, Lu W, Li L. SEPRES: Intensive Care Unit Clinical Data Integration System to Predict Sepsis. Appl Clin Inform 2023; 14:65-75. [PMID: 36452980 PMCID: PMC9876660 DOI: 10.1055/a-1990-3037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Accepted: 11/28/2022] [Indexed: 12/05/2022] Open
Abstract
BACKGROUND The lack of information interoperability between different devices and systems in the intensive care unit (ICU) hinders further utilization of data, especially for early warning of specific diseases in the ICU. OBJECTIVES We aimed to establish a data integration system. Based on this system, the sepsis prediction module was added to compose the Sepsis PREdiction System (SEPRES), where real-time early warning of sepsis can be implemented at the bedside in the ICU. METHODS Data are collected from bedside devices through the integration hub and uploaded to the integration system through the local area network. The data integration system was designed to integrate vital signs data, laboratory data, ventilator data, demographic data, pharmacy data, nursing data, etc. from multiple medical devices and systems. It integrates, standardizes, and stores information, making the real-time inference of the early warning module possible. The built-in sepsis early warning module can detect the onset of sepsis within 5 hours preceding at most. RESULTS Our data integration system has already been deployed in Ruijin Hospital, confirming the feasibility of our system. CONCLUSION We highlight that SEPRES has the potential to improve ICU management by helping medical practitioners identify at-sepsis-risk patients and prepare for timely diagnosis and intervention.
Collapse
Affiliation(s)
- Qiyu Chen
- Division of Applied Mathematics, Fudan University, Shanghai, China
| | - Ranran Li
- Department of Critical Care Medicine, Shanghai Jiaotong University School of Medicine, Ruijin Hospital, Shanghai, China
| | - ChihChe Lin
- Department of Intelligent Medical Products, Shanghai Electric Group Co., Ltd. Central Academe, Shanghai, China
| | - Chiming Lai
- Department of Intelligent Medical Products, Shanghai Electric Group Co., Ltd. Central Academe, Shanghai, China
| | - Yaling Huang
- Department of Intelligent Medical Products, Shanghai Electric Group Co., Ltd. Central Academe, Shanghai, China
| | - Wenlian Lu
- Division of Applied Mathematics, Fudan University, Shanghai, China
| | - Lei Li
- Department of Critical Care Medicine, Shanghai Jiaotong University School of Medicine, Ruijin Hospital, Shanghai, China
| |
Collapse
|
18
|
DeShon B, Dummitt B, Allen J, Yount B. Prediction of sepsis onset in hospital admissions using survival analysis. J Clin Monit Comput 2022; 36:1611-1619. [PMID: 35076834 DOI: 10.1007/s10877-022-00804-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Accepted: 01/03/2022] [Indexed: 11/25/2022]
Abstract
To determine the efficacy of modern survival analysis methods for predicting sepsis onset in ICU, emergency, medical/surgical, and TCU departments. We performed a retrospective analysis on ICU, med/surg, ED, and TCU cases from multiple Mercy Health hospitals from August 2018 to March 2020. Patients in these departments were monitored by the Mercy Virtual vSepsis team and sepsis cases were determined and documented in the Mercy EHR via a rule-based engine utilizing clinical data. We used survival-based modeling methods to predict sepsis onset in these cases. The three survival methods that were used to predict the onset of severe sepsis and septic shock produced AUC values > 0.85 and each provided a median lead time of > 20 h prior to disease onset. This methodology improves upon previous work by demonstrating excellent model performance when generalizing survival-based prediction methods to both severe sepsis and septic shock as well as non-ICU departments.IRB InformationTrial Registration ID: 1,532,327-1.Trial Effective Date: 12/02/2019.
Collapse
Affiliation(s)
- Brandon DeShon
- Mercy Health, 14528 S. Outer Forty, Chesterfield, MO, 63017, USA.
| | - Benjamin Dummitt
- Mercy Health, 14528 S. Outer Forty, Chesterfield, MO, 63017, USA
| | - Joshua Allen
- Mercy Health, 14528 S. Outer Forty, Chesterfield, MO, 63017, USA
| | - Byron Yount
- Mercy Health, 14528 S. Outer Forty, Chesterfield, MO, 63017, USA
| |
Collapse
|
19
|
Lombardi S, Partanen P, Francia P, Calamai I, Deodati R, Luchini M, Spina R, Bocchi L. Classifying sepsis from photoplethysmography. Health Inf Sci Syst 2022; 10:30. [PMID: 36330224 PMCID: PMC9622958 DOI: 10.1007/s13755-022-00199-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2022] [Accepted: 10/09/2022] [Indexed: 11/05/2022] Open
Abstract
Sepsis is a life-threatening organ dysfunction. It is caused by a dysregulated immune response to an infection and is one of the leading causes of death in the intensive care unit (ICU). Early detection and treatment of sepsis can increase the survival rate of patients. The use of devices such as the photoplethysmograph could allow the early evaluation in addition to continuous monitoring of septic patients. The aim of this study was to verify the possibility of detecting sepsis in patients from whom the photoplethysmographic signal was acquired via a pulse oximeter. In this work, we developed a deep learning-based model for sepsis identification. The model takes a single input, the photoplethysmographic signal acquired by pulse oximeter, and performs a binary classification between septic and nonseptic samples. To develop the method, we used MIMIC-III database, which contains data from ICU patients. Specifically, the selected dataset includes 85 septic subjects and 101 control subjects. The PPG signals acquired from these patients were segmented, processed and used as input for the developed model with the aim of identifying sepsis. The proposed method achieved an accuracy of 76.37% with a sensitivity of 70.95% and a specificity of 81.04% on the test set. As regards the ROC curve, the Area Under Curve reached a value of 0.842. The results of this study indicate how the plethysmographic signal can be used as a warning sign for the early detection of sepsis with the aim of reducing the time for diagnosis and therapeutic intervention. Furthermore, the proposed method is suitable for integration in continuous patient monitoring.
Collapse
Affiliation(s)
- Sara Lombardi
- Department of Information Engineering, University of Florence, Via S. Marta, 3, 50139 Florence, Italy
| | - Petri Partanen
- Faculty of Information Technology and Electrical Engineering, University of Oulu, Pentti Kaiteran katu 1, 90570 Oulu, Finland
| | - Piergiorgio Francia
- Department of Information Engineering, University of Florence, Via S. Marta, 3, 50139 Florence, Italy
| | - Italo Calamai
- S.O.C. Anestesia e Rianimazione, Ospedale S. Giuseppe, viale Giovanni Boccaccio, 16, 50053 Empoli, Italy
| | - Rossella Deodati
- S.O.C. Anestesia e Rianimazione, Ospedale S. Giuseppe, viale Giovanni Boccaccio, 16, 50053 Empoli, Italy
| | - Marco Luchini
- S.O.C. Anestesia e Rianimazione, Ospedale S. Giuseppe, viale Giovanni Boccaccio, 16, 50053 Empoli, Italy
| | - Rosario Spina
- S.O.C. Anestesia e Rianimazione, Ospedale S. Giuseppe, viale Giovanni Boccaccio, 16, 50053 Empoli, Italy
| | - Leonardo Bocchi
- Department of Information Engineering, University of Florence, Via S. Marta, 3, 50139 Florence, Italy
| |
Collapse
|
20
|
Explainable Artificial Intelligence Helps in Understanding the Effect of Fibronectin on Survival of Sepsis. Cells 2022; 11:cells11152433. [PMID: 35954279 PMCID: PMC9368279 DOI: 10.3390/cells11152433] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Revised: 07/30/2022] [Accepted: 08/03/2022] [Indexed: 11/25/2022] Open
Abstract
Fibronectin (FN) plays an essential role in the host’s response to infection. In previous studies, a significant decrease in the FN level was observed in sepsis; however, it has not been clearly elucidated how this parameter affects the patient’s survival. To better understand the relationship between FN and survival, we utilized innovative approaches from the field of explainable machine learning, including local explanations (Break Down, Shapley Additive Values, Ceteris Paribus), to understand the contribution of FN to predicting individual patient survival. The methodology provides new opportunities to personalize informative predictions for patients. The results showed that the most important indicators for predicting survival in sepsis were INR, FN, age, and the APACHE II score. ROC curve analysis showed that the model’s successful classification rate was 0.92, its sensitivity was 0.92, its positive predictive value was 0.76, and its accuracy was 0.79. To illustrate these possibilities, we have developed and shared a web-based risk calculator for exploring individual patient risk. The web application can be continuously updated with new data in order to further improve the model.
Collapse
|
21
|
Shen J, Casie Chetty S, Shokouhi S, Maharjan J, Chuba Y, Calvert J, Mao Q. Massive external validation of a machine learning algorithm to predict pulmonary embolism in hospitalized patients. Thromb Res 2022; 216:14-21. [DOI: 10.1016/j.thromres.2022.05.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2022] [Revised: 05/26/2022] [Accepted: 05/30/2022] [Indexed: 10/18/2022]
|
22
|
Machine Learning Models for Early Prediction of Sepsis on Large Healthcare Datasets. ELECTRONICS 2022. [DOI: 10.3390/electronics11091507] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Sepsis is a highly lethal syndrome with heterogeneous clinical manifestation that can be hard to identify and treat. Early diagnosis and appropriate treatment are critical to reduce mortality and promote survival in suspected cases and improve the outcomes. Several screening prediction systems have been proposed for evaluating the early detection of patient deterioration, but the efficacy is still limited at individual level. The increasing amount and the versatility of healthcare data suggest implementing machine learning techniques to develop models for predicting sepsis. This work presents an experimental study of some machine-learning-based models for sepsis prediction considering vital signs, laboratory test results, and demographics using Medical Information Mart for Intensive Care III (MIMIC-III) (v1.4), a publicly available dataset. The experimental results demonstrate an overall higher performance of machine learning models over the commonly used Sequential Organ Failure Assessment (SOFA) and Quick SOFA (qSOFA) scoring systems at the time of sepsis onset.
Collapse
|
23
|
Liu S, Fu B, Wang W, Liu M, Sun X. Dynamic sepsis prediction for intensive care unit patients using XGBoost-based model with novel time-dependent features. IEEE J Biomed Health Inform 2022; 26:4258-4269. [PMID: 35503852 DOI: 10.1109/jbhi.2022.3171673] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Sepsis is a systemic inflammatory response caused by pathogens such as bacteria. Because its pathogenesis is not clear, the clinical manifestations of patients vary greatly, and the alarming incidence and mortality pose a great threat to patients and medical systems, especially in Intensive Care Unit (ICU). The traditional judgment criteria have the problem of low specificity. Artificial intelligence models could greatly improve the accuracy of sepsis prediction and judgment. Based on the XGBoost machine learning framework taken demographic, vital signs, laboratory tests and medical intervention data as input, this paper proposes a novel model for dynamically predicting sepsis and assessing risk. To realize the model, two methods for feature construction are introduced. For the observed time series data of vital signs and laboratory tests, time-dependent method performs to construct the time- dependent characteristics after statistical screening. For the clinical intervention data, statistical counting method is applied to construct count-dependent characteristics. Moreover, a new objective function is proposed for the XGBoost framework, and the first-order and second-order gradient of the objective function are also given for model training. Compared with the state-of-the-art methods at present, the proposed model has the best performance, with AUROC improved by 5.4% on the MIMIC-III dataset and 2.1% on PhysioNet Challenge 2019 dataset. The data processing and training methods of this model can be conveniently applied in different electronic health record systems, and has a wide application prospect.
Collapse
|
24
|
Sadasivuni S, Saha M, Bhatia N, Banerjee I, Sanyal A. Fusion of fully integrated analog machine learning classifier with electronic medical records for real-time prediction of sepsis onset. Sci Rep 2022; 12:5711. [PMID: 35383233 PMCID: PMC8983688 DOI: 10.1038/s41598-022-09712-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Accepted: 03/28/2022] [Indexed: 12/20/2022] Open
Abstract
The objective of this work is to develop a fusion artificial intelligence (AI) model that combines patient electronic medical record (EMR) and physiological sensor data to accurately predict early risk of sepsis. The fusion AI model has two components—an on-chip AI model that continuously analyzes patient electrocardiogram (ECG) data and a cloud AI model that combines EMR and prediction scores from on-chip AI model to predict fusion sepsis onset score. The on-chip AI model is designed using analog circuits for sepsis prediction with high energy efficiency for integration with resource constrained wearable device. Combination of EMR and sensor physiological data improves prediction performance compared to EMR or physiological data alone, and the late fusion model has an accuracy of 93% in predicting sepsis 4 h before onset. The key differentiation of this work over existing sepsis prediction literature is the use of single modality patient vital (ECG) and simple demographic information, instead of comprehensive laboratory test results and multiple vital signs. Such simple configuration and high accuracy makes our solution favorable for real-time, at-home use for self-monitoring.
Collapse
Affiliation(s)
| | - Monjoy Saha
- Department of Biomedical Informatics, Emory University, Atlanta, GA, 30322, USA
| | - Neal Bhatia
- Division of Cardiology, Emory University School of Medicine, Atlanta, GA, 30322, USA
| | - Imon Banerjee
- Department of Biomedical Informatics, Emory University, Atlanta, GA, 30322, USA.,Department of Radiology, Emory University, Atlanta, GA, 30322, USA
| | - Arindam Sanyal
- Electrical Engineering, University at Buffalo, Buffalo, NY, 14260, USA.
| |
Collapse
|
25
|
Sepsis prediction in intensive care unit based on genetic feature optimization and stacked deep ensemble learning. Neural Comput Appl 2022. [DOI: 10.1007/s00521-021-06631-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
26
|
Kim JK, Ahn W, Park S, Lee SH, Kim L. Early Prediction of Sepsis Onset Using Neural Architecture Search Based on Genetic Algorithms. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19042349. [PMID: 35206537 PMCID: PMC8872017 DOI: 10.3390/ijerph19042349] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/13/2022] [Revised: 02/13/2022] [Accepted: 02/14/2022] [Indexed: 12/14/2022]
Abstract
Sepsis is a life-threatening condition with a high mortality rate. Early prediction and treatment are the most effective strategies for increasing survival rates. This paper proposes a neural architecture search (NAS) model to predict the onset of sepsis with a low computational cost and high search performance by applying a genetic algorithm (GA). The proposed model shares the weights of all possible connection nodes internally within the neural network. Externally, the search cost is reduced through the weight-sharing effect between the genotypes of the GA. A predictive analysis was performed using the Medical Information Mart for Intensive Care III (MIMIC-III), a medical time-series dataset, with the primary objective of predicting sepsis onset 3 h before occurrence. In addition, experiments were conducted under various prediction times (0-12 h) for comparison. The proposed model exhibited an area under the receiver operating characteristic curve (AUROC) score of 0.94 (95% CI: 0.92-0.96) for 3 h, which is 0.31-0.26 higher than the scores obtained using the Sequential Organ Failure Assessment (SOFA), quick SOFA (qSOFA), and Simplified Acute Physiology Score (SAPS) II scoring systems. Furthermore, the proposed model exhibited a 12% improvement in the AUROC value over a simple model based on the long short-term memory neural network. Additionally, it is not only optimally searchable for sepsis onset prediction, but also outperforms conventional models that use similar predictive purposes and datasets. Notably, it is sufficiently robust to shape changes in the input data and has less structural dependence.
Collapse
Affiliation(s)
- Jae Kwan Kim
- Center for Bionics, Korea Institute of Science and Technology, Seoul 02792, Korea
- School of Mechanical Engineering, Yonsei University, Seoul 03722, Korea
| | - Wonbin Ahn
- Applied AI Research Lab, LG AI Research, Seoul 07796, Korea
| | - Sangin Park
- Center for Bionics, Korea Institute of Science and Technology, Seoul 02792, Korea
| | - Soo-Hong Lee
- School of Mechanical Engineering, Yonsei University, Seoul 03722, Korea
| | - Laehyun Kim
- Center for Bionics, Korea Institute of Science and Technology, Seoul 02792, Korea
- Department of HY-KIST Bio-Convergence, Hanyang University, Seoul 04763, Korea
| |
Collapse
|
27
|
Chalumuri YR, Kimball JP, Mousavi A, Zia JS, Rolfes C, Parreira JD, Inan OT, Hahn JO. Classification of Blood Volume Decompensation State via Machine Learning Analysis of Multi-Modal Wearable-Compatible Physiological Signals. SENSORS (BASEL, SWITZERLAND) 2022; 22:1336. [PMID: 35214238 PMCID: PMC8963055 DOI: 10.3390/s22041336] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/21/2021] [Revised: 01/14/2022] [Accepted: 01/25/2022] [Indexed: 12/15/2022]
Abstract
This paper presents a novel computational algorithm to estimate blood volume decompensation state based on machine learning (ML) analysis of multi-modal wearable-compatible physiological signals. To the best of our knowledge, our algorithm may be the first of its kind which can not only discriminate normovolemia from hypovolemia but also classify hypovolemia into absolute hypovolemia and relative hypovolemia. We realized our blood volume classification algorithm by (i) extracting a multitude of features from multi-modal physiological signals including the electrocardiogram (ECG), the seismocardiogram (SCG), the ballistocardiogram (BCG), and the photoplethysmogram (PPG), (ii) constructing two ML classifiers using the features, one to classify normovolemia vs. hypovolemia and the other to classify hypovolemia into absolute hypovolemia and relative hypovolemia, and (iii) sequentially integrating the two to enable multi-class classification (normovolemia, absolute hypovolemia, and relative hypovolemia). We developed the blood volume decompensation state classification algorithm using the experimental data collected from six animals undergoing normovolemia, relative hypovolemia, and absolute hypovolemia challenges. Leave-one-subject-out analysis showed that our classification algorithm achieved an F1 score and accuracy of (i) 0.93 and 0.89 in classifying normovolemia vs. hypovolemia, (ii) 0.88 and 0.89 in classifying hypovolemia into absolute hypovolemia and relative hypovolemia, and (iii) 0.77 and 0.81 in classifying the overall blood volume decompensation state. The analysis of the features embedded in the ML classifiers indicated that many features are physiologically plausible, and that multi-modal SCG-BCG fusion may play an important role in achieving good blood volume classification efficacy. Our work may complement existing computational algorithms to estimate blood volume compensatory reserve as a potential decision-support tool to provide guidance on context-sensitive hypovolemia therapeutic strategy.
Collapse
Affiliation(s)
- Yekanth Ram Chalumuri
- Department of Mechanical Engineering, University of Maryland, College Park, MD 20742, USA; (A.M.); (J.D.P.)
| | - Jacob P. Kimball
- Department of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30308, USA; (J.P.K.); (J.S.Z.); (O.T.I.)
| | - Azin Mousavi
- Department of Mechanical Engineering, University of Maryland, College Park, MD 20742, USA; (A.M.); (J.D.P.)
| | - Jonathan S. Zia
- Department of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30308, USA; (J.P.K.); (J.S.Z.); (O.T.I.)
| | - Christopher Rolfes
- Global Center for Medical Innovation, Translational Training and Testing Laboratories, Inc. (T3 Labs), Atlanta, GA 30313, USA;
| | - Jesse D. Parreira
- Department of Mechanical Engineering, University of Maryland, College Park, MD 20742, USA; (A.M.); (J.D.P.)
| | - Omer T. Inan
- Department of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30308, USA; (J.P.K.); (J.S.Z.); (O.T.I.)
| | - Jin-Oh Hahn
- Department of Mechanical Engineering, University of Maryland, College Park, MD 20742, USA; (A.M.); (J.D.P.)
| |
Collapse
|
28
|
Lazzaro A, De Girolamo G, Filippi V, Innocenti GP, Santinelli L, Ceccarelli G, Trecarichi EM, Torti C, Mastroianni CM, d’Ettorre G, Russo A. The Interplay between Host Defense, Infection, and Clinical Status in Septic Patients: A Narrative Review. Int J Mol Sci 2022; 23:ijms23020803. [PMID: 35054993 PMCID: PMC8776148 DOI: 10.3390/ijms23020803] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2021] [Revised: 01/05/2022] [Accepted: 01/09/2022] [Indexed: 01/08/2023] Open
Abstract
Sepsis is a life-threatening condition that arises when the body's response to an infection injures its own tissues and organs. Despite significant morbidity and mortality throughout the world, its pathogenesis and mechanisms are not clearly understood. In this narrative review, we aimed to summarize the recent developments in our understanding of the hallmarks of sepsis pathogenesis (immune and adaptive immune response, the complement system, the endothelial disfunction, and autophagy) and highlight novel laboratory diagnostic approaches. Clinical management is also discussed with pivotal consideration for antimicrobic therapy management in particular settings, such as intensive care unit, altered renal function, obesity, and burn patients.
Collapse
Affiliation(s)
- Alessandro Lazzaro
- Department of Public Health and Infectious Diseases, “Sapienza” University of Rome, 00161 Rome, Italy; (A.L.); (G.D.G.); (V.F.); (G.P.I.); (L.S.); (G.C.); (C.M.M.); (G.d.)
| | - Gabriella De Girolamo
- Department of Public Health and Infectious Diseases, “Sapienza” University of Rome, 00161 Rome, Italy; (A.L.); (G.D.G.); (V.F.); (G.P.I.); (L.S.); (G.C.); (C.M.M.); (G.d.)
| | - Valeria Filippi
- Department of Public Health and Infectious Diseases, “Sapienza” University of Rome, 00161 Rome, Italy; (A.L.); (G.D.G.); (V.F.); (G.P.I.); (L.S.); (G.C.); (C.M.M.); (G.d.)
| | - Giuseppe Pietro Innocenti
- Department of Public Health and Infectious Diseases, “Sapienza” University of Rome, 00161 Rome, Italy; (A.L.); (G.D.G.); (V.F.); (G.P.I.); (L.S.); (G.C.); (C.M.M.); (G.d.)
| | - Letizia Santinelli
- Department of Public Health and Infectious Diseases, “Sapienza” University of Rome, 00161 Rome, Italy; (A.L.); (G.D.G.); (V.F.); (G.P.I.); (L.S.); (G.C.); (C.M.M.); (G.d.)
| | - Giancarlo Ceccarelli
- Department of Public Health and Infectious Diseases, “Sapienza” University of Rome, 00161 Rome, Italy; (A.L.); (G.D.G.); (V.F.); (G.P.I.); (L.S.); (G.C.); (C.M.M.); (G.d.)
| | - Enrico Maria Trecarichi
- Infectious and Tropical Disease Unit, Department of Medical and Surgical Sciences, “Magna Graecia” University of Catanzaro, 88100 Catanzaro, Italy; (E.M.T.); (C.T.)
| | - Carlo Torti
- Infectious and Tropical Disease Unit, Department of Medical and Surgical Sciences, “Magna Graecia” University of Catanzaro, 88100 Catanzaro, Italy; (E.M.T.); (C.T.)
| | - Claudio Maria Mastroianni
- Department of Public Health and Infectious Diseases, “Sapienza” University of Rome, 00161 Rome, Italy; (A.L.); (G.D.G.); (V.F.); (G.P.I.); (L.S.); (G.C.); (C.M.M.); (G.d.)
| | - Gabriella d’Ettorre
- Department of Public Health and Infectious Diseases, “Sapienza” University of Rome, 00161 Rome, Italy; (A.L.); (G.D.G.); (V.F.); (G.P.I.); (L.S.); (G.C.); (C.M.M.); (G.d.)
| | - Alessandro Russo
- Infectious and Tropical Disease Unit, Department of Medical and Surgical Sciences, “Magna Graecia” University of Catanzaro, 88100 Catanzaro, Italy; (E.M.T.); (C.T.)
- Correspondence:
| |
Collapse
|
29
|
Lipatov K, Daniels CE, Park JG, Elmer J, Hanson AC, Madsen BE, Clements CM, Gajic O, Pickering BW, Herasevich V. Implementation and evaluation of sepsis surveillance and decision support in medical ICU and emergency department. Am J Emerg Med 2021; 51:378-383. [PMID: 34823194 DOI: 10.1016/j.ajem.2021.09.086] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Revised: 09/13/2021] [Accepted: 09/17/2021] [Indexed: 02/08/2023] Open
Abstract
OBJECTIVE To improve the timely diagnosis and treatment of sepsis many institutions implemented automated sepsis alerts. Poor specificity, time delays, and a lack of actionable information lead to limited adoption by bedside clinicians and no change in practice or clinical outcomes. We aimed to compare sepsis care compliance before and after a multi-year implementation of a sepsis surveillance coupled with decision support in a tertiary care center. DESIGN Single center before and after study. SETTING Large academic Medical Intensive Care Unit (MICU) and Emergency Department (ED). POPULATION Patients 18 years of age or older admitted to *** Hospital MICU and ED from 09/4/2011 to 05/01/2018 with severe sepsis or septic shock. INTERVENTIONS Electronic medical record-based sepsis surveillance system augmented by clinical decision support and completion feedback. MEASUREMENTS AND MAIN RESULTS There were 1950 patients admitted to the MICU with the diagnosis of severe sepsis or septic shock during the study period. The baseline characteristics were similar before (N = 854) and after (N = 1096) implementation of sepsis surveillance. The performance of the alert was modest with a sensitivity of 79.9%, specificity of 76.9%, positive predictive value (PPV) 27.9%, and negative predictive value (NPV) 97.2%. There were 3424 unique alerts and 1131 confirmed sepsis patients after the sniffer implementation. During the study period average care bundle compliance was higher; however after taking into account improvements in compliance leading up to the intervention, there was no association between intervention and improved care bundle compliance (Odds ratio: 1.16; 95% CI: 0.71 to 1.89; p-value 0.554). Similarly, the intervention was not associated with improvement in hospital mortality (Odds ratio: 1.55; 95% CI: 0.95 to 2.52; p-value: 0.078). CONCLUSIONS A sepsis surveillance system incorporating decision support or completion feedback was not associated with improved sepsis care and patient outcomes.
Collapse
Affiliation(s)
- Kirill Lipatov
- Department of Pulmonary and Critical Care Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Craig E Daniels
- Department of Pulmonary and Critical Care Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - John G Park
- Department of Pulmonary and Critical Care Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Jennifer Elmer
- Department of Nursing, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Andrew C Hanson
- Department of Quantitative Health Sciences, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Bo E Madsen
- Department of Emergency Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Casey M Clements
- Department of Emergency Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Ognjen Gajic
- Department of Pulmonary and Critical Care Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Brian W Pickering
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Vitaly Herasevich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA.
| |
Collapse
|
30
|
Wang D, Li J, Sun Y, Ding X, Zhang X, Liu S, Han B, Wang H, Duan X, Sun T. A Machine Learning Model for Accurate Prediction of Sepsis in ICU Patients. Front Public Health 2021; 9:754348. [PMID: 34722452 PMCID: PMC8553999 DOI: 10.3389/fpubh.2021.754348] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Accepted: 09/20/2021] [Indexed: 12/23/2022] Open
Abstract
Background: Although numerous studies are conducted every year on how to reduce the fatality rate associated with sepsis, it is still a major challenge faced by patients, clinicians, and medical systems worldwide. Early identification and prediction of patients at risk of sepsis and adverse outcomes associated with sepsis are critical. We aimed to develop an artificial intelligence algorithm that can predict sepsis early. Methods: This was a secondary analysis of an observational cohort study from the Intensive Care Unit of the First Affiliated Hospital of Zhengzhou University. A total of 4,449 infected patients were randomly assigned to the development and validation data set at a ratio of 4:1. After extracting electronic medical record data, a set of 55 features (variables) was calculated and passed to the random forest algorithm to predict the onset of sepsis. Results: The pre-procedure clinical variables were used to build a prediction model from the training data set using the random forest machine learning method; a 5-fold cross-validation was used to evaluate the prediction accuracy of the model. Finally, we tested the model using the validation data set. The area obtained by the model under the receiver operating characteristic (ROC) curve (AUC) was 0.91, the sensitivity was 87%, and the specificity was 89%. Conclusions: This newly established machine learning-based model has shown good predictive ability in Chinese sepsis patients. External validation studies are necessary to confirm the universality of our method in the population and treatment practice.
Collapse
Affiliation(s)
- Dong Wang
- General Intensive Care Unit, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.,Key Laboratory for Critical Care Medicine of Henan Province, Zhengzhou, China.,Key Laboratory for Sepsis of Zhengzhou, Zhengzhou, China
| | - Jinbo Li
- General Intensive Care Unit, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.,Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada
| | - Yali Sun
- General Intensive Care Unit, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.,Key Laboratory for Critical Care Medicine of Henan Province, Zhengzhou, China.,Key Laboratory for Sepsis of Zhengzhou, Zhengzhou, China
| | - Xianfei Ding
- General Intensive Care Unit, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.,Key Laboratory for Critical Care Medicine of Henan Province, Zhengzhou, China.,Key Laboratory for Sepsis of Zhengzhou, Zhengzhou, China
| | - Xiaojuan Zhang
- General Intensive Care Unit, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.,Key Laboratory for Critical Care Medicine of Henan Province, Zhengzhou, China.,Key Laboratory for Sepsis of Zhengzhou, Zhengzhou, China
| | - Shaohua Liu
- General Intensive Care Unit, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.,Key Laboratory for Critical Care Medicine of Henan Province, Zhengzhou, China.,Key Laboratory for Sepsis of Zhengzhou, Zhengzhou, China
| | - Bing Han
- General Intensive Care Unit, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.,Key Laboratory for Critical Care Medicine of Henan Province, Zhengzhou, China.,Key Laboratory for Sepsis of Zhengzhou, Zhengzhou, China
| | - Haixu Wang
- General Intensive Care Unit, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.,Key Laboratory for Critical Care Medicine of Henan Province, Zhengzhou, China.,Key Laboratory for Sepsis of Zhengzhou, Zhengzhou, China
| | - Xiaoguang Duan
- General Intensive Care Unit, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.,Key Laboratory for Critical Care Medicine of Henan Province, Zhengzhou, China.,Key Laboratory for Sepsis of Zhengzhou, Zhengzhou, China
| | - Tongwen Sun
- General Intensive Care Unit, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China.,Key Laboratory for Critical Care Medicine of Henan Province, Zhengzhou, China.,Key Laboratory for Sepsis of Zhengzhou, Zhengzhou, China
| |
Collapse
|
31
|
Zargoush M, Sameh A, Javadi M, Shabani S, Ghazalbash S, Perri D. The impact of recency and adequacy of historical information on sepsis predictions using machine learning. Sci Rep 2021; 11:20869. [PMID: 34675275 PMCID: PMC8531301 DOI: 10.1038/s41598-021-00220-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2021] [Accepted: 10/07/2021] [Indexed: 12/11/2022] Open
Abstract
Sepsis is a major public and global health concern. Every hour of delay in detecting sepsis significantly increases the risk of death, highlighting the importance of accurately predicting sepsis in a timely manner. A growing body of literature has examined developing new or improving the existing machine learning (ML) approaches for timely and accurate predictions of sepsis. This study contributes to this literature by providing clear insights regarding the role of the recency and adequacy of historical information in predicting sepsis using ML. To this end, we implemented a deep learning model using a bidirectional long short-term memory (BiLSTM) algorithm and compared it with six other ML algorithms based on numerous combinations of the prediction horizons (to capture information recency) and observation windows (to capture information adequacy) using different measures of predictive performance. Our results indicated that the BiLSTM algorithm outperforms all other ML algorithms and provides a great separability of the predicted risk of sepsis among septic versus non-septic patients. Moreover, decreasing the prediction horizon (in favor of information recency) always boosts the predictive performance; however, the impact of expanding the observation window (in favor of information adequacy) depends on the prediction horizon and the purpose of prediction. More specifically, when the prediction is responsive to the positive label (i.e., Sepsis), increasing historical data improves the predictive performance when the prediction horizon is short-moderate.
Collapse
Affiliation(s)
- Manaf Zargoush
- Health Policy and Management Area, DeGroote School of Business, McMaster University, Hamilton, ON, Canada.
| | - Alireza Sameh
- Department of Industrial Engineering, Amirkabir University of Technology, Tehran, Iran
| | - Mahdi Javadi
- Department of Decision Sciences, HEC Montréal, Montréal, QC, Canada
| | - Siyavash Shabani
- Department of Biomedical Engineering, Amirkabir University of Technology, Tehran, Iran
| | - Somayeh Ghazalbash
- Health Policy and Management Area, DeGroote School of Business, McMaster University, Hamilton, ON, Canada
| | - Dan Perri
- Department of Medicine, Faculty of Health Sciences, Department of Critical Care, and Chief Medical Information Officer, McMaster University and Staff Intensivist, St. Joseph's Healthcare Hamilton, Hamilton, ON, Canada
| |
Collapse
|
32
|
Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation. J Pers Med 2021; 11:jpm11111055. [PMID: 34834406 PMCID: PMC8623760 DOI: 10.3390/jpm11111055] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Revised: 10/11/2021] [Accepted: 10/18/2021] [Indexed: 12/23/2022] Open
Abstract
Accurate stratification of sepsis can effectively guide the triage of patient care and shared decision making in the emergency department (ED). However, previous research on sepsis identification models focused mainly on ICU patients, and discrepancies in model performance between the development and external validation datasets are rarely evaluated. The aim of our study was to develop and externally validate a machine learning model to stratify sepsis patients in the ED. We retrospectively collected clinical data from two geographically separate institutes that provided a different level of care at different time periods. The Sepsis-3 criteria were used as the reference standard in both datasets for identifying true sepsis cases. An eXtreme Gradient Boosting (XGBoost) algorithm was developed to stratify sepsis patients and the performance of the model was compared with traditional clinical sepsis tools; quick Sequential Organ Failure Assessment (qSOFA) and Systemic Inflammatory Response Syndrome (SIRS). There were 8296 patients (1752 (21%) being septic) in the development and 1744 patients (506 (29%) being septic) in the external validation datasets. The mortality of septic patients in the development and validation datasets was 13.5% and 17%, respectively. In the internal validation, XGBoost achieved an area under the receiver operating characteristic curve (AUROC) of 0.86, exceeding SIRS (0.68) and qSOFA (0.56). The performance of XGBoost deteriorated in the external validation (the AUROC of XGBoost, SIRS and qSOFA was 0.75, 0.57 and 0.66, respectively). Heterogeneity in patient characteristics, such as sepsis prevalence, severity, age, comorbidity and infection focus, could reduce model performance. Our model showed good discriminative capabilities for the identification of sepsis patients and outperformed the existing sepsis identification tools. Implementation of the ML model in the ED can facilitate timely sepsis identification and treatment. However, dataset discrepancies should be carefully evaluated before implementing the ML approach in clinical practice. This finding reinforces the necessity for future studies to perform external validation to ensure the generalisability of any developed ML approaches.
Collapse
|
33
|
Kollef MH, Shorr AF, Bassetti M, Timsit JF, Micek ST, Michelson AP, Garnacho-Montero J. Timing of antibiotic therapy in the ICU. Crit Care 2021; 25:360. [PMID: 34654462 PMCID: PMC8518273 DOI: 10.1186/s13054-021-03787-z] [Citation(s) in RCA: 57] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2021] [Accepted: 10/06/2021] [Indexed: 12/15/2022] Open
Abstract
Severe or life threatening infections are common among patients in the intensive care unit (ICU). Most infections in the ICU are bacterial or fungal in origin and require antimicrobial therapy for clinical resolution. Antibiotics are the cornerstone of therapy for infected critically ill patients. However, antibiotics are often not optimally administered resulting in less favorable patient outcomes including greater mortality. The timing of antibiotics in patients with life threatening infections including sepsis and septic shock is now recognized as one of the most important determinants of survival for this population. Individuals who have a delay in the administration of antibiotic therapy for serious infections can have a doubling or more in their mortality. Additionally, the timing of an appropriate antibiotic regimen, one that is active against the offending pathogens based on in vitro susceptibility, also influences survival. Thus not only is early empiric antibiotic administration important but the selection of those agents is crucial as well. The duration of antibiotic infusions, especially for β-lactams, can also influence antibiotic efficacy by increasing antimicrobial drug exposure for the offending pathogen. However, due to mounting antibiotic resistance, aggressive antimicrobial de-escalation based on microbiology results is necessary to counterbalance the pressures of early broad-spectrum antibiotic therapy. In this review, we examine time related variables impacting antibiotic optimization as it relates to the treatment of life threatening infections in the ICU. In addition to highlighting the importance of antibiotic timing in the ICU we hope to provide an approach to antimicrobials that also minimizes the unnecessary use of these agents. Such approaches will increasingly be linked to advances in molecular microbiology testing and artificial intelligence/machine learning. Such advances should help identify patients needing empiric antibiotic therapy at an earlier time point as well as the specific antibiotics required in order to avoid unnecessary administration of broad-spectrum antibiotics.
Collapse
Affiliation(s)
- Marin H Kollef
- Division of Pulmonary and Critical Care Medicine, Washington University School of Medicine, 660 South Euclid Avenue, MSC 8052-43-14, St. Louis, MO, 63110, USA.
| | - Andrew F Shorr
- Pulmonary and Critical Care Medicine, Medstar Washington Hospital, Washington, DC, USA
| | - Matteo Bassetti
- Infectious Diseases Unit, Department of Health Sciences, San Martino Policlinico Hospital - IRCCS, University of Genoa, Genoa, Italy
| | - Jean-Francois Timsit
- AP-HP, Bichat Claude Bernard Hospital, Medical and Infectious Diseases ICU (MI2), IAME, INSERM, Université de Paris, Paris, France
| | - Scott T Micek
- Department of Pharmacy Practice, University of Health Sciences and Pharmacy, St. Louis, MO, USA
| | - Andrew P Michelson
- Division of Pulmonary and Critical Care Medicine, Washington University School of Medicine, 660 South Euclid Avenue, MSC 8052-43-14, St. Louis, MO, 63110, USA
| | | |
Collapse
|
34
|
Machine learning identification of specific changes in myeloid cell phenotype during bloodstream infections. Sci Rep 2021; 11:20288. [PMID: 34645893 PMCID: PMC8514545 DOI: 10.1038/s41598-021-99628-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2021] [Accepted: 09/29/2021] [Indexed: 11/18/2022] Open
Abstract
The early identification of bacteremia is critical for ensuring appropriate treatment of nosocomial infections in intensive care unit (ICU) patients. The aim of this study was to use flow cytometric data of myeloid cells as a biomarker of bloodstream infection (BSI). An eight-color antibody panel was used to identify seven monocyte and two dendritic cell subsets. In the learning cohort, immunophenotyping was applied to (1) control subjects, (2) postoperative heart surgery patients, as a model of noninfectious inflammatory responses, and (3) blood culture-positive patients. Of the complex changes in the myeloid cell phenotype, a decrease in myeloid and plasmacytoid dendritic cell numbers, increase in CD14+CD16+ inflammatory monocyte numbers, and upregulation of neutrophils CD64 and CD123 expression were prominent in BSI patients. An extreme gradient boosting (XGBoost) algorithm called the “infection detection and ranging score” (iDAR), ranging from 0 to 100, was developed to identify infection-specific changes in 101 phenotypic variables related to neutrophils, monocytes and dendritic cells. The tenfold cross-validation achieved an area under the receiver operating characteristic (AUROC) of 0.988 (95% CI 0.985–1) for the detection of bacteremic patients. In an out-of-sample, in-house validation, iDAR achieved an AUROC of 0.85 (95% CI 0.71–0.98) in differentiating localized from bloodstream infection and 0.95 (95% CI 0.89–1) in discriminating infected from noninfected ICU patients. In conclusion, a machine learning approach was used to translate the changes in myeloid cell phenotype in response to infection into a score that could identify bacteremia with high specificity in ICU patients.
Collapse
|
35
|
Tsopra R, Fernandez X, Luchinat C, Alberghina L, Lehrach H, Vanoni M, Dreher F, Sezerman OU, Cuggia M, de Tayrac M, Miklasevics E, Itu LM, Geanta M, Ogilvie L, Godey F, Boldisor CN, Campillo-Gimenez B, Cioroboiu C, Ciusdel CF, Coman S, Hijano Cubelos O, Itu A, Lange B, Le Gallo M, Lespagnol A, Mauri G, Soykam HO, Rance B, Turano P, Tenori L, Vignoli A, Wierling C, Benhabiles N, Burgun A. A framework for validating AI in precision medicine: considerations from the European ITFoC consortium. BMC Med Inform Decis Mak 2021; 21:274. [PMID: 34600518 PMCID: PMC8487519 DOI: 10.1186/s12911-021-01634-3] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2020] [Accepted: 09/22/2021] [Indexed: 12/22/2022] Open
Abstract
BACKGROUND Artificial intelligence (AI) has the potential to transform our healthcare systems significantly. New AI technologies based on machine learning approaches should play a key role in clinical decision-making in the future. However, their implementation in health care settings remains limited, mostly due to a lack of robust validation procedures. There is a need to develop reliable assessment frameworks for the clinical validation of AI. We present here an approach for assessing AI for predicting treatment response in triple-negative breast cancer (TNBC), using real-world data and molecular -omics data from clinical data warehouses and biobanks. METHODS The European "ITFoC (Information Technology for the Future Of Cancer)" consortium designed a framework for the clinical validation of AI technologies for predicting treatment response in oncology. RESULTS This framework is based on seven key steps specifying: (1) the intended use of AI, (2) the target population, (3) the timing of AI evaluation, (4) the datasets used for evaluation, (5) the procedures used for ensuring data safety (including data quality, privacy and security), (6) the metrics used for measuring performance, and (7) the procedures used to ensure that the AI is explainable. This framework forms the basis of a validation platform that we are building for the "ITFoC Challenge". This community-wide competition will make it possible to assess and compare AI algorithms for predicting the response to TNBC treatments with external real-world datasets. CONCLUSIONS The predictive performance and safety of AI technologies must be assessed in a robust, unbiased and transparent manner before their implementation in healthcare settings. We believe that the consideration of the ITFoC consortium will contribute to the safe transfer and implementation of AI in clinical settings, in the context of precision oncology and personalized care.
Collapse
Affiliation(s)
- Rosy Tsopra
- Centre de Recherche Des Cordeliers, Inserm, Université de Paris, Sorbonne Université, 75006, Paris, France. .,Inria, HeKA, Inria Paris, France. .,Department of Medical Informatics, Hôpital Européen Georges-Pompidou, AP-HP, Paris, France. .,Univ Rennes, CHU Rennes, Inserm, LTSI - UMR 1099, 35000, Rennes, France.
| | | | - Claudio Luchinat
- Centro Risonanze Magnetiche - CERM/CIRMMP and Department of Chemistry, University of Florence, 50019, Sesto Fiorentino (Florence), Italy
| | - Lilia Alberghina
- Department of Biotechnology and Biosciences, University of Milano Bicocca and ISBE-Italy/SYSBIO - Candidate National Node of Italy for ISBE, Research Infrastructure for Systems Biology Europe, Milan, Italy
| | - Hans Lehrach
- Max Planck Institute for Molecular Genetics, Berlin, Germany.,Alacris Theranostics GmbH, Berlin, Germany
| | - Marco Vanoni
- Department of Biotechnology and Biosciences, University of Milano Bicocca and ISBE-Italy/SYSBIO - Candidate National Node of Italy for ISBE, Research Infrastructure for Systems Biology Europe, Milan, Italy
| | | | - O Ugur Sezerman
- School of Medicine Biostatistics and Medical Informatics Dept., Acibadem University, Istanbul, Turkey
| | - Marc Cuggia
- Univ Rennes, CHU Rennes, Inserm, LTSI - UMR 1099, 35000, Rennes, France
| | - Marie de Tayrac
- Univ Rennes, Department of Molecular Genetics and Genomics, CHU Rennes, IGDR-UMR6290, CNRS, 35000, Rennes, France
| | | | | | - Marius Geanta
- Centre for Innovation in Medicine, Bucharest, Romania
| | - Lesley Ogilvie
- Max Planck Institute for Molecular Genetics, Berlin, Germany.,Alacris Theranostics GmbH, Berlin, Germany
| | - Florence Godey
- INSERM U1242 « Chemistry, Oncogenesis Stress Signaling », Université de Rennes, 35042, CEDEX, Rennes, France.,Centre de Lutte Contre Le Cancer Eugène Marquis, CRB Santé (BRIF Number: BB-0033-00056), 35042, CEDEX, Rennes, France
| | | | | | | | | | - Simona Coman
- Transilvania University of Brasov, Brasov, Romania
| | | | - Alina Itu
- Transilvania University of Brasov, Brasov, Romania
| | - Bodo Lange
- Alacris Theranostics GmbH, Berlin, Germany
| | - Matthieu Le Gallo
- INSERM U1242 « Chemistry, Oncogenesis Stress Signaling », Université de Rennes, 35042, CEDEX, Rennes, France.,Centre de Lutte Contre Le Cancer Eugène Marquis, CRB Santé (BRIF Number: BB-0033-00056), 35042, CEDEX, Rennes, France
| | - Alexandra Lespagnol
- Department of Molecular Genetics and Genomics, CHU Rennes, 35000, Rennes, France
| | - Giancarlo Mauri
- Department of Informatics, Systems and Communication, University of Milano Bicocca and ISBE-Italy/SYSBIO - Candidate National Node of Italy for ISBE, Research Infrastructure for Systems Biology Europe, Milan, Italy
| | | | - Bastien Rance
- Centre de Recherche Des Cordeliers, Inserm, Université de Paris, Sorbonne Université, 75006, Paris, France.,Inria, HeKA, Inria Paris, France.,Department of Medical Informatics, Hôpital Européen Georges-Pompidou, AP-HP, Paris, France
| | - Paola Turano
- Centro Risonanze Magnetiche - CERM/CIRMMP and Department of Chemistry, University of Florence, 50019, Sesto Fiorentino (Florence), Italy
| | - Leonardo Tenori
- Centro Risonanze Magnetiche - CERM/CIRMMP and Department of Chemistry, University of Florence, 50019, Sesto Fiorentino (Florence), Italy
| | - Alessia Vignoli
- Centro Risonanze Magnetiche - CERM/CIRMMP and Department of Chemistry, University of Florence, 50019, Sesto Fiorentino (Florence), Italy
| | | | - Nora Benhabiles
- Direction de La Recherche Fondamentale (DRF), CEA, Université Paris-Saclay, 91191, Gif-sur-Yvette, France
| | - Anita Burgun
- Centre de Recherche Des Cordeliers, Inserm, Université de Paris, Sorbonne Université, 75006, Paris, France.,Inria, HeKA, Inria Paris, France.,Department of Medical Informatics, Hôpital Européen Georges-Pompidou, AP-HP, Paris, France.,PaRis Artificial Intelligence Research InstitutE (Prairie), Paris, France
| |
Collapse
|
36
|
Jazayeri A, Capan M, Ivy J, Arnold R, Yang CC. Proximity of Cellular and Physiological Response Failures in Sepsis. IEEE J Biomed Health Inform 2021; 25:4089-4097. [PMID: 34288881 DOI: 10.1109/jbhi.2021.3098428] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Sepsis is a devastating multi-stage health condition with a high mortality rate. Its complexity, prevalence, and dependency of its outcomes on early detection have attracted substantial attention from data science and machine learning communities. Previous studies rely on individual cellular and physiological responses representing organ system failures to predict health outcomes or the onset of different sepsis stages. However, it is known that organ systems' failures and dynamics are not independent events. In this study, we identify the dependency patterns of significant proximate sepsis-related failures of cellular and physiological responses using data from 12,223 adult patients hospitalized between July 2013 and December 2015. The results show that proximate failures of cellular and physiological responses create better feature sets for outcome prediction than individual responses. Our findings reveal the few significant proximate failures that play the major roles in predicting patients' outcomes. This study's results can be simply translated into clinical practices and inform the prediction and improvement of patients' conditions and outcomes.
Collapse
|
37
|
Lee JY, Molani S, Fang C, Jade K, O'Mahony DS, Kornilov SA, Mico LT, Hadlock JJ. Ambulatory Risk Models for the Long-Term Prevention of Sepsis: Retrospective Study. JMIR Med Inform 2021; 9:e29986. [PMID: 34086596 PMCID: PMC8299345 DOI: 10.2196/29986] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Accepted: 06/02/2021] [Indexed: 12/29/2022] Open
Abstract
Background Sepsis is a life-threatening condition that can rapidly lead to organ damage and death. Existing risk scores predict outcomes for patients who have already become acutely ill. Objective We aimed to develop a model for identifying patients at risk of getting sepsis within 2 years in order to support the reduction of sepsis morbidity and mortality. Methods Machine learning was applied to 2,683,049 electronic health records (EHRs) with over 64 million encounters across five states to develop models for predicting a patient’s risk of getting sepsis within 2 years. Features were selected to be easily obtainable from a patient’s chart in real time during ambulatory encounters. Results The models showed consistent prediction scores, with the highest area under the receiver operating characteristic curve of 0.82 and a positive likelihood ratio of 2.9 achieved with gradient boosting on all features combined. Predictive features included age, sex, ethnicity, average ambulatory heart rate, standard deviation of BMI, and the number of prior medical conditions and procedures. The findings identified both known and potential new risk factors for long-term sepsis. Model variations also illustrated trade-offs between incrementally higher accuracy, implementability, and interpretability. Conclusions Accurate implementable models were developed to predict the 2-year risk of sepsis, using EHR data that is easy to obtain from ambulatory encounters. These results help advance the understanding of sepsis and provide a foundation for future trials of risk-informed preventive care.
Collapse
Affiliation(s)
- Jewel Y Lee
- Institute for Systems Biology, Seattle, WA, United States
| | - Sevda Molani
- Institute for Systems Biology, Seattle, WA, United States
| | - Chen Fang
- Institute for Systems Biology, Seattle, WA, United States
| | - Kathleen Jade
- Institute for Systems Biology, Seattle, WA, United States
| | - D Shane O'Mahony
- Swedish Center for Research and Innovation, Swedish Medical Center, Seattle, WA, United States
| | | | | | | |
Collapse
|
38
|
Kazzazi F. The automation of doctors and machines: A classification for AI in medicine (ADAM framework). Future Healthc J 2021; 8:e257-e262. [PMID: 34286194 PMCID: PMC8285145 DOI: 10.7861/fhj.2020-0189] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
The advances in artificial intelligence (AI) provide an opportunity to expand the frontier of medicine to improve diagnosis, efficiency and management. By extension of being able to perform any task that a human could, a machine that meets the requirements of artificial general intelligence ('strong' AI; AGI) possesses the basic necessities to perform as, or at least qualify to become, a doctor. In this emerging field, this article explores the distinctions between doctors and AGI, and the prerequisites for AGI performing as clinicians. In doing so, it necessitates the requirement for a classification of medical AI and prepares for the development of AGI. With its imminent arrival, it is beneficial to create a framework from which leading institutions can define specific criteria for AGI.
Collapse
Affiliation(s)
- Fawz Kazzazi
- Mason Institute for Medicine, Life Sciences and Law, Edinburgh, UK
| |
Collapse
|
39
|
Mohammed A, Van Wyk F, Chinthala LK, Khojandi A, Davis RL, Coopersmith CM, Kamaleswaran R. Temporal Differential Expression of Physiomarkers Predicts Sepsis in Critically Ill Adults. Shock 2021; 56:58-64. [PMID: 32991797 PMCID: PMC8352046 DOI: 10.1097/shk.0000000000001670] [Citation(s) in RCA: 27] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
BACKGROUND Sepsis is a life-threatening condition with high mortality rates. Early detection and treatment are critical to improving outcomes. Our primary objective was to develop artificial intelligence capable of predicting sepsis earlier using a minimal set of streaming physiological data in real time. METHODS AND FINDINGS A total of 29,552 adult patients were admitted to the intensive care unit across five regional hospitals in Memphis, Tenn, over 18 months from January 2017 to July 2018. From these, 5,958 patients were selected after filtering for continuous (minute-by-minute) physiological data availability. A total of 617 (10.4%) patients were identified as sepsis cases, using the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) criteria. Physiomarkers, a set of signal processing features, were derived from five physiological data streams including heart rate, respiratory rate, and blood pressure (systolic, diastolic, and mean), captured every minute from the bedside monitors. A support vector machine classifier was used for classification. The model accurately predicted sepsis up to a mean and 95% confidence interval of 17.4 ± 0.22 h before sepsis onset, with an average test accuracy of 83.0% (average sensitivity, specificity, and area under the receiver operating characteristics curve of 0.757, 0.902, and 0.781, respectively). CONCLUSIONS This study demonstrates that salient physiomarkers derived from continuous bedside monitoring are temporally and differentially expressed in septic patients. Using this information, minimalistic artificial intelligence models can be developed to predict sepsis earlier in critically ill patients.
Collapse
Affiliation(s)
- Akram Mohammed
- University of Tennessee Health Science Center, Memphis TN
| | | | | | | | | | | | | |
Collapse
|
40
|
Wang Z, Yao B. Multi-Branching Temporal Convolutional Network for Sepsis Prediction. IEEE J Biomed Health Inform 2021; 26:876-887. [PMID: 34181558 DOI: 10.1109/jbhi.2021.3092835] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Sepsis is among the leading causes of morbidity and mortality in modern intensive care units. Accurate sepsis prediction is of critical importance to save lives and reduce medical costs. The rapid advancements in sensing and information technology facilitate the effective monitoring of patients health conditions, generating a wealth of medical data, and provide an unprecedented opportunity for data-driven diagnosis of sepsis. However, real-world medical data are often complexly structured with a high level of uncertainty (e.g., missing values, imbalanced data). Realizing the full data potential depends on developing effective analytical models. In this paper, we propose a novel predictive framework with Multi-Branching Temporal Convolutional Network (MB-TCN) to model the complexly structured medical data for robust prediction of sepsis. The MB-TCN framework not only efficiently handles the missing value and imbalanced data issues but also effectively captures the temporal pattern and heterogeneous variable interactions. We evaluate the performance of the proposed MB-TCN in predicting sepsis using real-world medical data from PhysioNet/Computing in Cardiology Challenge 2019. Experimental results show that MB-TCN outperforms existing methods that are commonly used in current practice.
Collapse
|
41
|
Abstract
Objectives Sepsis and septic shock are leading causes of in-hospital mortality. Timely treatment is crucial in improving patient outcome, yet treatment delays remain common. Early prediction of those patients with sepsis who will progress to its most severe form, septic shock, can increase the actionable window for interventions. We aim to extend a time-evolving risk score, previously developed in adult patients, to predict pediatric sepsis patients who are likely to develop septic shock before its onset, and to determine whether or not these risk scores stratify into groups with distinct temporal evolution once this prediction is made. Design Retrospective cohort study. Setting Academic medical center from July 1, 2016, to December 11, 2020. Patients Six-thousand one-hundred sixty-one patients under 18 admitted to the Johns Hopkins Hospital PICU. Interventions None. Measurements and Main Results We trained risk models to predict impending transition into septic shock and compute time-evolving risk scores representative of a patient's probability of developing septic shock. We obtain early prediction performance of 0.90 area under the receiver operating curve, 43% overall positive predictive value, patient-specific positive predictive value as high as 62%, and an 8.9-hour median early warning time using Sepsis-3 labels based on age-adjusted Sequential Organ Failure Assessment score. Using spectral clustering, we stratified pediatric sepsis patients into two clusters differing in septic shock prevalence, mortality, and proportion of patients adequately fluid resuscitated. CONCLUSIONS We demonstrate the applicability of our methodology for early prediction and stratification for risk of septic shock in pediatric sepsis patients. Through analyses of risk score evolution over time, we corroborate our past finding of an abrupt transition preceding onset of septic shock in children and are able to stratify pediatric sepsis patients using their risk score trajectories into low and high-risk categories.
Collapse
|
42
|
Chen M, Hernández A. Towards an explainable model for Sepsis detection based on sensitivity analysis. Ing Rech Biomed 2021. [DOI: 10.1016/j.irbm.2021.05.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
43
|
Moor M, Rieck B, Horn M, Jutzeler CR, Borgwardt K. Early Prediction of Sepsis in the ICU Using Machine Learning: A Systematic Review. Front Med (Lausanne) 2021; 8:607952. [PMID: 34124082 PMCID: PMC8193357 DOI: 10.3389/fmed.2021.607952] [Citation(s) in RCA: 39] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2020] [Accepted: 03/04/2021] [Indexed: 12/12/2022] Open
Abstract
Background: Sepsis is among the leading causes of death in intensive care units (ICUs) worldwide and its recognition, particularly in the early stages of the disease, remains a medical challenge. The advent of an affluence of available digital health data has created a setting in which machine learning can be used for digital biomarker discovery, with the ultimate goal to advance the early recognition of sepsis. Objective: To systematically review and evaluate studies employing machine learning for the prediction of sepsis in the ICU. Data Sources: Using Embase, Google Scholar, PubMed/Medline, Scopus, and Web of Science, we systematically searched the existing literature for machine learning-driven sepsis onset prediction for patients in the ICU. Study Eligibility Criteria: All peer-reviewed articles using machine learning for the prediction of sepsis onset in adult ICU patients were included. Studies focusing on patient populations outside the ICU were excluded. Study Appraisal and Synthesis Methods: A systematic review was performed according to the PRISMA guidelines. Moreover, a quality assessment of all eligible studies was performed. Results: Out of 974 identified articles, 22 and 21 met the criteria to be included in the systematic review and quality assessment, respectively. A multitude of machine learning algorithms were applied to refine the early prediction of sepsis. The quality of the studies ranged from "poor" (satisfying ≤ 40% of the quality criteria) to "very good" (satisfying ≥ 90% of the quality criteria). The majority of the studies (n = 19, 86.4%) employed an offline training scenario combined with a horizon evaluation, while two studies implemented an online scenario (n = 2, 9.1%). The massive inter-study heterogeneity in terms of model development, sepsis definition, prediction time windows, and outcomes precluded a meta-analysis. Last, only two studies provided publicly accessible source code and data sources fostering reproducibility. Limitations: Articles were only eligible for inclusion when employing machine learning algorithms for the prediction of sepsis onset in the ICU. This restriction led to the exclusion of studies focusing on the prediction of septic shock, sepsis-related mortality, and patient populations outside the ICU. Conclusions and Key Findings: A growing number of studies employs machine learning to optimize the early prediction of sepsis through digital biomarker discovery. This review, however, highlights several shortcomings of the current approaches, including low comparability and reproducibility. Finally, we gather recommendations how these challenges can be addressed before deploying these models in prospective analyses. Systematic Review Registration Number: CRD42020200133.
Collapse
Affiliation(s)
- Michael Moor
- Machine Learning and Computational Biology Lab, Department of Biosystems Science and Engineering, Eidgenössische Technische Hochschule Zürich (ETH Zurich), Basel, Switzerland
- SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland
| | - Bastian Rieck
- Machine Learning and Computational Biology Lab, Department of Biosystems Science and Engineering, Eidgenössische Technische Hochschule Zürich (ETH Zurich), Basel, Switzerland
- SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland
| | - Max Horn
- Machine Learning and Computational Biology Lab, Department of Biosystems Science and Engineering, Eidgenössische Technische Hochschule Zürich (ETH Zurich), Basel, Switzerland
- SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland
| | - Catherine R. Jutzeler
- Machine Learning and Computational Biology Lab, Department of Biosystems Science and Engineering, Eidgenössische Technische Hochschule Zürich (ETH Zurich), Basel, Switzerland
- SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland
| | - Karsten Borgwardt
- Machine Learning and Computational Biology Lab, Department of Biosystems Science and Engineering, Eidgenössische Technische Hochschule Zürich (ETH Zurich), Basel, Switzerland
- SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland
| |
Collapse
|
44
|
Utilization of the Signature Method to Identify the Early Onset of Sepsis From Multivariate Physiological Time Series in Critical Care Monitoring. Crit Care Med 2021; 48:e976-e981. [PMID: 32897664 DOI: 10.1097/ccm.0000000000004510] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
OBJECTIVES Patients in an ICU are particularly vulnerable to sepsis. It is therefore important to detect its onset as early as possible. This study focuses on the development and validation of a new signature-based regression model, augmented with a particular choice of the handcrafted features, to identify a patient's risk of sepsis based on physiologic data streams. The model makes a positive or negative prediction of sepsis for every time interval since admission to the ICU. DESIGN The data were sourced from the PhysioNet/Computing in Cardiology Challenge 2019 on the "Early Prediction of Sepsis from Clinical Data." It consisted of ICU patient data from three separate hospital systems. Algorithms were scored against a specially designed utility function that rewards early predictions in the most clinically relevant region around sepsis onset and penalizes late predictions and false positives. SETTING The work was completed as part of the PhysioNet 2019 Challenge alongside 104 other teams. PATIENTS PhysioNet sourced over 60,000 ICU patients with up to 40 clinical variables for each hour of a patient's ICU stay. The Sepsis-3 criteria was used to define the onset of sepsis. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS The algorithm yielded a utility function score which was the first placed entry in the official phase of the challenge.
Collapse
|
45
|
Abstract
OBJECTIVES Early detection of sepsis is critical in clinical practice since each hour of delayed treatment has been associated with an increase in mortality due to irreversible organ damage. This study aimed to develop an explainable artificial intelligence model for early predicting sepsis by analyzing the electronic health record data from ICU provided by the PhysioNet/Computing in Cardiology Challenge 2019. DESIGN Retrospective observational study. SETTING We developed our model on the shared ICUs publicly data and verified on the full hidden populations for challenge scoring. PATIENTS Public database included 40,336 patients' electronic health records sourced from Beth Israel Deaconess Medical Center (hospital system A) and Emory University Hospital (hospital system B). A total of 24,819 patients from hospital systems A, B, and C (an unidentified hospital system) were sequestered as full hidden test sets. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS A total of 168 features were extracted on hourly basis. Explainable artificial intelligence sepsis predictor model was trained to predict sepsis in real time. Impact of each feature on hourly sepsis prediction was explored in-depth to show the interpretability. The algorithm demonstrated the final clinical utility score of 0.364 in this challenge when tested on the full hidden test sets, and the scores on three separate test sets were 0.430, 0.422, and -0.048, respectively. CONCLUSIONS Explainable artificial intelligence sepsis predictor model achieves superior performance for predicting sepsis risk in a real-time way and provides interpretable information for understanding sepsis risk in ICU.
Collapse
|
46
|
Liu X, Anstey J, Li R, Sarabu C, Sono R, Butte AJ. Rethinking PICO in the Machine Learning Era: ML-PICO. Appl Clin Inform 2021; 12:407-416. [PMID: 34010977 DOI: 10.1055/s-0041-1729752] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Machine learning (ML) has captured the attention of many clinicians who may not have formal training in this area but are otherwise increasingly exposed to ML literature that may be relevant to their clinical specialties. ML papers that follow an outcomes-based research format can be assessed using clinical research appraisal frameworks such as PICO (Population, Intervention, Comparison, Outcome). However, the PICO frameworks strain when applied to ML papers that create new ML models, which are akin to diagnostic tests. There is a need for a new framework to help assess such papers. OBJECTIVE We propose a new framework to help clinicians systematically read and evaluate medical ML papers whose aim is to create a new ML model: ML-PICO (Machine Learning, Population, Identification, Crosscheck, Outcomes). We describe how the ML-PICO framework can be applied toward appraising literature describing ML models for health care. CONCLUSION The relevance of ML to practitioners of clinical medicine is steadily increasing with a growing body of literature. Therefore, it is increasingly important for clinicians to be familiar with how to assess and best utilize these tools. In this paper we have described a practical framework on how to read ML papers that create a new ML model (or diagnostic test): ML-PICO. We hope that this can be used by clinicians to better evaluate the quality and utility of ML papers.
Collapse
Affiliation(s)
- Xinran Liu
- Division of Hospital Medicine, University of California, San Francisco, San Francisco, California, United States.,University of California, San Francisco, San Francisco, California, United States
| | - James Anstey
- Division of Hospital Medicine, University of California, San Francisco, San Francisco, California, United States
| | - Ron Li
- Division of Hospital Medicine, Stanford University, Stanford, California, United States
| | - Chethan Sarabu
- doc.ai, Palo Alto, California, United States.,Department of Pediatrics, Stanford University, Stanford, California, United States
| | - Reiri Sono
- University of California, San Francisco, San Francisco, California, United States
| | - Atul J Butte
- Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, California, United States
| |
Collapse
|
47
|
Li K, Shi Q, Liu S, Xie Y, Liu J. Predicting in-hospital mortality in ICU patients with sepsis using gradient boosting decision tree. Medicine (Baltimore) 2021; 100:e25813. [PMID: 34106618 PMCID: PMC8133100 DOI: 10.1097/md.0000000000025813] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/05/2020] [Accepted: 04/14/2021] [Indexed: 02/05/2023] Open
Abstract
Sepsis is a leading cause of mortality in the intensive care unit. Early prediction of sepsis can reduce the overall mortality rate and cost of sepsis treatment. Some studies have predicted mortality and development of sepsis using machine learning models. However, there is a gap between the creation of different machine learning algorithms and their implementation in clinical practice.This study utilized data from the Medical Information Mart for Intensive Care III. We established and compared the gradient boosting decision tree (GBDT), logistic regression (LR), k-nearest neighbor (KNN), random forest (RF), and support vector machine (SVM).A total of 3937 sepsis patients were included, with 34.3% mortality in the Medical Information Mart for Intensive Care III group. In our comparison of 5 machine learning models (GBDT, LR, KNN, RF, and SVM), the GBDT model showed the best performance with the highest area under the receiver operating characteristic curve (0.992), recall (94.8%), accuracy (95.4%), and F1 score (0.933). The RF, SVM, and KNN models showed better performance (area under the receiver operating characteristic curve: 0.980, 0.898, and 0.877, respectively) than the LR (0.876).The GBDT model showed better performance than other machine learning models (LR, KNN, RF, and SVM) in predicting the mortality of patients with sepsis in the intensive care unit. This could be used to develop a clinical decision support system in the future.
Collapse
Affiliation(s)
- Ke Li
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Qinwen Shi
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Siru Liu
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT
| | - Yilin Xie
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Jialin Liu
- Information Center, West China Hospital, Sichuan University, Chengdu
- Department of Medical Informatics, West China Medical School, Sichuan, China
| |
Collapse
|
48
|
Rosnati M, Fortuin V. MGP-AttTCN: An interpretable machine learning model for the prediction of sepsis. PLoS One 2021; 16:e0251248. [PMID: 33961681 PMCID: PMC8104377 DOI: 10.1371/journal.pone.0251248] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2020] [Accepted: 04/22/2021] [Indexed: 12/29/2022] Open
Abstract
With a mortality rate of 5.4 million lives worldwide every year and a healthcare cost of more than 16 billion dollars in the USA alone, sepsis is one of the leading causes of hospital mortality and an increasing concern in the ageing western world. Recently, medical and technological advances have helped re-define the illness criteria of this disease, which is otherwise poorly understood by the medical society. Together with the rise of widely accessible Electronic Health Records, the advances in data mining and complex nonlinear algorithms are a promising avenue for the early detection of sepsis. This work contributes to the research effort in the field of automated sepsis detection with an open-access labelling of the medical MIMIC-III data set. Moreover, we propose MGP-AttTCN: a joint multitask Gaussian Process and attention-based deep learning model to early predict the occurrence of sepsis in an interpretable manner. We show that our model outperforms the current state-of-the-art and present evidence that different labelling heuristics lead to discrepancies in task difficulty. For instance, when predicting sepsis five hours prior to onset on our new realistic labels, our proposed model achieves an area under the ROC curve of 0.660 and an area under the PR curve of 0.483, whereas the (less interpretable) previous state-of-the-art model (MGP-TCN) achieves 0.635 AUROC and 0.460 AUPR and the popular commercial InSight model achieves 0.490 AUROC and 0.359 AUPR.
Collapse
Affiliation(s)
- Margherita Rosnati
- Department of Computing, Imperial College London, London, United Kingdom
| | - Vincent Fortuin
- Department of Computer Science, ETH Zürich, Zürich, Switzerland
- * E-mail:
| |
Collapse
|
49
|
Tensor learning of pointwise mutual information from EHR data for early prediction of sepsis. Comput Biol Med 2021; 134:104430. [PMID: 33991856 DOI: 10.1016/j.compbiomed.2021.104430] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2021] [Revised: 04/14/2021] [Accepted: 04/19/2021] [Indexed: 11/21/2022]
Abstract
Early detection of sepsis can facilitate early clinical intervention with effective treatment and may reduce sepsis mortality rates. In view of this, machine learning-based automated diagnosis of sepsis using easily recordable physiological data can be more promising as compared to the gold standard rule-based clinical criteria in current practice. This study aims to develop such a machine learning framework that demonstrates the quantification of heterogeneity within the tabular electronic health records (EHR) data of clinical covariates to capture both linear relationships and nonlinear correlation for the early prediction of sepsis. Here, the statistics of pairwise association for each hour-covariate pair within the EHR data for every 6-hours window-duration with selected 24 covariates is described using pointwise mutual information (PMI) matrix. This matrix gives the heterogeneity of data as a two-dimensional map. Such matrices are fused horizontally along the z-axis as vertical slices in the xy plane to form a 3-way tensor for each record with the corresponding Length of Stay (L). Tensor factorization of such fused tensor for every record is performed using Tucker decomposition, and only the core tensors are retained later, excluding the 3 unitary matrices to provide the latent feature set for the prediction of sepsis onset. A five-fold cross-validation scheme is employed wherein the obtained 120 latent features from the reshaped core tensor, are fed to Light Gradient Boosting Machine Learning models (LightGBM) for binary classification, further alleviating the involved class imbalance. The machine-learning framework is designed via Bayesian optimization, yielding an average normalized utility score of 0.4519 as defined by challenge organizers and area under the receiver operating characteristic curve (AUROC) of 0.8621 on publicly available PhysioNet/Computing in Cardiology Challenge 2019 training data. The proposed tensor decomposition of 3-way fused tensor formulated using PMI matrices leverages higher-order temporal interactions between the pairwise associations among the clinical values for early prediction of sepsis. This is validated with improved risk prediction power for every hour of admission to the ICU in terms of utility score, AUROC, and F1 score. The results obtained show a significant improvement particularly in terms of utility score of ~1.5-2% under a 5-fold cross-validation scheme on entire training data as compared to a top entrant research study that participated in the challenge.
Collapse
|
50
|
Huang B, Wang R, Masino AJ, Obstfeld AE. Aiding clinical assessment of neonatal sepsis using hematological analyzer data with machine learning techniques. Int J Lab Hematol 2021; 43:1341-1356. [PMID: 33949115 DOI: 10.1111/ijlh.13549] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2020] [Revised: 03/14/2021] [Accepted: 03/29/2021] [Indexed: 11/27/2022]
Abstract
INTRODUCTION Early diagnosis and antibiotic administration are essential for reducing sepsis morbidity and mortality; however, diagnosis remains difficult due to complex pathogenesis and presentation. We created a machine learning model for bacterial sepsis identification in the neonatal intensive care unit (NICU) using hematological analyzer data. METHODS Hematological analyzer data were gathered from NICU patients up to 48 hours prior to clinical evaluation for bacterial sepsis. Five models, Support Vector Machine, K-nearest-neighbors, Logistic Regression, Random Forest (RF), and Extreme Gradient boosting (XGBoost), were trained on 60 hematological and nine clinical variables for 2357 cases (1692 control, 665 septic). Clinical feature only models (nine variables) were additionally trained and compared with models including hematological variables. Feature importance was used to assess relative contributions of parameters to performance. RESULTS The three best performing models were RF, Logistic Regression, and XGBoost. RF achieved an average accuracy of 0.74, AUC-ROC of 0.73, Sensitivity of 0.38, and Specificity of 0.88. Logistic Regression achieved an average accuracy of 0.70, AUC-ROC of 0.74, Sensitivity of 0.62, and Specificity of 0.73. XGBoost achieved an average accuracy of 0.72, AUC-ROC of 0.71, Sensitivity of 0.40, and Specificity of 0.85. All models with hematological variables had significantly stronger performance than models trained on only clinical features. Neutrophil parameters had the highest average feature importance. CONCLUSIONS Machine learning models using hematological analyzer data can classify NICU patients as sepsis positive or negative with stronger performance compared to clinical feature only models. Hematological analyzer variables could augment current sepsis classification machine learning algorithms.
Collapse
Affiliation(s)
- Brian Huang
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Robin Wang
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Aaron J Masino
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.,Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, PA, USA
| | - Amrom E Obstfeld
- Department of Pathology and Laboratory Medicine, Children's Hospital of Philadelphia, Philadelphia, PA, USA
| |
Collapse
|