1
|
Islam KR, Prithula J, Kumar J, Tan TL, Reaz MBI, Sumon MSI, Chowdhury MEH. Machine Learning-Based Early Prediction of Sepsis Using Electronic Health Records: A Systematic Review. J Clin Med 2023; 12:5658. [PMID: 37685724 PMCID: PMC10488449 DOI: 10.3390/jcm12175658] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 08/13/2023] [Accepted: 08/23/2023] [Indexed: 09/10/2023] Open
Abstract
BACKGROUND Sepsis, a life-threatening infection-induced inflammatory condition, has significant global health impacts. Timely detection is crucial for improving patient outcomes as sepsis can rapidly progress to severe forms. The application of machine learning (ML) and deep learning (DL) to predict sepsis using electronic health records (EHRs) has gained considerable attention for timely intervention. METHODS PubMed, IEEE Xplore, Google Scholar, and Scopus were searched for relevant studies. All studies that used ML/DL to detect or early-predict the onset of sepsis in the adult population using EHRs were considered. Data were extracted and analyzed from all studies that met the criteria and were also evaluated for their quality. RESULTS This systematic review examined 1942 articles, selecting 42 studies while adhering to strict criteria. The chosen studies were predominantly retrospective (n = 38) and spanned diverse geographic settings, with a focus on the United States. Different datasets, sepsis definitions, and prevalence rates were employed, necessitating data augmentation. Heterogeneous parameter utilization, diverse model distribution, and varying quality assessments were observed. Longitudinal data enabled early sepsis prediction, and quality criteria fulfillment varied, with inconsistent funding-article quality correlation. CONCLUSIONS This systematic review underscores the significance of ML/DL methods for sepsis detection and early prediction through EHR data.
Collapse
Affiliation(s)
- Khandaker Reajul Islam
- Department of Physiology, Faculty of Medicine, University Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Johayra Prithula
- Department of Electrical and Electronics Engineering, University of Dhaka, Dhaka 1000, Bangladesh
| | - Jaya Kumar
- Department of Physiology, Faculty of Medicine, University Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Toh Leong Tan
- Department of Emergency Medicine, Faculty of Medicine, Universiti Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Mamun Bin Ibne Reaz
- Department of Electrical and Electronic Engineering, Independent University, Bangladesh Bashundhara, Dhaka 1229, Bangladesh
| | - Md. Shaheenur Islam Sumon
- Department of Biomedical Engineering, Military Institute of Science and Technology (MIST), Dhaka 1216, Bangladesh
| | | |
Collapse
|
2
|
Wang X, Guo Z, Chai Y, Wang Z, Liao H, Wang Z, Wang Z. Application Prospect of the SOFA Score and Related Modification Research Progress in Sepsis. J Clin Med 2023; 12:jcm12103493. [PMID: 37240599 DOI: 10.3390/jcm12103493] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 05/05/2023] [Accepted: 05/12/2023] [Indexed: 05/28/2023] Open
Abstract
In 2016, the SOFA score was proposed as the main evaluation system for diagnosis in the definition of sepsis 3.0, and the SOFA score has become a new research focus in sepsis. Some people are skeptical about diagnosing sepsis using the SOFA score. Experts and scholars from different regions have proposed different, modified versions of SOFA score to make up for the related problems with the use of the SOFA score in the diagnosis of sepsis. While synthesizing the different improved versions of SOFA proposed by experts and scholars in various regions, this paper also summarizes the relevant definitions of sepsis put forward in recent years to build a clear, improved application framework of SOFA score. In addition, the comparison between machine learning and SOFA scores related to sepsis is described and discussed in the article. Taken together, by summarizing the application of the improved SOFA score proposed in recent years in the related definition of sepsis, we believe that the SOFA score is still an effective means of diagnosing sepsis, but in the process of the continuous refinement and development of sepsis in the future, the SOFA score needs to be further refined and improved to provide more accurate coping strategies for different patient populations or application directions regarding sepsis. Against the big data background, machine learning has immeasurable value and significance, but its future applications should add more humanistic references and assistance.
Collapse
Affiliation(s)
- Xuesong Wang
- School of Clinical Medicine, Tsinghua University, Beijing 100190, China
| | - Zhe Guo
- School of Clinical Medicine, Tsinghua University, Beijing 100190, China
| | - Yan Chai
- School of Clinical Medicine, Tsinghua University, Beijing 100190, China
| | - Ziyi Wang
- School of Clinical Medicine, Tsinghua University, Beijing 100190, China
| | - Haiyan Liao
- School of Clinical Medicine, Tsinghua University, Beijing 100190, China
| | - Ziwen Wang
- School of Clinical Medicine, Tsinghua University, Beijing 100190, China
| | - Zhong Wang
- Beijing Tsinghua Changgung Hospital, Tsinghua University, Beijing 100084, China
| |
Collapse
|
3
|
Padmanabhan R, Elomri A, Taha RY, El Omri H, Elsabah H, El Omri A. Prediction of Multiple Clinical Complications in Cancer Patients to Ensure Hospital Preparedness and Improved Cancer Care. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 20:526. [PMID: 36612856 PMCID: PMC9819091 DOI: 10.3390/ijerph20010526] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/25/2022] [Revised: 11/22/2022] [Accepted: 12/04/2022] [Indexed: 06/17/2023]
Abstract
Reliable and rapid medical diagnosis is the cornerstone for improving the survival rate and quality of life of cancer patients. The problem of clinical decision-making pertaining to the management of patients with hematologic cancer is multifaceted and intricate due to the risk of therapy-induced myelosuppression, multiple infections, and febrile neutropenia (FN). Myelosuppression due to treatment increases the risk of sepsis and mortality in hematological cancer patients with febrile neutropenia. A high prevalence of multidrug-resistant organisms is also noted in such patients, which implies that these patients are left with limited or no-treatment options amidst severe health complications. Hence, early screening of patients for such organisms in their bodies is vital to enable hospital preparedness, curtail the spread to other weak patients in hospitals, and limit community outbreaks. Even though predictive models for sepsis and mortality exist, no model has been suggested for the prediction of multidrug-resistant organisms in hematological cancer patients with febrile neutropenia. Hence, for predicting three critical clinical complications, such as sepsis, the presence of multidrug-resistant organisms, and mortality, from the data available from medical records, we used 1166 febrile neutropenia episodes reported in 513 patients. The XGboost algorithm is suggested from 10-fold cross-validation on 6 candidate models. Other highlights are (1) a novel set of easily available features for the prediction of the aforementioned clinical complications and (2) the use of data augmentation methods and model-scoring-based hyperparameter tuning to address the problem of class disproportionality, a common challenge in medical datasets and often the reason behind poor event prediction rate of various predictive models reported so far. The proposed model depicts improved recall and AUC (area under the curve) for sepsis (recall = 98%, AUC = 0.85), multidrug-resistant organism (recall = 96%, AUC = 0.91), and mortality (recall = 86%, AUC = 0.88) prediction. Our results encourage the need to popularize artificial intelligence-based devices to support clinical decision-making.
Collapse
Affiliation(s)
- Regina Padmanabhan
- Division of Engineering Management and Decision Sciences, College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, Doha 34110, Qatar
| | - Adel Elomri
- Division of Engineering Management and Decision Sciences, College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, Doha 34110, Qatar
| | - Ruba Yasin Taha
- Department of Hematology and Bone Marrow Transplant, National Center for Cancer Care and Research, Hamad Medical Corporation, Doha 3050, Qatar
| | - Halima El Omri
- Department of Hematology and Bone Marrow Transplant, National Center for Cancer Care and Research, Hamad Medical Corporation, Doha 3050, Qatar
| | - Hesham Elsabah
- Department of Hematology and Bone Marrow Transplant, National Center for Cancer Care and Research, Hamad Medical Corporation, Doha 3050, Qatar
| | - Abdelfatteh El Omri
- Surgical Research Section, Department of Surgery, Hamad Medical Corporation, Doha 3050, Qatar
| |
Collapse
|
4
|
Hu J, Kang XH, Xu FF, Huang KZ, Du B, Weng L. Dynamic prediction of life-threatening events for patients in intensive care unit. BMC Med Inform Decis Mak 2022; 22:276. [PMID: 36273130 PMCID: PMC9587604 DOI: 10.1186/s12911-022-02026-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Accepted: 10/17/2022] [Indexed: 11/18/2022] Open
Abstract
Background Early prediction of patients’ deterioration is helpful in early intervention for patients at greater risk of deterioration in Intensive Care Unit (ICU). This study aims to apply machine learning approaches to heterogeneous clinical data for predicting life-threatening events of patients in ICU.
Methods We collected clinical data from a total of 3151 patients admitted to the Medical Intensive Care Unit of Peking Union Medical College Hospital in China from January 1st, 2014, to October 1st, 2019. After excluding the patients who were under 18 years old or stayed less than 24 h at the ICU, a total of 2170 patients were enrolled in this study. Multiple machine learning approaches were utilized to predict life-threatening events (i.e., death) in seven 24-h windows (day 1 to day 7) and their performance was compared. Results Light Gradient Boosting Machine showed the best performance. We found that life-threatening events during the short-term windows can be better predicted than those in the medium-term windows. For example, death in 24 h can be predicted with an Area Under Curve of 0.905. Features like infusion pump related fluid input were highly related to life-threatening events. Furthermore, the prediction power of static features such as age and cardio-pulmonary function increased with the extended prediction window. Conclusion This study demonstrates that the integration of machine learning approaches and large-scale high-quality clinical data in ICU could accurately predict life-threatening events for ICU patients for early intervention. Supplementary Information The online version contains supplementary material available at 10.1186/s12911-022-02026-x.
Collapse
Affiliation(s)
- Jiang Hu
- Medical Intensive Care Unit, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, 1 Shuai Fu Yuan, Beijing, 100730, China.,Hangzhou Maicim Medical Tech Co., Ltd, Hangzhou, Zhejiang, China
| | - Xiao-Hui Kang
- Medical Intensive Care Unit, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, 1 Shuai Fu Yuan, Beijing, 100730, China
| | - Fang-Fang Xu
- Hangzhou Maicim Medical Tech Co., Ltd, Hangzhou, Zhejiang, China
| | - Ke-Zhi Huang
- Hangzhou Maicim Medical Tech Co., Ltd, Hangzhou, Zhejiang, China
| | - Bin Du
- Medical Intensive Care Unit, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, 1 Shuai Fu Yuan, Beijing, 100730, China
| | - Li Weng
- Medical Intensive Care Unit, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, 1 Shuai Fu Yuan, Beijing, 100730, China.
| |
Collapse
|
5
|
Prior Distribution Estimation of Monitored Information in the Intensive Care Unit with the Hidden Markov Model and Decision Tree Methods. JOURNAL OF HEALTHCARE ENGINEERING 2022; 2022:7892408. [PMID: 35368916 PMCID: PMC8970853 DOI: 10.1155/2022/7892408] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/07/2021] [Accepted: 03/15/2022] [Indexed: 11/17/2022]
Abstract
In the intensive care unit, the monitored variables collected from sensors may have different behaviors among patients with different clinical basic information. Giving prior information of the monitored variables based on their specific basic information as soon as the patient is admitted will support the clinicians with better decisions during the surgery. Instead of black box models, the explainable hidden Markov model is proposed, which can estimate the possible distribution parameters of the monitored variables under different clinical basic information. A Student's t-test or correlation test is conducted further to test whether the parameters have a significant relationship with the basic variables. The specific relationship is explored by using a conditional inference tree, which is an explainable model giving deciding rules. Instead of point estimation, interval forecast is chosen as the performance metrics including coverage rate and relative interval width, which provide more reliable results. By applying the methods to an intensive care unit data set with more than 20 thousand patients, the model has good performance with an area under the ROC Curve value of 0.75, which means the hidden states can generally be correctly labelled. The significant test shows that only a few combinations of the basic and monitored variables are not significant under the 0.01 significant level. The tree model based on different quantile intervals provides different coverage and width combination choices. A coverage rate around 0.8 is suggested, which has a relative interval width of 0.77.
Collapse
|
6
|
Schinkel M, Nanayakkara PWB, Wiersinga WJ. Sepsis Performance Improvement Programs: From Evidence Toward Clinical Implementation. Crit Care 2022; 26:77. [PMID: 35337358 PMCID: PMC8951662 DOI: 10.1186/s13054-022-03917-1] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
This article is one of ten reviews selected from the Annual Update in Intensive Care and Emergency Medicine 2022. Other selected articles can be found online at https://www.biomedcentral.com/collections/annualupdate2022. Further information about the Annual Update in Intensive Care and Emergency Medicine is available from https://link.springer.com/bookseries/8901.
Collapse
Affiliation(s)
- Michiel Schinkel
- Center for Experimental and Molecular Medicine, Amsterdam UMC, Location Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands.,Section General Internal Medicine, Department of Internal Medicine, Amsterdam Public Health Research Institute, Amsterdam UMC, Location VU University Medical Center, Amsterdam, The Netherlands
| | - Prabath W B Nanayakkara
- Section General Internal Medicine, Department of Internal Medicine, Amsterdam Public Health Research Institute, Amsterdam UMC, Location VU University Medical Center, Amsterdam, The Netherlands
| | - W Joost Wiersinga
- Center for Experimental and Molecular Medicine, Amsterdam UMC, Location Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands. .,Department of Medicine, Division of Infectious Diseases, Amsterdam UMC, Location Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands.
| |
Collapse
|
7
|
Arriaga-Pizano LA, Gonzalez-Olvera MA, Ferat-Osorio EA, Escobar J, Hernandez-Perez AL, Revilla-Monsalve C, Lopez-Macias C, León-Pedroza JI, Cabrera-Rivera GL, Guadarrama-Aranda U, Leder R, Gallardo-Hernandez AG. Accurate diagnosis of sepsis using a neural network: Pilot study using routine clinical variables. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 210:106366. [PMID: 34500141 DOI: 10.1016/j.cmpb.2021.106366] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/16/2021] [Accepted: 08/17/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVES Sepsis is a severe infection that increases mortality risk and is one if the main causes of death in intensive care units. Accurate detection is key to successful interventions, but diagnosis of sepsis is complicated because the initial signs and symptoms are not specific. Biomarkers that have been proposed have low specificity and sensitivity, are expensive, and not available in every hospital. In this study, we propose the use of artificial intelligence in the form of a neural network to diagnose sepsis using only common laboratory tests and vital signs that are routine and widely available. METHODS A retrospective, cross sectional cohort of 113 patients from an intensive care unit, each with 48 routinely evaluated vital signs and biochemical parameters was used to train, validate and test a neural network with 48 inputs, 10 neurons in a single hidden layer and one output. The sensitivity and specificity of the neural network as a point sampled diagnostic test was calculated. RESULTS All but one case were correctly diagnosed by the neural network, with 91% sensitivity and 100% specificity in the validation data set, and 100% sensitivity and specificity in the test data set. CONCLUSIONS The designed neural network system can identify patients with sepsis, with minimal resources using standard laboratory tests widely available in most health care facilities. This should reduce the burden on the medical staff of a difficult diagnosis and should improve outcomes for patients with sepsis.
Collapse
Affiliation(s)
- Lourdes Andrea Arriaga-Pizano
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | - Marcos Angel Gonzalez-Olvera
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | - Eduardo Antonio Ferat-Osorio
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | - Jesica Escobar
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | - Ana Luisa Hernandez-Perez
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | - Cristina Revilla-Monsalve
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | - Constatino Lopez-Macias
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | - José Israel León-Pedroza
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | - Graciela Libier Cabrera-Rivera
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | - Uriel Guadarrama-Aranda
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | - Ron Leder
- Instituto Mexicano del Seguro Social, Centro Medico Nacional Siglo XXI, Hospital de cardiología, Mexico 0672, DF, Mexico
| | | |
Collapse
|
8
|
Tabaie A, Orenstein EW, Nemati S, Basu RK, Clifford GD, Kamaleswaran R. Deep Learning Model to Predict Serious Infection Among Children With Central Venous Lines. Front Pediatr 2021; 9:726870. [PMID: 34604142 PMCID: PMC8480258 DOI: 10.3389/fped.2021.726870] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Accepted: 08/06/2021] [Indexed: 12/23/2022] Open
Abstract
Objective: Predict the onset of presumed serious infection, defined as a positive blood culture drawn and new antibiotic course of at least 4 days (PSI*), among pediatric patients with Central Venous Lines (CVLs). Design: Retrospective cohort study. Setting: Single academic children's hospital. Patients: All hospital encounters from January 2013 to December 2018, excluding the ones without a CVL or with a length-of-stay shorter than 24 h. Measurements and Main Results: Clinical features including demographics, laboratory results, vital signs, characteristics of the CVLs and medications used were extracted retrospectively from electronic medical records. Data were aggregated across all hospitals within a single pediatric health system and used to train a deep learning model to predict the occurrence of PSI* during the next 48 h of hospitalization. The proposed model prediction was compared to prediction of PSI* by a marker of illness severity (PELOD-2). The baseline prevalence of line infections was 0.34% over all segmented 48-h time windows. Events were identified among cases using onset time. All data from admission till the onset was used for cases and among controls we used all data from admission till discharge. The benchmarks were aggregated over all 48 h time windows [N=748,380 associated with 27,137 patient encounters]. The model achieved an area under the receiver operating characteristic curve of 0.993 (95% CI = [0.990, 0.996]), the enriched positive predictive value (PPV) was 23 times greater than the base prevalence. Conversely, prediction by PELOD-2 achieved a lower PPV of 1.5% [0.9%, 2.1%] which was 5 times the baseline prevalence. Conclusion: A deep learning model that employs common clinical features in the electronic health record can help predict the onset of CLABSI in hospitalized children with central venous line 48 hours prior to the time of specimen collection.
Collapse
Affiliation(s)
- Azade Tabaie
- Department of Biomedical Informatics, Emory School of Medicine, Atlanta, GA, United States
| | - Evan W. Orenstein
- Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, United States
| | - Shamim Nemati
- Department of Biomedical Informatics, University of California, San Diego, San Diego, CA, United States
| | - Rajit K. Basu
- Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, United States
| | - Gari D. Clifford
- Department of Biomedical Informatics, Emory School of Medicine, Atlanta, GA, United States
- Department of Biomedical Engineering, Georgia Institute of Technology and Emory School of Medicine, Atlanta, GA, United States
| | - Rishikesan Kamaleswaran
- Department of Biomedical Informatics, Emory School of Medicine, Atlanta, GA, United States
- Department of Biomedical Engineering, Georgia Institute of Technology and Emory School of Medicine, Atlanta, GA, United States
| |
Collapse
|
9
|
Moor M, Rieck B, Horn M, Jutzeler CR, Borgwardt K. Early Prediction of Sepsis in the ICU Using Machine Learning: A Systematic Review. Front Med (Lausanne) 2021; 8:607952. [PMID: 34124082 PMCID: PMC8193357 DOI: 10.3389/fmed.2021.607952] [Citation(s) in RCA: 39] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2020] [Accepted: 03/04/2021] [Indexed: 12/12/2022] Open
Abstract
Background: Sepsis is among the leading causes of death in intensive care units (ICUs) worldwide and its recognition, particularly in the early stages of the disease, remains a medical challenge. The advent of an affluence of available digital health data has created a setting in which machine learning can be used for digital biomarker discovery, with the ultimate goal to advance the early recognition of sepsis. Objective: To systematically review and evaluate studies employing machine learning for the prediction of sepsis in the ICU. Data Sources: Using Embase, Google Scholar, PubMed/Medline, Scopus, and Web of Science, we systematically searched the existing literature for machine learning-driven sepsis onset prediction for patients in the ICU. Study Eligibility Criteria: All peer-reviewed articles using machine learning for the prediction of sepsis onset in adult ICU patients were included. Studies focusing on patient populations outside the ICU were excluded. Study Appraisal and Synthesis Methods: A systematic review was performed according to the PRISMA guidelines. Moreover, a quality assessment of all eligible studies was performed. Results: Out of 974 identified articles, 22 and 21 met the criteria to be included in the systematic review and quality assessment, respectively. A multitude of machine learning algorithms were applied to refine the early prediction of sepsis. The quality of the studies ranged from "poor" (satisfying ≤ 40% of the quality criteria) to "very good" (satisfying ≥ 90% of the quality criteria). The majority of the studies (n = 19, 86.4%) employed an offline training scenario combined with a horizon evaluation, while two studies implemented an online scenario (n = 2, 9.1%). The massive inter-study heterogeneity in terms of model development, sepsis definition, prediction time windows, and outcomes precluded a meta-analysis. Last, only two studies provided publicly accessible source code and data sources fostering reproducibility. Limitations: Articles were only eligible for inclusion when employing machine learning algorithms for the prediction of sepsis onset in the ICU. This restriction led to the exclusion of studies focusing on the prediction of septic shock, sepsis-related mortality, and patient populations outside the ICU. Conclusions and Key Findings: A growing number of studies employs machine learning to optimize the early prediction of sepsis through digital biomarker discovery. This review, however, highlights several shortcomings of the current approaches, including low comparability and reproducibility. Finally, we gather recommendations how these challenges can be addressed before deploying these models in prospective analyses. Systematic Review Registration Number: CRD42020200133.
Collapse
Affiliation(s)
- Michael Moor
- Machine Learning and Computational Biology Lab, Department of Biosystems Science and Engineering, Eidgenössische Technische Hochschule Zürich (ETH Zurich), Basel, Switzerland
- SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland
| | - Bastian Rieck
- Machine Learning and Computational Biology Lab, Department of Biosystems Science and Engineering, Eidgenössische Technische Hochschule Zürich (ETH Zurich), Basel, Switzerland
- SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland
| | - Max Horn
- Machine Learning and Computational Biology Lab, Department of Biosystems Science and Engineering, Eidgenössische Technische Hochschule Zürich (ETH Zurich), Basel, Switzerland
- SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland
| | - Catherine R. Jutzeler
- Machine Learning and Computational Biology Lab, Department of Biosystems Science and Engineering, Eidgenössische Technische Hochschule Zürich (ETH Zurich), Basel, Switzerland
- SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland
| | - Karsten Borgwardt
- Machine Learning and Computational Biology Lab, Department of Biosystems Science and Engineering, Eidgenössische Technische Hochschule Zürich (ETH Zurich), Basel, Switzerland
- SIB Swiss Institute of Bioinformatics, Lausanne, Switzerland
| |
Collapse
|
10
|
Schvetz M, Fuchs L, Novack V, Moskovitch R. Outcomes prediction in longitudinal data: Study designs evaluation, use case in ICU acquired sepsis. J Biomed Inform 2021; 117:103734. [PMID: 33711544 DOI: 10.1016/j.jbi.2021.103734] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2020] [Revised: 02/27/2021] [Accepted: 03/01/2021] [Indexed: 12/23/2022]
Abstract
Outcomes' prediction in Electronic Health Records (EHR) and specifically in Critical Care is increasingly attracting more exploration and research. In this study, we used clinical data from the Intensive Care Unit (ICU), focusing on ICU acquired sepsis. Looking at the current literature, several evaluation approaches are reported, inspired by epidemiological designs, in which some do not always reflect real-life application's conditions. This problem seems relevant generally to outcomes' prediction in longitudinal EHR data, or generally longitudinal data, while in this study we focused on ICU data. Unlike in most previous studies that investigated all sepsis admissions, we focused specifically on ICU-Acquired Sepsis. Due to the sparse nature of the longitudinal data, we employed the use of Temporal Abstraction and Time Interval-Related Patterns discovery, which are further used as classification features. Two experiments were designed using three different outcomes prediction study designs from the literature, implementing various levels of real-life conditions to evaluate the prediction models. The first experiment focused on predicting whether a patient would suffer from ICU-acquired sepsis and when during her admission, given a sliding observation time window, and the comparison of the three study designs behavior. The second experiment focused only on predicting whether the patient will suffer from ICU-acquired sepsis, based on data taken relatively to his admission start time. Our results show that using Temporal Discretization for Classification (TD4C) led to better performance than using the Equal-Width Discretization, Knowledge-Based, or SAX. Also, using two states abstraction was better than three or four. Using the default Binary TIRP representation method performed better than Mean Duration, Horizontal Support, and horizontally normalized horizontal support. Using XGBoost as a classifier performed better than Logistic Regression, Neural Net, or Random Forest. Additionally, it is demonstrated why the use of case-crossover-control is most appropriate for real life application conditions evaluation, unlike other incomplete designs that may even result in "better performance".
Collapse
Affiliation(s)
- Maya Schvetz
- Department of Software and Information Systems Engineering, Ben Gurion University of the Negev, Beer-Sheva, Israel.
| | - Lior Fuchs
- Medical Intensive Care Unit and Clinical Research Center, Soroka University Medical Center, Faculty of Health Sciences, Ben-Gurion University of the Negev, Beer-Sheva, Israel.
| | - Victor Novack
- Clinical Research Center, Soroka University Medical Center, Faculty of Health Sciences, Ben-Gurion University of the Negev, Beer-Sheva, Israel.
| | - Robert Moskovitch
- Department of Software and Information Systems Engineering, Ben Gurion University of the Negev, Beer-Sheva, Israel.
| |
Collapse
|
11
|
Liu Z, Khojandi A, Mohammed A, Li X, Chinthala LK, Davis RL, Kamaleswaran R. HeMA: A hierarchically enriched machine learning approach for managing false alarms in real time: A sepsis prediction case study. Comput Biol Med 2021; 131:104255. [PMID: 33639353 DOI: 10.1016/j.compbiomed.2021.104255] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2020] [Revised: 02/01/2021] [Accepted: 02/03/2021] [Indexed: 01/09/2023]
Abstract
Early detection of sepsis can be life-saving. Machine learning models have shown great promise in early sepsis prediction when applied to patient physiological data in real-time. However, these existing models often under-perform in terms of positive predictive value, an important metric in clinical settings. This is especially the case when the models are applied to data with less than 50% sepsis prevalence, reflective of the incidence rate of sepsis on the floor or in the ICU. In this study, we develop HeMA, a hierarchically enriched machine learning approach for managing false alarms in real time, and conduct a case study for early sepsis prediction. Specifically, we develop a two-stage framework, where a first stage machine learning model is paired with statistical tests, particularly Kolmogorov-Smirnov tests, in the second stage, to predict whether a patient would develop sepsis. Compared with machine learning models alone, the framework results in an increase in specificity and positive predictive value, without compromising F1 score. In particular, the framework shows improved performance when applied to data with 50% and 25% sepsis prevalence, collected from a large hospital system in the US, resulting in up to 18% and 7% increase in specificity and positive predictive value, respectively. Despite the significant improvements observed, and although F1 score is not negatively affected, because of the up to 6% decrease in sensitivity, further improvements and pilot studies may be necessary before deploying the framework in a clinical setting. Finally, external validation conducted using a publicly available dataset produces similar results, validating that the proposed framework is generalizable.
Collapse
Affiliation(s)
- Zeyu Liu
- Department of Industrial & Systems Engineering, The University of Tennessee, Knoxville, TN, 37996, USA
| | - Anahita Khojandi
- Department of Industrial & Systems Engineering, The University of Tennessee, Knoxville, TN, 37996, USA.
| | - Akram Mohammed
- Center for Biomedical Informatics-Pediatrics, The University of Tennessee Health Science Center, Memphis, TN, 38103, USA
| | - Xueping Li
- Department of Industrial & Systems Engineering, The University of Tennessee, Knoxville, TN, 37996, USA
| | - Lokesh K Chinthala
- Center for Biomedical Informatics-Pediatrics, The University of Tennessee Health Science Center, Memphis, TN, 38103, USA
| | - Robert L Davis
- Center for Biomedical Informatics-Pediatrics, The University of Tennessee Health Science Center, Memphis, TN, 38103, USA
| | - Rishikesan Kamaleswaran
- Departments of Biomedical Informatics, Pediatrics, and Emergency Medicine, Emory University School of Medicine, Atlanta, GA, 30322, USA
| |
Collapse
|
12
|
Giacobbe DR, Signori A, Del Puente F, Mora S, Carmisciano L, Briano F, Vena A, Ball L, Robba C, Pelosi P, Giacomini M, Bassetti M. Early Detection of Sepsis With Machine Learning Techniques: A Brief Clinical Perspective. Front Med (Lausanne) 2021; 8:617486. [PMID: 33644097 PMCID: PMC7906970 DOI: 10.3389/fmed.2021.617486] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2020] [Accepted: 01/19/2021] [Indexed: 12/15/2022] Open
Abstract
Sepsis is a major cause of death worldwide. Over the past years, prediction of clinically relevant events through machine learning models has gained particular attention. In the present perspective, we provide a brief, clinician-oriented vision on the following relevant aspects concerning the use of machine learning predictive models for the early detection of sepsis in the daily practice: (i) the controversy of sepsis definition and its influence on the development of prediction models; (ii) the choice and availability of input features; (iii) the measure of the model performance, the output, and their usefulness in the clinical practice. The increasing involvement of artificial intelligence and machine learning in health care cannot be disregarded, despite important pitfalls that should be always carefully taken into consideration. In the long run, a rigorous multidisciplinary approach to enrich our understanding in the application of machine learning techniques for the early recognition of sepsis may show potential to augment medical decision-making when facing this heterogeneous and complex syndrome.
Collapse
Affiliation(s)
- Daniele Roberto Giacobbe
- Infectious Diseases Unit, San Martino Policlinico Hospital – IRCCS for Oncology and Neurosciences, Genoa, Italy
- Department of Health Sciences (DISSAL), University of Genoa, Genoa, Italy
| | - Alessio Signori
- Department of Health Sciences (DISSAL), University of Genoa, Genoa, Italy
| | - Filippo Del Puente
- Department of Health Sciences (DISSAL), University of Genoa, Genoa, Italy
| | - Sara Mora
- Department of Informatics Bioengineering, Robotics, and Systems Engineering (DIBRIS), University of Genoa, Genoa, Italy
| | - Luca Carmisciano
- Department of Health Sciences (DISSAL), University of Genoa, Genoa, Italy
| | - Federica Briano
- Infectious Diseases Unit, San Martino Policlinico Hospital – IRCCS for Oncology and Neurosciences, Genoa, Italy
- Department of Health Sciences (DISSAL), University of Genoa, Genoa, Italy
| | - Antonio Vena
- Infectious Diseases Unit, San Martino Policlinico Hospital – IRCCS for Oncology and Neurosciences, Genoa, Italy
| | - Lorenzo Ball
- Anaesthesia and Intensive Care, San Martino Policlinico Hospital – IRCCS for Oncology and Neurosciences, Genoa, Italy
- Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Chiara Robba
- Anaesthesia and Intensive Care, San Martino Policlinico Hospital – IRCCS for Oncology and Neurosciences, Genoa, Italy
- Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Paolo Pelosi
- Anaesthesia and Intensive Care, San Martino Policlinico Hospital – IRCCS for Oncology and Neurosciences, Genoa, Italy
- Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy
| | - Mauro Giacomini
- Department of Informatics Bioengineering, Robotics, and Systems Engineering (DIBRIS), University of Genoa, Genoa, Italy
| | - Matteo Bassetti
- Infectious Diseases Unit, San Martino Policlinico Hospital – IRCCS for Oncology and Neurosciences, Genoa, Italy
- Department of Health Sciences (DISSAL), University of Genoa, Genoa, Italy
| |
Collapse
|
13
|
Sakib N, Ahamed SI, Khan RA, Griffin PM, Haque MM. Unpacking Prevalence and Dichotomy in Quick Sequential Organ Failure Assessment and Systemic Inflammatory Response Syndrome Parameters: Observational Data-Driven Approach Backed by Sepsis Pathophysiology. JMIR Med Inform 2020; 8:e18352. [PMID: 33270030 PMCID: PMC7746497 DOI: 10.2196/18352] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2020] [Revised: 08/10/2020] [Accepted: 09/15/2020] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND Considering morbidity, mortality, and annual treatment costs, the dramatic rise in the incidence of sepsis and septic shock among intensive care unit (ICU) admissions in US hospitals is an increasing concern. Recent changes in the sepsis definition (sepsis-3), based on the quick Sequential Organ Failure Assessment (qSOFA), have motivated the international medical informatics research community to investigate score recalculation and information retrieval, and to study the intersection between sepsis-3 and the previous definition (sepsis-2) based on systemic inflammatory response syndrome (SIRS) parameters. OBJECTIVE The objective of this study was three-fold. First, we aimed to unpack the most prevalent criterion for sepsis (for both sepsis-3 and sepsis-2 predictors). Second, we intended to determine the most prevalent sepsis scenario in the ICU among 4 possible scenarios for qSOFA and 11 possible scenarios for SIRS. Third, we investigated the multicollinearity or dichotomy among qSOFA and SIRS predictors. METHODS This observational study was conducted according to the most recent update of Medical Information Mart for Intensive Care (MIMIC-III, Version 1.4), the critical care database developed by MIT. The qSOFA (sepsis-3) and SIRS (sepsis-2) parameters were analyzed for patients admitted to critical care units from 2001 to 2012 in Beth Israel Deaconess Medical Center (Boston, MA, USA) to determine the prevalence and underlying relation between these parameters among patients undergoing sepsis screening. We adopted a multiblind Delphi method to seek a rationale for decisions in several stages of the research design regarding handling missing data and outlier values, statistical imputations and biases, and generalizability of the study. RESULTS Altered mental status in the Glasgow Coma Scale (59.28%, 38,854/65,545 observations) was the most prevalent sepsis-3 (qSOFA) criterion and the white blood cell count (53.12%, 17,163/32,311 observations) was the most prevalent sepsis-2 (SIRS) criterion confronted in the ICU. In addition, the two-factored sepsis criterion of high respiratory rate (≥22 breaths/minute) and altered mental status (28.19%, among four possible qSOFA scenarios besides no sepsis) was the most prevalent sepsis-3 (qSOFA) scenario, and the three-factored sepsis criterion of tachypnea, high heart rate, and high white blood cell count (12.32%, among 11 possible scenarios besides no sepsis) was the most prevalent sepsis-2 (SIRS) scenario in the ICU. Moreover, the absolute Pearson correlation coefficients were not significant, thereby nullifying the likelihood of any linear correlation among the critical parameters and assuring the lack of multicollinearity between the parameters. Although this further bolsters evidence for their dichotomy, the absence of multicollinearity cannot guarantee that two random variables are statistically independent. CONCLUSIONS Quantifying the prevalence of the qSOFA criteria of sepsis-3 in comparison with the SIRS criteria of sepsis-2, and understanding the underlying dichotomy among these parameters provides significant inferences for sepsis treatment initiatives in the ICU and informing hospital resource allocation. These data-driven results further offer design implications for multiparameter intelligent sepsis prediction in the ICU.
Collapse
Affiliation(s)
- Nazmus Sakib
- Ubicomp Lab, Department of Computer Science, Marquette University, Milwaukee, WI, United States
| | - Sheikh Iqbal Ahamed
- Ubicomp Lab, Department of Computer Science, Marquette University, Milwaukee, WI, United States
| | - Rumi Ahmed Khan
- College of Medicine, University of Central Florida, Orlando, FL, United States
| | - Paul M Griffin
- Regenstrief Center for Healthcare Engineering, Purdue University, West Lafayette, IN, United States
| | - Md Munirul Haque
- RB Annis School of Engineering, University of Indianapolis, Indianapolis, IN, United States
| |
Collapse
|
14
|
Fleuren LM, Klausch TLT, Zwager CL, Schoonmade LJ, Guo T, Roggeveen LF, Swart EL, Girbes ARJ, Thoral P, Ercole A, Hoogendoorn M, Elbers PWG. Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med 2020; 46:383-400. [PMID: 31965266 PMCID: PMC7067741 DOI: 10.1007/s00134-019-05872-y] [Citation(s) in RCA: 260] [Impact Index Per Article: 65.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2019] [Accepted: 11/16/2019] [Indexed: 12/19/2022]
Abstract
PURPOSE Early clinical recognition of sepsis can be challenging. With the advancement of machine learning, promising real-time models to predict sepsis have emerged. We assessed their performance by carrying out a systematic review and meta-analysis. METHODS A systematic search was performed in PubMed, Embase.com and Scopus. Studies targeting sepsis, severe sepsis or septic shock in any hospital setting were eligible for inclusion. The index test was any supervised machine learning model for real-time prediction of these conditions. Quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, with a tailored Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist to evaluate risk of bias. Models with a reported area under the curve of the receiver operating characteristic (AUROC) metric were meta-analyzed to identify strongest contributors to model performance. RESULTS After screening, a total of 28 papers were eligible for synthesis, from which 130 models were extracted. The majority of papers were developed in the intensive care unit (ICU, n = 15; 54%), followed by hospital wards (n = 7; 25%), the emergency department (ED, n = 4; 14%) and all of these settings (n = 2; 7%). For the prediction of sepsis, diagnostic test accuracy assessed by the AUROC ranged from 0.68-0.99 in the ICU, to 0.96-0.98 in-hospital and 0.87 to 0.97 in the ED. Varying sepsis definitions limit pooling of the performance across studies. Only three papers clinically implemented models with mixed results. In the multivariate analysis, temperature, lab values, and model type contributed most to model performance. CONCLUSION This systematic review and meta-analysis show that on retrospective data, individual machine learning models can accurately predict sepsis onset ahead of time. Although they present alternatives to traditional scoring systems, between-study heterogeneity limits the assessment of pooled results. Systematic reporting and clinical implementation studies are needed to bridge the gap between bytes and bedside.
Collapse
Affiliation(s)
- Lucas M Fleuren
- Department of Intensive Care Medicine, Research VUmc Intensive Care (REVIVE), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Sciences (ACS), Amsterdam Infection and Immunity Institute (AI&II), Amsterdam UMC, location VUmc, VU Amsterdam, Amsterdam, The Netherlands.
- Computational Intelligence Group, Department of Computer Science, VU Amsterdam, Amsterdam, The Netherlands.
| | - Thomas L T Klausch
- Department of Epidemiology and Biostatistics, Amsterdam UMC, location VUmc, VU Amsterdam, Amsterdam, The Netherlands
| | - Charlotte L Zwager
- Department of Intensive Care Medicine, Research VUmc Intensive Care (REVIVE), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Sciences (ACS), Amsterdam Infection and Immunity Institute (AI&II), Amsterdam UMC, location VUmc, VU Amsterdam, Amsterdam, The Netherlands
| | - Linda J Schoonmade
- Medical Library, Amsterdam UMC, location VUmc, VU Amsterdam, Amsterdam, The Netherlands
| | - Tingjie Guo
- Department of Intensive Care Medicine, Research VUmc Intensive Care (REVIVE), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Sciences (ACS), Amsterdam Infection and Immunity Institute (AI&II), Amsterdam UMC, location VUmc, VU Amsterdam, Amsterdam, The Netherlands
| | - Luca F Roggeveen
- Department of Intensive Care Medicine, Research VUmc Intensive Care (REVIVE), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Sciences (ACS), Amsterdam Infection and Immunity Institute (AI&II), Amsterdam UMC, location VUmc, VU Amsterdam, Amsterdam, The Netherlands
- Computational Intelligence Group, Department of Computer Science, VU Amsterdam, Amsterdam, The Netherlands
| | - Eleonora L Swart
- Department of Pharmacy, Amsterdam UMC, location VUmc, VU Amsterdam, Amsterdam, The Netherlands
| | - Armand R J Girbes
- Department of Intensive Care Medicine, Research VUmc Intensive Care (REVIVE), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Sciences (ACS), Amsterdam Infection and Immunity Institute (AI&II), Amsterdam UMC, location VUmc, VU Amsterdam, Amsterdam, The Netherlands
| | - Patrick Thoral
- Department of Intensive Care Medicine, Research VUmc Intensive Care (REVIVE), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Sciences (ACS), Amsterdam Infection and Immunity Institute (AI&II), Amsterdam UMC, location VUmc, VU Amsterdam, Amsterdam, The Netherlands
| | - Ari Ercole
- Division of Anaesthesia, University of Cambridge, Cambridge, UK
- Data Science Section, European Society of Intensive Care Medicine, Brussels, Belgium
| | - Mark Hoogendoorn
- Computational Intelligence Group, Department of Computer Science, VU Amsterdam, Amsterdam, The Netherlands
| | - Paul W G Elbers
- Department of Intensive Care Medicine, Research VUmc Intensive Care (REVIVE), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Sciences (ACS), Amsterdam Infection and Immunity Institute (AI&II), Amsterdam UMC, location VUmc, VU Amsterdam, Amsterdam, The Netherlands
- Data Science Section, European Society of Intensive Care Medicine, Brussels, Belgium
| |
Collapse
|
15
|
Luz CF, Vollmer M, Decruyenaere J, Nijsten MW, Glasner C, Sinha B. Machine learning in infection management using routine electronic health records: tools, techniques, and reporting of future technologies. Clin Microbiol Infect 2020; 26:1291-1299. [PMID: 32061798 DOI: 10.1016/j.cmi.2020.02.003] [Citation(s) in RCA: 46] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Revised: 02/01/2020] [Accepted: 02/03/2020] [Indexed: 12/12/2022]
Abstract
BACKGROUND Machine learning (ML) is increasingly being used in many areas of health care. Its use in infection management is catching up as identified in a recent review in this journal. We present here a complementary review to this work. OBJECTIVES To support clinicians and researchers in navigating through the methodological aspects of ML approaches in the field of infection management. SOURCES A Medline search was performed with the keywords artificial intelligence, machine learning, infection∗, and infectious disease∗ for the years 2014-2019. Studies using routinely available electronic hospital record data from an inpatient setting with a focus on bacterial and fungal infections were included. CONTENT Fifty-two studies were included and divided into six groups based on their focus. These studies covered detection/prediction of sepsis (n = 19), hospital-acquired infections (n = 11), surgical site infections and other postoperative infections (n = 11), microbiological test results (n = 4), infections in general (n = 2), musculoskeletal infections (n = 2), and other topics (urinary tract infections, deep fungal infections, antimicrobial prescriptions; n = 1 each). In total, 35 different ML techniques were used. Logistic regression was applied in 18 studies followed by random forest, support vector machines, and artificial neural networks in 18, 12, and seven studies, respectively. Overall, the studies were very heterogeneous in their approach and their reporting. Detailed information on data handling and software code was often missing. Validation on new datasets and/or in other institutions was rarely done. Clinical studies on the impact of ML in infection management were lacking. IMPLICATIONS Promising approaches for ML use in infectious diseases were identified. But building trust in these new technologies will require improved reporting. Explainability and interpretability of the models used were rarely addressed and should be further explored. Independent model validation and clinical studies evaluating the added value of ML approaches are needed.
Collapse
Affiliation(s)
- C F Luz
- University of Groningen, University Medical Center Groningen, Department of Medical Microbiology and Infection Prevention, Groningen, the Netherlands.
| | - M Vollmer
- Institute of Bioinformatics, University Medicine Greifswald, Greifswald, Germany
| | - J Decruyenaere
- Ghent University, Ghent University Hospital, Department of Intensive Care, Ghent, Belgium
| | - M W Nijsten
- University of Groningen, University Medical Center Groningen, Department of Critical Care, Groningen, the Netherlands
| | - C Glasner
- University of Groningen, University Medical Center Groningen, Department of Medical Microbiology and Infection Prevention, Groningen, the Netherlands
| | - B Sinha
- University of Groningen, University Medical Center Groningen, Department of Medical Microbiology and Infection Prevention, Groningen, the Netherlands
| |
Collapse
|
16
|
Schinkel M, Paranjape K, Nannan Panday RS, Skyttberg N, Nanayakkara PWB. Clinical applications of artificial intelligence in sepsis: A narrative review. Comput Biol Med 2019; 115:103488. [PMID: 31634699 DOI: 10.1016/j.compbiomed.2019.103488] [Citation(s) in RCA: 55] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2019] [Revised: 09/25/2019] [Accepted: 10/05/2019] [Indexed: 12/27/2022]
Abstract
Many studies have been published on a variety of clinical applications of artificial intelligence (AI) for sepsis, while there is no overview of the literature. The aim of this review is to give an overview of the literature and thereby identify knowledge gaps and prioritize areas with high priority for further research. A literature search was conducted in PubMed from inception to February 2019. Search terms related to AI were combined with terms regarding sepsis. Articles were included when they reported an area under the receiver operator characteristics curve (AUROC) as outcome measure. Fifteen articles on diagnosis of sepsis with AI models were included. The best performing model reached an AUROC of 0.97. There were also seven articles on prognosis, predicting mortality over time with an AUROC of up to 0.895. Finally, there were three articles on assistance of treatment of sepsis, where the use of AI was associated with the lowest mortality rates. Of the articles, twenty-two were judged to be at high risk of bias or had major concerns regarding applicability. This was mostly because predictor variables in these models, such as blood pressure, were also part of the definition of sepsis, which led to overestimation of the performance. We conclude that AI models have great potential for improving early identification of patients who may benefit from administration of antibiotics. Current AI prediction models to diagnose sepsis are at major risks of bias when the diagnosis criteria are part of the predictor variables in the model. Furthermore, generalizability of these models is poor due to overfitting and a lack of standardized protocols for the construction and validation of the models. Until these problems have been resolved, a large gap remains between the creation of an AI algorithm and its implementation in clinical practice.
Collapse
Affiliation(s)
- M Schinkel
- Section Acute Medicine, Department of Internal Medicine, Amsterdam UMC, VU University Medical Center, De Boelelaan 1118, 1081HZ, Amsterdam, the Netherlands
| | - K Paranjape
- Section Acute Medicine, Department of Internal Medicine, Amsterdam UMC, VU University Medical Center, De Boelelaan 1118, 1081HZ, Amsterdam, the Netherlands
| | - R S Nannan Panday
- Section Acute Medicine, Department of Internal Medicine, Amsterdam UMC, VU University Medical Center, De Boelelaan 1118, 1081HZ, Amsterdam, the Netherlands
| | - N Skyttberg
- Department of Learning, Informatics, Management and Ethics, Health Informatics Centre, Karolinska Institutet, 171 77, Stockholm, Sweden
| | - P W B Nanayakkara
- Section Acute Medicine, Department of Internal Medicine, Amsterdam UMC, VU University Medical Center, De Boelelaan 1118, 1081HZ, Amsterdam, the Netherlands.
| |
Collapse
|