Evaluating machine learning models for sepsis prediction: A systematic review of methodologies.
iScience 2022;
25:103651. [PMID:
35028534 PMCID:
PMC8741489 DOI:
10.1016/j.isci.2021.103651]
[Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2021] [Revised: 11/16/2021] [Accepted: 12/15/2021] [Indexed: 12/29/2022] Open
Abstract
Studies for sepsis prediction using machine learning are developing rapidly in medical science recently. In this review, we propose a set of new evaluation criteria and reporting standards to assess 21 qualified machine learning models for quality analysis based on PRISMA. Our assessment shows that (1.) the definition of sepsis is not consistent among the studies; (2.) data sources and data preprocessing methods, machine learning models, feature engineering, and inclusion types vary widely among the studies; (3.) the closer to the onset of sepsis, the higher the value of AUROC is; (4.) the improvement in AUROC is primarily due to using machine learning as a feature engineering tool; (5.) deep neural networks coupled with Sepsis-3 diagnostic criteria tend to yield better results on the time series data collected from patients with sepsis. The new evaluation criteria and reporting standards will facilitate the development of improved machine learning models for clinical applications.
New evaluation/reporting standard for sepsis prediction machine learning models
Major limitations in the current models for sepsis prediction have been identified
We strongly suggest using machine learning as a feature engineering tool
Recommending multilayer neural networks and Sepsis 3.0 for yield better result
Collapse