Machine learning-based prediction of in-hospital mortality using admission laboratory data: A retrospective, single-site study using electronic health record data

Pre-Interventional Risk Assessment in The Elderly (PIRATE): Development of a scoring system to predict 30-day mortality using data of the Peri-Interventional Outcome Study in the Elderly

Risk assessment before interventions in elderly patients becomes more and more vital due to an increasing number of elderly patients requiring surgery. Existing risk scores are often not tailored to marginalized groups such as patients aged 80 years or older. We aimed to develop an easy-to-use and readily applicable risk assessment tool that implements pre-interventional predictors of 30-day mortality in elderly patients (≥80 years) undergoing interventions under anesthesia. Using Cox regression analysis, we compared different sets of predictors by taking into account their ease of availability and by evaluating predictive accuracy. Coefficient estimates were utilized to set up a scoring system that was internally validated. Model building and evaluation were based on data from the Peri-Interventional Outcome Study in the Elderly (POSE), which was conducted as a European multicenter, observational prospective cohort study. Our risk assessment tool, named PIRATE, contains three predictors assessable at admission (urgency, severity and living conditions). Discriminatory power, as measured by the concordance index, was 0.75. The estimated prediction error, as measured by the Brier score, was 0.036 (covariate-free reference model: 0.043). PIRATE is an easy-to-use risk assessment tool that helps stratifying elderly patients undergoing interventions with anesthesia at increased risk of mortality. PIRATE is readily available and applies to a wide variety of settings. In particular, it covers patients needing elective or emergency surgery and undergoing in-hospital or day-case surgery. Also, it applies to all types of interventions, from minor to major. It may serve as a basis for multidisciplinary and informed shared decision-making.

Early prediction of in-hospital mortality utilizing multivariate predictive modelling of electronic medical records and socio-determinants of health of the first day of hospitalization

BACKGROUND

In France an average of 4% of hospitalized patients die during their hospital stay. To aid medical decision making and the attribution of resources, within a few days of admission the identification of patients at high risk of dying in hospital is essential.

METHODS

We used de-identified routine patient data available in the first 2 days of hospitalization in a French University Hospital (between 2016 and 2018) to build models predicting in-hospital mortality (at ≥ 2 and ≤ 30 days after admission). We tested nine different machine learning algorithms with repeated 10-fold cross-validation. Models were trained with 283 variables including age, sex, socio-determinants of health, laboratory test results, procedures (Classification of Medical Acts), medications (Anatomical Therapeutic Chemical code), hospital department/unit and home address (urban, rural etc.). The models were evaluated using various performance metrics. The dataset contained 123,729 admissions, of which the outcome for 3542 was all-cause in-hospital mortality and 120,187 admissions (no death reported within 30 days) were controls.

RESULTS

The support vector machine, logistic regression and Xgboost algorithms demonstrated high discrimination with a balanced accuracy of 0.81 (95%CI 0.80-0.82), 0.82 (95%CI 0.80-0.83) and 0.83 (95%CI 0.80-0.83) and AUC of 0.90 (95%CI 0.88-0.91), 0.90 (95%CI 0.89-0.91) and 0.90 (95%CI 0.89-0.91) respectively. The most predictive variables for in-hospital mortality in all three models were older age (greater risk), and admission with a confirmed appointment (reduced risk).

CONCLUSION

We propose three highly discriminating machine-learning models that could improve clinical and organizational decision making for adult patients at hospital admission.

Short-term All-cause In-hospital Mortality Prediction by Machine Learning Using Numeric Laboratory Results

Introduction

A critical value (or panic value) is a laboratory test result that significantly deviates from the normal value and represents a potentially life-threatening condition requiring immediate action. Although notification of critical values by critical value list (CVL) is a well-established method, their contribution to mortality prediction is unclear.

Methods

A total of 335,430 clinical laboratory results from 92,673 patients from July 2018 to December 2019 were used. Data in the first 12 months were divided into two datasets at a ratio of 70:30, and a 7-day mortality prediction model by machine learning (eXtreme Gradient Boosting [XGB] decision tree) was created using stratified random undersampling data of the 70% dataset. Mortality predictions by the CVL and XGB model were validated using the remaining 30% of the data, as well as different 6-month datasets from July to December 2019.

Results

The true results which were the sum of correct predictions by the XGB model and CVL using the remaining 30% data were 61,535 and 61,024 tests, and the false results which were the sum of incorrect predictions were 5,492 and 6,003, respectively. Furthermore, the true results with the different datasets were 105,956 and 102,061 tests, and the false results were 6,052 and 9,947, respectively. The XGB model was significantly better than CVL (p < 0.001) in both datasets.The receiver operating characteristic-area under the curve values for the 30% and validation data by XGB were 0.9807 and 0.9646, respectively, which were significantly higher than those by CVL (0.7549 and 0.7172, respectively).

Conclusions

Mortality prediction within 7 days by machine learning using numeric laboratory results was significantly better than that by conventional CVL. The results indicate that machine learning enables timely notification to healthcare providers and may be safer than prediction by conventional CVL.

Machine learning-based prediction of postoperative mortality in emergency colorectal surgery: A retrospective, multicenter cohort study using Tokushukai medical database

Background

Although prognostic factors associated with mortality in patients with emergency colorectal surgery have been identified, an accurate mortality risk assessment is still necessary to determine the range of therapeutic resources in accordance with the severity of patients. We established machine-learning models to predict in-hospital mortality for patients who had emergency colorectal surgery using clinical data at admission and attempted to identify prognostic factors associated with in-hospital mortality.

Methods

This retrospective cohort study included adult patients undergoing emergency colorectal surgery in 42 hospitals between 2012 and 2020. We employed logistic regression and three supervised machine-learning models: random forests, gradient-boosting decision trees (GBDT), and multilayer perceptron (MLP). The area under the receiver operating characteristics curve (AUROC) was calculated for each model. The Shapley additive explanations (SHAP) values are also calculated to identify the significant variables in GBDT.

Results

There were 8792 patients who underwent emergency colorectal surgery. As a result, the AUROC values of 0.742, 0.782, 0.814, and 0.768 were obtained for logistic regression, random forests, GBDT, and MLP. According to SHAP values, age, colorectal cancer, use of laparoscopy, and some laboratory variables, including serum lactate dehydrogenase serum albumin, and blood urea nitrogen, were significantly associated with in-hospital mortality.

Conclusion

We successfully generated a machine-learning prediction model, including GBDT, with the best prediction performance and exploited the potential for use in evaluating in-hospital mortality risk for patients who undergo emergency colorectal surgery.

Cohort profile for development of machine learning models to predict healthcare-related adverse events (Demeter): clinical objectives, data requirements for modelling and overview of data set for 2016-2018

PURPOSE

In-hospital health-related adverse events (HAEs) are a major concern for hospitals worldwide. In high-income countries, approximately 1 in 10 patients experience HAEs associated with their hospital stay. Estimating the risk of an HAE at the individual patient level as accurately as possible is one of the first steps towards improving patient outcomes. Risk assessment can enable healthcare providers to target resources to patients in greatest need through adaptations in processes and procedures. Electronic health data facilitates the application of machine-learning methods for risk analysis. We aim, first to reveal correlations between HAE occurrence and patients' characteristics and/or the procedures they undergo during their hospitalisation, and second, to build models that allow the early identification of patients at an elevated risk of HAE.

PARTICIPANTS

143 865 adult patients hospitalised at Grenoble Alpes University Hospital (France) between 1 January 2016 and 31 December 2018.

FINDINGS TO DATE

In this set-up phase of the project, we describe the preconditions for big data analysis using machine-learning methods. We present an overview of the retrospective de-identified multisource data for a 2-year period extracted from the hospital's Clinical Data Warehouse, along with social determinants of health data from the National Institute of Statistics and Economic Studies, to be used in machine learning (artificial intelligence) training and validation. No supplementary information or evaluation on the part of medical staff will be required by the information system for risk assessment.

FUTURE PLANS

We are using this data set to develop predictive models for several general HAEs including secondary intensive care admission, prolonged hospital stay, 7-day and 30-day re-hospitalisation, nosocomial bacterial infection, hospital-acquired venous thromboembolism, and in-hospital mortality.

Dealing With Missing, Imbalanced, and Sparse Features During the Development of a Prediction Model for Sudden Death Using Emergency Medicine Data: Machine Learning Approach

BACKGROUND

In emergency departments (EDs), early diagnosis and timely rescue, which are supported by prediction modes using ED data, can increase patients' chances of survival. Unfortunately, ED data usually contain missing, imbalanced, and sparse features, which makes it challenging to build early identification models for diseases.

OBJECTIVE

This study aims to propose a systematic approach to deal with the problems of missing, imbalanced, and sparse features for developing sudden-death prediction models using emergency medicine (or ED) data.

METHODS

We proposed a 3-step approach to deal with data quality issues: a random forest (RF) for missing values, k-means for imbalanced data, and principal component analysis (PCA) for sparse features. For continuous and discrete variables, the decision coefficient R² and the κ coefficient were used to evaluate performance, respectively. The area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) were used to estimate the model's performance. To further evaluate the proposed approach, we carried out a case study using an ED data set obtained from the Hainan Hospital of Chinese PLA General Hospital. A logistic regression (LR) prediction model for patient condition worsening was built.

RESULTS

A total of 1085 patients with rescue records and 17,959 patients without rescue records were selected and significantly imbalanced. We extracted 275, 402, and 891 variables from laboratory tests, medications, and diagnosis, respectively. After data preprocessing, the median R² of the RF continuous variable interpolation was 0.623 (IQR 0.647), and the median of the κ coefficient for discrete variable interpolation was 0.444 (IQR 0.285). The LR model constructed using the initial diagnostic data showed poor performance and variable separation, which was reflected in the abnormally high odds ratio (OR) values of the 2 variables of cardiac arrest and respiratory arrest (201568034532 and 1211118945, respectively) and an abnormal 95% CI. Using processed data, the recall of the model reached 0.746, the F₁-score was 0.73, and the AUROC was 0.708.

CONCLUSIONS

The proposed systematic approach is valid for building a prediction model for emergency patients.

Preoperative prediction of the need for arterial and central venous catheterization using machine learning techniques

Some surgical patients require an arterial or central venous catheterization intraoperatively. This decision relied solely on the experience of individual anesthesiologists; however, these decisions are not easy for clinicians who are in an emergency or inexperienced. Therefore, applying recent artificial intelligence techniques to automatically extractable data from electronic medical record (EMR) could create a very clinically useful model in this situation. This study aimed to develop a model that is easy to apply in real clinical settings by implementing a prediction model for the preoperative decision to insert an arterial and central venous catheter and that can be automatically linked to the EMR. We collected and retrospectively analyzed data from 66,522 patients, > 18 years of age, who underwent non-cardiac surgeries from March 2019 to April 2021 at the single tertiary medical center. Data included demographics, pre-operative laboratory tests, surgical information, and catheterization information. When compared with other machine learning methods, the DNN model showed the best predictive performance in terms of the area under receiver operating characteristic curve and area under the precision-recall curve. Operation code information accounted for the largest portion of the prediction. This can be applied to clinical fields using operation code and minimal preoperative clinical information.

Multi-center validation of machine learning model for preoperative prediction of postoperative mortality

Accurate prediction of postoperative mortality is important for not only successful postoperative patient care but also for information-based shared decision-making with patients and efficient allocation of medical resources. This study aimed to create a machine-learning prediction model for 30-day mortality after a non-cardiac surgery that adapts to the manageable amount of clinical information as input features and is validated against multi-centered rather than single-centered data. Data were collected from 454,404 patients over 18 years of age who underwent non-cardiac surgeries from four independent institutions. We performed a retrospective analysis of the retrieved data. Only 12–18 clinical variables were used for model training. Logistic regression, random forest classifier, extreme gradient boosting (XGBoost), and deep neural network methods were applied to compare the prediction performances. To reduce overfitting and create a robust model, bootstrapping and grid search with tenfold cross-validation were performed. The XGBoost method in Seoul National University Hospital (SNUH) data delivers the best performance in terms of the area under receiver operating characteristic curve (AUROC) (0.9376) and the area under the precision-recall curve (0.1593). The predictive performance was the best when the SNUH model was validated with Ewha Womans University Medical Center data (AUROC, 0.941). Preoperative albumin, prothrombin time, and age were the most important features in the model for each hospital. It is possible to create a robust artificial intelligence prediction model applicable to multiple institutions through a light predictive model using only minimal preoperative information that can be automatically extracted from each hospital.