Machine learning-based prediction of 1-year mortality for acute coronary syndrome✰

Model for Predicting the Effect of Sibutramine Therapy in Obesity

Background: The development of models predicting response to weight loss therapy using sibutramine is found in only a few cases. The objective of the work is to develop a data-driven method of personalized recommendation for obesity treatment that would predict the response to sibutramine based on the current set of patient parameters. Methods: The decision system is built on the XGBoost classification algorithm along with recursive feature selection and Shapley data valuation. Using the results of clinical trials, it was trained to estimate the probability of overcoming a weight loss threshold. The model was evaluated by the accuracy metric using the Leave-One-Out cross-validation. Results: The model for predicting response to sibutramine treatment over 3 months has an accuracy of 71%. The model for predicting outcomes at the sixth month visit based on results at 3 months has an accuracy of 80%. Conclusions: Although our developed prediction model may not exhibit high precision compared to certain benchmarks, it significantly outperforms random chance or models relying only on BMI parameters. Our model used the available range of laboratory tests, which makes it possible to use this model for routine clinical use and help doctors decide whether to prescribe sibutramine.

Optimizing cardiovascular disease mortality prediction: a super learner approach in the tehran lipid and glucose study

BACKGROUND & AIM

Cardiovascular disease (CVD) is the most important cause of death in the world and has a potential impact on health care costs, this study aimed to evaluate the performance of machine learning survival models and determine the optimum model for predicting CVD-related mortality.

METHOD

In this study, the research population was all participants in Tehran Lipid and Glucose Study (TLGS) aged over 30 years. We used the Gradient Boosting model (GBM), Support Vector Machine (SVM), Super Learner (SL), and Cox proportional hazard (Cox-PH) models to predict the CVD-related mortality using 26 features. The dataset was randomly divided into training (80%) and testing (20%). To evaluate the performance of the methods, we used the Brier Score (BS), Prediction Error (PE), Concordance Index (C-index), and time-dependent Area Under the Curve (TD-AUC) criteria. Four different clinical models were also performed to improve the performance of the methods.

RESULTS

Out of 9258 participants with a mean age of (SD; range) 43.74 (15.51; 20-91), 56.60% were female. The CVD death proportion was 2.5% (228 participants). The death proportion was significantly higher in men (67.98% M, 32.02% F). Based on predefined selection criteria, the SL method has the best performance in predicting CVD-related mortality (TD-AUC > 93.50%). Among the machine learning (ML) methods, The SVM has the worst performance (TD-AUC = 90.13%). According to the relative effect, age, fasting blood sugar, systolic blood pressure, smoking, taking aspirin, diastolic blood pressure, Type 2 diabetes mellitus, hip circumference, body mss index (BMI), and triglyceride were identified as the most influential variables in predicting CVD-related mortality.

CONCLUSION

According to the results of our study, compared to the Cox-PH model, Machine Learning models showed promising and sometimes better performance in predicting CVD-related mortality. This finding is based on the analysis of a large and diverse urban population from Tehran, Iran.

Identifying Frailty in Older Adults Receiving Home Care Assessment Using Machine Learning: Longitudinal Observational Study on the Role of Classifier, Feature Selection, and Sample Size

BACKGROUND

Machine learning techniques are starting to be used in various health care data sets to identify frail persons who may benefit from interventions. However, evidence about the performance of machine learning techniques compared to conventional regression is mixed. It is also unclear what methodological and database factors are associated with performance.

OBJECTIVE

This study aimed to compare the mortality prediction accuracy of various machine learning classifiers for identifying frail older adults in different scenarios.

METHODS

We used deidentified data collected from older adults (65 years of age and older) assessed with interRAI-Home Care instrument in New Zealand between January 1, 2012, and December 31, 2016. A total of 138 interRAI assessment items were used to predict 6-month and 12-month mortality, using 3 machine learning classifiers (random forest [RF], extreme gradient boosting [XGBoost], and multilayer perceptron [MLP]) and regularized logistic regression. We conducted a simulation study comparing the performance of machine learning models with logistic regression and interRAI Home Care Frailty Scale and examined the effects of sample sizes, the number of features, and train-test split ratios.

RESULTS

A total of 95,042 older adults (median age 82.66 years, IQR 77.92-88.76; n=37,462, 39.42% male) receiving home care were analyzed. The average area under the curve (AUC) and sensitivities of 6-month mortality prediction showed that machine learning classifiers did not outperform regularized logistic regressions. In terms of AUC, regularized logistic regression had better performance than XGBoost, MLP, and RF when the number of features was ≤80 and the sample size ≤16,000; MLP outperformed regularized logistic regression in terms of sensitivities when the number of features was ≥40 and the sample size ≥4000. Conversely, RF and XGBoost demonstrated higher specificities than regularized logistic regression in all scenarios.

CONCLUSIONS

The study revealed that machine learning models exhibited significant variation in prediction performance when evaluated using different metrics. Regularized logistic regression was an effective model for identifying frail older adults receiving home care, as indicated by the AUC, particularly when the number of features and sample sizes were not excessively large. Conversely, MLP displayed superior sensitivity, while RF exhibited superior specificity when the number of features and sample sizes were large.

A comparison of random survival forest and Cox regression for prediction of mortality in patients with hemorrhagic stroke

OBJECTIVE

To evaluate RSF and Cox models for mortality prediction of hemorrhagic stroke (HS) patients in intensive care unit (ICU).

METHODS

In the training set, the optimal models were selected using five-fold cross-validation and grid search method. In the test set, the bootstrap method was used to validate. The area under the curve(AUC) was used for discrimination, Brier Score (BS) was used for calibration, positive predictive value(PPV), negative predictive value(NPV), and F1 score were combined to compare.

RESULTS

A total of 2,990 HS patients were included. For predicting the 7-day mortality, the mean AUCs for RSF and Cox regression were 0.875 and 0.761, while the mean BS were 0.083 and 0.108. For predicting the 28-day mortality, the mean AUCs for RSF and Cox regression were 0.794 and 0.649, while the mean BS were 0.129 and 0.174. The mean AUCs of RSF and Cox versus conventional scores for predicting patients' 7-day mortality were 0.875 (RSF), 0.761 (COX), 0.736 (SAPS II), 0.723 (OASIS), 0.632 (SIRS), and 0.596 (SOFA), respectively.

CONCLUSIONS

RSF provided a better clinical reference than Cox. Creatine, temperature, anion gap and sodium were important variables in both models.

Deep learning model for predicting the survival of patients with primary gastrointestinal lymphoma based on the SEER database and a multicentre external validation cohort

PURPOSE

Due to the rarity of primary gastrointestinal lymphoma (PGIL), the prognostic factors and optimal management of PGIL have not been clearly defined. We aimed to establish prognostic models using a deep learning algorithm for survival prediction.

METHODS

We collected 11,168 PGIL patients from the Surveillance, Epidemiology, and End Results (SEER) database to form the training and test cohorts. At the same time, we collected 82 PGIL patients from three medical centres to form the external validation cohort. We constructed a Cox proportional hazards (CoxPH) model, random survival forest (RSF) model, and neural multitask logistic regression (DeepSurv) model to predict PGIL patients' overall survival (OS).

RESULTS

The 1-, 3-, 5-, and 10-year OS rates of PGIL patients in the SEER database were 77.1%, 69.4%, 63.7%, and 50.3%, respectively. The RSF model based on all variables showed that the top three most important variables for predicting OS were age, histological type, and chemotherapy. The independent risk factors for PGIL patient prognosis included sex, age, race, primary site, Ann Arbor stage, histological type, symptom, radiotherapy, and chemotherapy, according to the Lasso regression analysis. Using these factors, we built the CoxPH and DeepSurv models. The DeepSurv model's C-index values were 0.760 in the training cohort, 0.742 in the test cohort, and 0.707 in the external validation cohort, which demonstrated that the DeepSurv model performed better compared to the RSF model (0.728) and the CoxPH model (0.724). The DeepSurv model accurately predicted 1-, 3-, 5- and 10-year OS. Both calibration curves and decision curve analysis curves demonstrated the superior performance of the DeepSurv model. We developed the DeepSurv model as an online web calculator for survival prediction, which can be accessed at http://124.222.228.112:8501/ .

CONCLUSIONS

This DeepSurv model with external validation is superior to previous studies in predicting short-term and long-term survival and can help us make better-individualized decisions for PGIL patients.

Prediction of posttraumatic functional recovery in middle-aged and older patients through dynamic ensemble selection modeling

Introduction

Age-specific risk factors may delay posttraumatic functional recovery; complex interactions exist between these factors. In this study, we investigated the prediction ability of machine learning models for posttraumatic (6 months) functional recovery in middle-aged and older patients on the basis of their preexisting health conditions.

Methods

Data obtained from injured patients aged ≥45 years were divided into training-validation (n = 368) and test (n = 159) data sets. The input features were the sociodemographic characteristics and baseline health conditions of the patients. The output feature was functional status 6 months after injury; this was assessed using the Barthel Index (BI). On the basis of their BI scores, the patients were categorized into functionally independent (BI >60) and functionally dependent (BI ≤60) groups. The permutation feature importance method was used for feature selection. Six algorithms were validated through cross-validation with hyperparameter optimization. The algorithms exhibiting satisfactory performance were subjected to bagging to construct stacking, voting, and dynamic ensemble selection models. The best model was evaluated on the test data set. Partial dependence (PD) and individual conditional expectation (ICE) plots were created.

Results

In total, nineteen of twenty-seven features were selected. Logistic regression, linear discrimination analysis, and Gaussian Naive Bayes algorithms exhibited satisfactory performances and were, therefore, used to construct ensemble models. The k-Nearest Oracle Elimination model outperformed the other models when evaluated on the training-validation data set (sensitivity: 0.732, 95% CI: 0.702-0.761; specificity: 0.813, 95% CI: 0.805-0.822); it exhibited compatible performance on the test data set (sensitivity: 0.779, 95% CI: 0.559-0.950; specificity: 0.859, 95% CI: 0.799-0.912). The PD and ICE plots showed consistent patterns with practical tendencies.

Conclusion

Preexisting health conditions can predict long-term functional outcomes in injured middle-aged and older patients, thus predicting prognosis and facilitating clinical decision-making.

Predictors of Carbohydrate Metabolism Disorders and Lethal Outcome in Patients after Myocardial Infarction: A Place of Glucose Level

BACKGROUND AND AIM

The aim of this study was to reveal statistical patterns in patients with acute myocardial infarction (AMI) that cause the development of carbohydrate metabolism disorders (CMD) (type 2 diabetes mellitus and prediabetes) and death within 5 years after AMI.

METHODS

1079 patients who were treated with AMI in the Almazov National Medical Research Center were retrospectively selected for the study. For each patient, all data from electronic medical records were downloaded. Statistical patterns that determine the development of CMDs and death within 5 years after AMI were identified. To create and train the models used in this study, the classic methods of Data Mining, Data Exploratory Analysis, and Machine Learning were used.

RESULTS

The main predictors of mortality within 5 years after AMI were advanced age, low relative level of lymphocytes, circumflex artery lesion, and glucose level. Main predictors of CMDs were low basophils, high neutrophils, high platelet distribution width, and high blood glucose level. High values of age and glucose together were relatively independent predictors. With glucose level >11 mmol/L and age >70 years, the 5-year risk of death is about 40% and it rises with increasing glucose levels.

CONCLUSION

The obtained results make it possible to predict the development of CMDs and death based on simple parameters that are easily available in clinical practice. Glucose level measured on the 1st day of AMI was among the most important predictors of CMDs and death.

Deep learning models for predicting the survival of patients with chondrosarcoma based on a surveillance, epidemiology, and end results analysis

Background

Accurate prediction of prognosis is critical for therapeutic decisions in chondrosarcoma patients. Several prognostic models have been created utilizing multivariate Cox regression or binary classification-based machine learning approaches to predict the 3- and 5-year survival of patients with chondrosarcoma, but few studies have investigated the results of combining deep learning with time-to-event prediction. Compared with simplifying the prediction as a binary classification problem, modeling the probability of an event as a function of time by combining it with deep learning can provide better accuracy and flexibility.

Materials and methods

Patients with the diagnosis of chondrosarcoma between 2000 and 2018 were extracted from the Surveillance, Epidemiology, and End Results (SEER) registry. Three algorithms—two based on neural networks (DeepSurv, neural multi-task logistic regression [NMTLR]) and one on ensemble learning (random survival forest [RSF])—were selected for training. Meanwhile, a multivariate Cox proportional hazards (CoxPH) model was also constructed for comparison. The dataset was randomly divided into training and testing datasets at a ratio of 7:3. Hyperparameter tuning was conducted through a 1000-repeated random search with 5-fold cross-validation on the training dataset. The model performance was assessed using the concordance index (C-index), Brier score, and Integrated Brier Score (IBS). The accuracy of predicting 1-, 3-, 5- and 10-year survival was evaluated using receiver operating characteristic curves (ROC), calibration curves, and the area under the ROC curves (AUC).

Results

A total of 3145 patients were finally enrolled in our study. The mean age at diagnosis was 52 ± 18 years, 1662 of the 3145 patients were male (53%), and mean survival time was 83 ± 67 months. Two deep learning models outperformed the RSF and classical CoxPH models, with the C-index on test datasets achieving values of 0.832 (DeepSurv) and 0.821 (NMTLR). The DeepSurv model produced better accuracy and calibrated survival estimates in predicting 1-, 3- 5- and 10-year survival (AUC:0.895-0.937). We deployed the DeepSurv model as a web application for use in clinical practice; it can be accessed through https://share.streamlit.io/whuh-ml/chondrosarcoma/Predict/app.py.

Conclusions

Time-to-event prediction models based on deep learning algorithms are successful in predicting chondrosarcoma prognosis, with DeepSurv producing the best discriminative performance and calibration.

An energy-efficient in-memory computing architecture for survival data analysis based on resistive switching memories

One of the objectives fostered in medical science is the so-called precision medicine, which requires the analysis of a large amount of survival data from patients to deeply understand treatment options. Tools like machine learning (ML) and deep neural networks are becoming a de-facto standard. Nowadays, computing facilities based on the Von Neumann architecture are devoted to these tasks, yet rapidly hitting a bottleneck in performance and energy efficiency. The in-memory computing (IMC) architecture emerged as a revolutionary approach to overcome that issue. In this work, we propose an IMC architecture based on resistive switching memory (RRAM) crossbar arrays to provide a convenient primitive for matrix-vector multiplication in a single computational step. This opens massive performance improvement in the acceleration of a neural network that is frequently used in survival analysis of biomedical records, namely the DeepSurv. We explored how the synaptic weights mapping strategy and the programming algorithms developed to counter RRAM non-idealities expose a performance/energy trade-off. Finally, we discussed how this application is tailored for the IMC architecture rather than being executed on commodity systems.

Avoiding C-hacking when evaluating survival distribution predictions with discrimination measures

MOTIVATION

In this article, we consider how to evaluate survival distribution predictions with measures of discrimination. This is non-trivial as discrimination measures are the most commonly used in survival analysis and yet there is no clear method to derive a risk prediction from a distribution prediction. We survey methods proposed in literature and software and consider their respective advantages and disadvantages.

RESULTS

Whilst distributions are frequently evaluated by discrimination measures, we find that the method for doing so is rarely described in the literature and often leads to unfair comparisons or 'C-hacking'. We demonstrate by example how simple it can be to manipulate results and use this to argue for better reporting guidelines and transparency in the literature. We recommend that machine learning survival analysis software implements clear transformations between distribution and risk predictions in order to allow more transparent and accessible model evaluation.

AVAILABILITY AND IMPLEMENTATION

The code used in the final experiment is available at https://github.com/RaphaelS1/distribution_discrimination.