A Web-Based Calculator to Predict Early Death Among Patients With Bone Metastasis Using Machine Learning Techniques: Development and Validation Study

Development and validation of an explainable machine learning model for predicting multidimensional frailty in hospitalized patients with cirrhosis

We sought to develop and validate a machine learning (ML) model for predicting multidimensional frailty based on clinical and laboratory data. Moreover, an explainable ML model utilizing SHapley Additive exPlanations (SHAP) was constructed. This study enrolled 622 patients hospitalized due to decompensating episodes at a tertiary hospital. The cohort data were randomly divided into training and test sets. External validation was carried out using 131 patients from other tertiary hospitals. The frail phenotype was defined according to a self-reported questionnaire (Frailty Index). The area under the receiver operating characteristics curve was adopted to compare the performance of five ML models. The importance of the features and interpretation of the ML models were determined using the SHAP method. The proportions of cirrhotic patients with nonfrail and frail phenotypes in combined training and test sets were 87.8% and 12.2%, respectively, while they were 88.5% and 11.5% in the external validation dataset. Five ML algorithms were used, and the random forest (RF) model exhibited substantially predictive performance. Regarding the external validation, the RF algorithm outperformed other ML models. Moreover, the SHAP method demonstrated that neutrophil-to-lymphocyte ratio, age, lymphocyte-to-monocyte ratio, ascites, and albumin served as the most important predictors for frailty. At the patient level, the SHAP force plot and decision plot exhibited a clinically meaningful explanation of the RF algorithm. We constructed an ML model (RF) providing accurate prediction of frail phenotype in decompensated cirrhosis. The explainability and generalizability may foster clinicians to understand contributors to this physiologically vulnerable situation and tailor interventions.

Predicting all-cause mortality and premature death using interpretable machine learning among a middle-aged and elderly Chinese population

Objective

To develop machine learning-based prediction models for all-cause and premature mortality among the middle-aged and elderly population in China.

Method

Adults aged 45 years or older at baseline of 2011 from the China Health and Retirement Longitudinal Study (CHARLS) were included. The stacked ensemble model was built utilizing five selected machine learning algorithms. These models underwent training and testing using the CHARLS 2011-2015 cohort (derivation cohort) and subsequently underwent external validation using the CHARLS 2015-2018 cohort (validation cohort). SHapley Additive exPlanations (SHAP) was introduced to quantify the importance of risk factors and explain machine learning algorithms.

Result

In derivation cohort, a total of 10,677 subjects were included, 478 died during the follow-up. The stacked ensemble model demonstrated the highest efficacy in terms of its discrimination capability for predicting all-cause mortality and premature death, with an AUC[95 % CI] of 0.826[0.792-0.859] and 0.773[0.725-0.821], respectively. In validation cohort, the corresponding AUC[95 % CI] were 0.803[0.743-0.864] and 0.791[0.719-0.863], respectively. Risk factors including age, sex, self-reported health, activities of daily living, cognitive function, ever smoker, levels of systolic blood pressure, Cystatin C and low density lipoprotein were strong predictors for both all-cause mortality and premature death.

Conclusion

Stacked ensemble models performed well in predicting all-cause and premature death in this Chinese cohort. Interpretable techniques can aid in identifying significant risk factors and non-linear relationships between predictors and mortality.

An artificial intelligence tool to assess the risk of severe mental distress among college students in terms of demographics, eating habits, lifestyles, and sport habits: an externally validated study using machine learning

BACKGROUND

Precisely estimating the probability of mental health challenges among college students is pivotal for facilitating timely intervention and preventative measures. However, to date, no specific artificial intelligence (AI) models have been reported to effectively forecast severe mental distress. This study aimed to develop and validate an advanced AI tool for predicting the likelihood of severe mental distress in college students.

METHODS

A total of 2088 college students from five universities were enrolled in this study. Participants were randomly divided into a training group (80%) and a validation group (20%). Various machine learning models, including logistic regression (LR), extreme gradient boosting machine (eXGBM), decision tree (DT), k-nearest neighbor (KNN), random forest (RF), and support vector machine (SVM), were employed and trained in this study. Model performance was evaluated using 11 metrics, and the highest scoring model was selected. In addition, external validation was conducted on 751 participants from three universities. The AI tool was then deployed as a web-based AI application.

RESULTS

Among the models developed, the eXGBM model achieved the highest area under the curve (AUC) value of 0.932 (95% CI: 0.911-0.949), closely followed by RF with an AUC of 0.927 (95% CI: 0.905-0.943). The eXGBM model demonstrated superior performance in accuracy (0.850), precision (0.824), recall (0.890), specificity (0.810), F1 score (0.856), Brier score (0.103), log loss (0.326), and discrimination slope (0.598). The eXGBM model also received the highest score of 60 based on the evaluation scoring system, while RF achieved a score of 49. The scores of LR, DT, and SVM were only 19, 32, and 36, respectively. External validation yielded an impressive AUC value of 0.918.

CONCLUSIONS

The AI tool demonstrates promising predictive performance for identifying college students at risk of severe mental distress. It has the potential to guide intervention strategies and support early identification and preventive measures.

Interpretability-based machine learning for predicting the risk of death from pulmonary inflammation in Chinese intensive care unit patients

Objective

The objective of this research was to create a machine learning predictive model that could be easily interpreted in order to precisely determine the risk of premature death in patients receiving intensive care after pulmonary inflammation.

Methods

In this study, information from the China intensive care units (ICU) Open Source database was used to examine data from 2790 patients who had infections between January 2019 and December 2020. A 7:3 ratio was used to randomly assign the whole patient population to training and validation groups. This study used six machine learning techniques: logistic regression, random forest, gradient boosting tree, extreme gradient boosting tree (XGBoost), multilayer perceptron, and K-nearest neighbor. A cross-validation grid search method was used to search the parameters in each model. Eight metrics were used to assess the models' performance: accuracy, precision, recall, F1 score, area under the curve (AUC) value, Brier score, Jordon's index, and calibration slope. The machine methods were ranked based on how well they performed in each of these metrics. The best-performing models were selected for interpretation using both the Shapley Additive exPlanations (SHAP) and Local interpretable model-agnostic explanations (LIME) interpretable techniques.

Results

A subset of the study cohort's patients (120/1668, or 7.19%) died in the hospital following screening for inclusion and exclusion criteria. Using a cross-validated grid search to evaluate the six machine learning techniques, XGBoost showed good discriminative ability, achieving an accuracy score of 0.889 (0.874-0.904), precision score of 0.871 (0.849-0.893), recall score of 0.913 (0.890-0.936), F1 score of 0.891 (0.876-0.906), and AUC of 0.956 (0.939-0.973). Additionally, XGBoost exhibited excellent performance with a Brier score of 0.050, Jordon index of 0.947, and calibration slope of 1.074. It was also possible to create an interactive internet page using the XGBoost model.

Conclusion

By identifying patients at higher risk of early mortality, machine learning-based mortality risk prediction models have the potential to significantly improve patient care by directing clinical decision making and enabling early detection of survival and mortality issues in patients with pulmonary inflammation disease.

Development and validation of an artificial intelligence mobile application for predicting 30-day mortality in critically ill patients with orthopaedic trauma

BACKGROUND

Given the intricate and grave nature of trauma-related injuries in ICU settings, it is imperative to develop and deploy reliable predictive tools that can aid in the early identification of high-risk patients who are at risk of early death. The objective of this study is to create and validate an artificial intelligence (AI) model that can accurately predict early mortality among critical fracture patients.

METHODS

A total of 2662 critically ill patients with orthopaedic trauma were included from the MIMIC III database. Early mortality was defined as death within 30 days in this study. The patients were randomly divided into a model training cohort and a model validation cohort. Various algorithms, including logistic regression (LR), extreme gradient boosting machine (eXGBM), decision tree (DT), support vector machine (SVM), random forest (RF), and neural network (NN), were employed. Evaluation metrics, including discrimination and calibration, were used to develop a comprehensive scoring system ranging from 0 to 60, with higher scores indicating better prediction performance. Furthermore, external validation was carried out using 131 patients. The optimal model was deployed as an internet-based AI tool.

RESULTS

Among all models, the eXGBM demonstrated the highest area under the curve (AUC) value (0.974, 95%CI: 0.959-0.983), followed by the RF model (0.951, 95%CI: 0.935-0.967) and the NN model (0.922, 95%CI: 0.905-0.941). Additionally, the eXGBM model outperformed other models in terms of accuracy (0.915), precision (0.906), recall (0.926), F1 score (0.916), Brier score (0.062), log loss (0.210), and discrimination slope (0.767). Based on the scoring system, the eXGBM model achieved the highest score (53), followed by RF (42) and NN (39). The LR, DT, and SVM models obtained scores of 28, 18, and 32, respectively. Decision curve analysis further confirmed the superior clinical net benefits of the eXGBM model. External validation of the model achieved an AUC value of 0.913 (95%CI: 0.878-0.948). Consequently, the model was deployed on the Internet at https://30-daymortalityincriticallyillpatients-fnfsynbpbp6rgineaspuim.streamlit.app/, allowing users to input patient features and obtain predicted risks of early mortality among critical fracture patients. Furthermore, the AI model successfully stratified patients into low or high risk of early mortality based on a predefined threshold and provided recommendations for appropriate therapeutic interventions.

CONCLUSION

This study successfully develops and validates an AI model, with the eXGBM algorithm demonstrating the highest predictive performance for early mortality in critical fracture patients. By deploying the model as a web-based AI application, healthcare professionals can easily access the tool, enabling them to predict 30-day mortality and aiding in the identification and management of high-risk patients among those critically ill with orthopedic trauma.

An Artificial Intelligence Platform to Stratify the Risk of Experiencing Sleep Disturbance in University Students After Analyzing Psychological Health, Lifestyle, and Sports: A Multicenter Externally Validated Study

Background

Sleep problems are prevalent among university students, yet there is a lack of effective models to assess the risk of sleep disturbance. Artificial intelligence (AI) provides an opportunity to develop a platform for evaluating the risk. This study aims to develop and validate an AI platform to stratify the risk of experiencing sleep disturbance for university students.

Methods

A total of 2243 university students were included, with 1882 students from five universities comprising the model derivation group and 361 students from two additional universities forming the external validation group. Six machine learning techniques, including extreme gradient boosting machine (eXGBM), decision tree (DT), k-nearest neighbor (KNN), random forest (RF), neural network (NN), and support vector machine (SVM), were employed to train models using the same set of features. The models' prediction performance was assessed based on discrimination and calibration, and feature importance was determined using Shapley Additive exPlanations (SHAP) analysis.

Results

The prevalence of sleep disturbance was 44.69% in the model derivation group and 49.58% in the external validation group. Among the developed models, eXGBM exhibited superior performance, surpassing other models in metrics such as area under the curve (0.779, 95% CI: 0.728-0.830), accuracy (0.710), precision (0.737), F1 score (0.692), Brier score (0.193), and log loss (0.569). Calibration and decision curve analyses demonstrated favorable calibration ability and clinical net benefits, respectively. SHAP analysis identified five key features: stress score, severity of depression, vegetable consumption, age, and sedentary time. The AI platform was made available online at https://sleepdisturbancestudents-xakgzwectsw85cagdgkax9.streamlit.app/, enabling users to calculate individualized risk of sleep disturbance.

Conclusion

Sleep disturbance is prevalent among university students. This study presents an AI model capable of identifying students at high risk for sleep disturbance. The AI platform offers a valuable resource to guide interventions and improve sleep outcomes for university students.