Interpretable machine learning models for predicting venous thromboembolism in the intensive care unit: an analysis based on data from 207 centers

Venous thromboembolism in the era of machine learning and artificial intelligence in medicine

In this review, we embark on a comprehensive exploration of venous thromboembolism (VTE) in the context of medical history and its current practice within medicine. We delve into the landscape of artificial intelligence (AI), exploring its present utility and envisioning its transformative roles within VTE management, from prevention to screening and beyond. Central to our discourse is a forward-looking perspective on the integration of AI within VTE in medicine, advocating for rigorous study design, robust validation processes, and meticulous statistical analysis to gauge the efficacy of AI applications. We further illuminate the potential of large language models and generative AI in revolutionizing VTE care, while acknowledging their inherent limitations and proposing innovative solutions to overcome challenges related to data availability and integrity, including the strategic use of synthetic data. The critical importance of navigating ethical, legal, and privacy concerns associated with AI is underscored, alongside the imperative for comprehensive governance and policy frameworks to regulate its deployment in VTE treatment. We conclude on a note of cautious optimism, where we highlight the significance of proactively addressing the myriad challenges that accompany the advent of AI in healthcare. Through diligent design, stringent validation, extensive education, and prudent regulation, we can harness AI's potential to significantly enhance our understanding and management of VTE. As we stand on the cusp of a new era, our commitment to these principles will be instrumental in ensuring that the promise of AI is fully realized within the realm of VTE care.

Machine learning approaches to identify the link between heavy metal exposure and ischemic stroke using the US NHANES data from 2003 to 2018

Purpose

There is limited understanding of the link between exposure to heavy metals and ischemic stroke (IS). This research aimed to develop efficient and interpretable machine learning (ML) models to associate the relationship between exposure to heavy metals and IS.

Methods

The data of this research were obtained from the National Health and Nutrition Examination Survey (US NHANES, 2003-2018) database. Seven ML models were used to identify IS caused by exposure to heavy metals. To assess the strength of the models, we employed 10-fold cross-validation, the area under the curve (AUC), F1 scores, Brier scores, Matthews correlation coefficient (MCC), precision-recall (PR) curves, and decision curve analysis (DCA) curves. Following these tests, the best-performing model was selected. Finally, the DALEX package was used for feature explanation and decision-making visualization.

Results

A total of 15,575 participants were involved in this study. The best-performing ML models, which included logistic regression (LR) (AUC: 0.796) and XGBoost (AUC: 0.789), were selected. The DALEX package revealed that age, total mercury in blood, poverty-to-income ratio (PIR), and cadmium were the most significant contributors to IS in the logistic regression and XGBoost models.

Conclusion

The logistic regression and XGBoost models showed high efficiency, accuracy, and robustness in identifying associations between heavy metal exposure and IS in NHANES 2003-2018 participants.

Machine learning-based derivation and validation of three immune phenotypes for risk stratification and prognosis in community-acquired pneumonia: a retrospective cohort study

Background

The clinical presentation of Community-acquired pneumonia (CAP) in hospitalized patients exhibits heterogeneity. Inflammation and immune responses play significant roles in CAP development. However, research on immunophenotypes in CAP patients is limited, with few machine learning (ML) models analyzing immune indicators.

Methods

A retrospective cohort study was conducted at Xinhua Hospital, affiliated with Shanghai Jiaotong University. Patients meeting predefined criteria were included and unsupervised clustering was used to identify phenotypes. Patients with distinct phenotypes were also compared in different outcomes. By machine learning methods, we comprehensively assess the disease severity of CAP patients.

Results

A total of 1156 CAP patients were included in this research. In the training cohort (n=809), we identified three immune phenotypes among patients: Phenotype A (42.0%), Phenotype B (40.2%), and Phenotype C (17.8%), with Phenotype C corresponding to more severe disease. Similar results can be observed in the validation cohort. The optimal prognostic model, SuperPC, achieved the highest average C-index of 0.859. For predicting CAP severity, the random forest model was highly accurate, with C-index of 0.998 and 0.794 in training and validation cohorts, respectively.

Conclusion

CAP patients can be categorized into three distinct immune phenotypes, each with prognostic relevance. Machine learning exhibits potential in predicting mortality and disease severity in CAP patients by leveraging clinical immunological data. Further external validation studies are crucial to confirm applicability.

Blood urea nitrogen to creatinine ratio is associated with in-hospital mortality in critically ill patients with venous thromboembolism: a retrospective cohort study

Background

The relationship between the blood urea nitrogen to creatinine ratio (BCR) and the risk of in-hospital mortality among intensive care unit (ICU) patients diagnosed with venous thromboembolism (VTE) remains unclear. This study aimed to assess the relationship between BCR upon admission to the ICU and in-hospital mortality in critically ill patients with VTE.

Methods

This retrospective cohort study included patients diagnosed with VTE from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database. The primary endpoint was in-hospital mortality. Univariate and multivariate logistic regression analyses were conducted to evaluate the prognostic significance of the BCR. Receiver operating characteristic (ROC) curve analysis was utilized to determine the optimal cut-off value of BCR. Additionally, survival analysis using a Kaplan-Meier curve was performed.

Results

A total of 2,560 patients were included, with a median age of 64.5 years, and 55.5% were male. Overall, the in-hospital mortality rate was 14.6%. The optimal cut-off value of the BCR for predicting in-hospital mortality in critically ill VTE patients was 26.84. The rate of in-hospital mortality among patients categorized in the high BCR group was significantly higher compared to those in the low BCR group (22.6% vs. 12.2%, P < 0.001). The multivariable logistic regression analysis results indicated that, even after accounting for potential confounding factors, patients with elevated BCR demonstrated a notably increased in-hospital mortality rate compared to those with lower BCR levels (all P < 0.05), regardless of the model used. Patients in the high BCR group exhibited a 77.77% higher risk of in-hospital mortality than those in the low BCR group [hazard ratio (HR): 1.7777; 95% CI: 1.4016-2.2547].

Conclusion

An elevated BCR level was independently linked with an increased risk of in-hospital mortality among critically ill patients diagnosed with VTE. Given its widespread availability and ease of measurement, BCR could be a valuable tool for risk stratification and prognostic prediction in VTE patients.

Development and validation of early prediction models for new-onset functional impairment in patients after being transferred from the ICU

A significant number of intensive care unit (ICU) survivors experience new-onset functional impairments that impede their activities of daily living (ADL). Currently, no effective assessment tools are available to identify these high-risk patients. This study aims to develop an interpretable machine learning (ML) model for predicting the onset of functional impairment in critically ill patients. Data for this study were sourced from a comprehensive hospital in China, focusing on adult patients admitted to the ICU from August 2022 to August 2023 without prior functional impairments. A least absolute shrinkage and selection operator (LASSO) model was utilized to select predictors for inclusion in the model. Four models, logistic regression, support vector machine (SVM), random forest (RF), and extreme gradient boosting (XGBoost), were constructed and validated. Model performance was assessed using the area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). Additionally, the DALEX package was employed to enhance the interpretability of the final models. The study ultimately included 1,380 patients, with 684 (49.6%) exhibiting new-onset functional impairment on the seventh day after leaving the ICU. Among the four models evaluated, the SVM model demonstrated the best performance, with an AUC of 0.909, accuracy of 0.838, sensitivity of 0.902, specificity of 0.772, PPV of 0.802, and NPV of 0.886. ML models are reliable tools for predicting new-onset functional impairments in critically ill patients. Notably, the SVM model emerged as the most effective, enabling early identification of patients at high risk and facilitating the implementation of timely interventions to improve ADL.

Interpretable machine learning model for early prediction of delirium in elderly patients following intensive care unit admission: a derivation and validation study

Background and objective

Delirium is the most common neuropsychological complication among older adults admitted to the intensive care unit (ICU) and is often associated with a poor prognosis. This study aimed to construct and validate an interpretable machine learning (ML) for early delirium prediction in older ICU patients.

Methods

This was a retrospective observational cohort study and patient data were extracted from the Medical Information Mart for Intensive Care-IV database. Feature variables associated with delirium, including predisposing factors, disease-related factors, and iatrogenic and environmental factors, were selected using least absolute shrinkage and selection operator regression, and prediction models were built using logistic regression, decision trees, support vector machines, extreme gradient boosting (XGBoost), k-nearest neighbors and naive Bayes methods. Multiple metrics were used for evaluation of performance of the models, including the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, recall, F1 score, calibration plot, and decision curve analysis. SHapley Additive exPlanations (SHAP) were used to improve the interpretability of the final model.

Results

Nine thousand seven hundred forty-eight adults aged 65 years or older were included for analysis. Twenty-six features were selected to construct ML prediction models. Among the models compared, the XGBoost model demonstrated the best performance including the highest AUC (0.836), accuracy (0.765), sensitivity (0.713), recall (0.713), and F1 score (0.725) in the training set. It also exhibited excellent discrimination with AUC of 0.810, good calibration, and had the highest net benefit in the validation cohort. The SHAP summary analysis showed that Glasgow Coma Scale, mechanical ventilation, and sedation were the top three risk features for outcome prediction. The SHAP dependency plot and SHAP force analysis interpreted the model at both the factor level and individual level, respectively.

Conclusion

ML is a reliable tool for predicting the risk of critical delirium in elderly patients. By combining XGBoost and SHAP, it can provide clear explanations for personalized risk prediction and more intuitive understanding of the effect of key features in the model. The establishment of such a model would facilitate the early risk assessment and prompt intervention for delirium.

Nomogram for hospital-acquired venous thromboembolism among patients with cardiovascular diseases

BACKGROUND

Identifying venous thromboembolism (VTE) is challenging for patients with cardiovascular diseases due to similar clinical presentation. Most hospital-acquired VTE events are preventable, whereas the implementation of VTE prophylaxis in clinical practice is far from sufficient. There is a lack of hospital-acquired VTE prediction models tailored specifically designed for patients with cardiovascular diseases. We aimed to develop a nomogram predicting hospital-acquired VTE specifically for patients with cardiovascular diseases.

MATERIAL AND METHODS

Consecutive patients with cardiovascular diseases admitted to internal medicine of Fuwai hospital between September 2020 and August 2021 were included. Univariable and multivariable logistic regression were applied to identify risk factors of hospital-acquired VTE. A nomogram was constructed according to multivariable logistic regression, and internally validated by bootstrapping.

RESULTS

A total of 27,235 patients were included. During a median hospitalization of four days, 154 (0.57%) patients developed hospital-acquired VTE. Multivariable logistic regression identified that female sex, age, infection, pulmonary hypertension, obstructive sleep apnea, acute coronary syndrome, cardiomyopathy, heart failure, immobility, central venous catheter, intra-aortic balloon pump and anticoagulation were independently associated with hospital-acquired VTE. The nomogram was constructed with high accuracy in both the training set and validation (concordance index 0.865 in the training set, and 0.864 in validation), which was further confirmed in calibration. Compared to Padua model, the Fuwai model demonstrated significantly better discrimination ability (area under curve 0.865 vs. 0.786, net reclassification index 0.052, 95% confidence interval 0.012-0.091, P = 0.009; integrated discrimination index 0.020, 95% confidence interval 0.001-0.039, P = 0.051).

CONCLUSION

The incidence of hospital-acquired VTE in patients with cardiovascular diseases is relatively low. The nomogram exhibits high accuracy in predicting hospital-acquired VTE in patients with cardiovascular diseases.

Interpretable machine learning for predicting chronic kidney disease progression risk

Objective

Chronic kidney disease (CKD) poses a major global health burden. Early CKD risk prediction enables timely interventions, but conventional models have limited accuracy. Machine learning (ML) enhances prediction, but interpretability is needed to support clinical usage with both in diagnostic and decision-making.

Methods

A cohort of 491 patients with clinical data was collected for this study. The dataset was randomly split into an 80% training set and a 20% testing set. To achieve the first objective, we developed four ML algorithms (logistic regression, random forests, neural networks, and eXtreme Gradient Boosting (XGBoost)) to classify patients into two classes-those who progressed to CKD stages 3-5 during follow-up (positive class) and those who did not (negative class). For the classification task, the area under the receiver operating characteristic curve (AUC-ROC) was used to evaluate model performance in discriminating between the two classes. For survival analysis, Cox proportional hazards regression (COX) and random survival forests (RSFs) were employed to predict CKD progression, and the concordance index (C-index) and integrated Brier score were used for model evaluation. Furthermore, variable importance, partial dependence plots, and restrict cubic splines were used to interpret the models' results.

Results

XGBOOST demonstrated the best predictive performance for CKD progression in the classification task, with an AUC-ROC of 0.867 (95% confidence interval (CI): 0.728-0.100), outperforming the other ML algorithms. In survival analysis, RSF showed slightly better discrimination and calibration on the test set compared to COX, indicating better generalization to new data. Variable importance analysis identified estimated glomerular filtration rate, age, and creatinine as the most important predictors for CKD survival analysis. Further analysis revealed non-linear associations between age and CKD progression, suggesting higher risks in patients aged 52-55 and 65-66 years. The association between cholesterol levels and CKD progression was also non-linear, with lower risks observed when cholesterol levels were in the range of 5.8-6.4 mmol/L.

Conclusions

Our study demonstrated the effectiveness of interpretable ML models for predicting CKD progression. The comparison between COX and RSF highlighted the advantages of ML in survival analysis, particularly in handling non-linearity and high-dimensional data. By leveraging interpretable ML for unraveling risk factor relationships, contrasting predictive techniques, and exposing non-linear associations, this study significantly advances CKD risk prediction to enable enhanced clinical decision-making.

External Validation of the ICU-Venous Thromboembolism Risk Assessment Model in Adult Critically Ill Patients

BACKGROUND

Currently, no universally accepted standardized VTE risk assessment model (RAM) is specifically designed for critically ill patients. Although the ICU-venous thromboembolism (ICU-VTE) RAM was initially developed in 2020, it lacks prospective external validation.

OBJECTIVES

To evaluate the predictive performance of the ICU-VTE RAM in terms of VTE occurrence in mixed medical-surgical ICU patients.

METHODS

We prospectively enrolled adult patients in the ICU. The ICU-VTE score and Caprini or Padua score were calculated at admission, and the incidence of in-hospital VTE was investigated. The performance of the ICU-VTE RAM was evaluated and compared with that of Caprini or Padua RAM using the receiver operating curve.

RESULTS

We included 269 patients (median age: 70 years; 62.5% male). Eighty-three (30.9%) patients experienced inpatient VTE. The AUC of the ICU-VTE RAM was 0.743 (95% CI, 0.682-0.804, P < 0.001) for mixed medical-surgical ICU patients. Comparatively, the performance of the ICU-VTE RAM was superior to that of the Pauda RAM (AUC: 0.727 vs 0.583, P < 0.001) in critically ill medical patients and the Caprini RAM (AUC: 0.774 vs 0.617, P = 0.128) in critically ill surgical patients, although the latter comparison was not statistically significant.

CONCLUSIONS

The ICU-VTE RAM may be a practical and valuable tool for identifying and stratifying VTE risk in mixed medical-surgical critically ill patients, aiding in managing and preventing VTE complications.