Issue of Data Imbalance on Low Birthweight Baby Outcomes Prediction and Associated Risk Factors Identification: Establishment of Benchmarking Key Machine Learning Models With Data Rebalancing Strategies

Personalized Prediction of Long-Term Renal Function Prognosis Following Nephrectomy Using Interpretable Machine Learning Algorithms: Case-Control Study

BACKGROUND

Acute kidney injury (AKI) is a common adverse outcome following nephrectomy. The progression from AKI to acute kidney disease (AKD) and subsequently to chronic kidney disease (CKD) remains a concern; yet, the predictive mechanisms for these transitions are not fully understood. Interpretable machine learning (ML) models offer insights into how clinical features influence long-term renal function outcomes after nephrectomy, providing a more precise framework for identifying patients at risk and supporting improved clinical decision-making processes.

OBJECTIVE

This study aimed to (1) evaluate postnephrectomy rates of AKI, AKD, and CKD, analyzing long-term renal outcomes along different trajectories; (2) interpret AKD and CKD models using Shapley Additive Explanations values and Local Interpretable Model-Agnostic Explanations algorithm; and (3) develop a web-based tool for estimating AKD or CKD risk after nephrectomy.

METHODS

We conducted a retrospective cohort study involving patients who underwent nephrectomy between July 2012 and June 2019. Patient data were randomly split into training, validation, and test sets, maintaining a ratio of 76.5:8.5:15. Eight ML algorithms were used to construct predictive models for postoperative AKD and CKD. The performance of the best-performing models was assessed using various metrics. We used various Shapley Additive Explanations plots and Local Interpretable Model-Agnostic Explanations bar plots to interpret the model and generated directed acyclic graphs to explore the potential causal relationships between features. Additionally, we developed a web-based prediction tool using the top 10 features for AKD prediction and the top 5 features for CKD prediction.

RESULTS

The study cohort comprised 1559 patients. Incidence rates for AKI, AKD, and CKD were 21.7% (n=330), 15.3% (n=238), and 10.6% (n=165), respectively. Among the evaluated ML models, the Light Gradient-Boosting Machine (LightGBM) model demonstrated superior performance, with an area under the receiver operating characteristic curve of 0.97 for AKD prediction and 0.96 for CKD prediction. Performance metrics and plots highlighted the model's competence in discrimination, calibration, and clinical applicability. Operative duration, hemoglobin, blood loss, urine protein, and hematocrit were identified as the top 5 features associated with predicted AKD. Baseline estimated glomerular filtration rate, pathology, trajectories of renal function, age, and total bilirubin were the top 5 features associated with predicted CKD. Additionally, we developed a web application using the LightGBM model to estimate AKD and CKD risks.

CONCLUSIONS

An interpretable ML model effectively elucidated its decision-making process in identifying patients at risk of AKD and CKD following nephrectomy by enumerating critical features. The web-based calculator, found on the LightGBM model, can assist in formulating more personalized and evidence-based clinical strategies.

Processing imbalanced medical data at the data level with assisted-reproduction data as an example

OBJECTIVE

Data imbalance is a pervasive issue in medical data mining, often leading to biased and unreliable predictive models. This study aims to address the urgent need for effective strategies to mitigate the impact of data imbalance on classification models. We focus on quantifying the effects of different imbalance degrees and sample sizes on model performance, identifying optimal cut-off values, and evaluating the efficacy of various methods to enhance model accuracy in highly imbalanced and small sample size scenarios.

METHODS

We collected medical records of patients receiving assisted reproductive treatment in a reproductive medicine center. Random forest was used to screen the key variables for the prediction target. Various datasets with different imbalance degrees and sample sizes were constructed to compare the classification performance of logistic regression models. Metrics such as AUC, G-mean, F1-Score, Accuracy, Recall, and Precision were used for evaluation. Four imbalance treatment methods (SMOTE, ADASYN, OSS, and CNN) were applied to datasets with low positive rates and small sample sizes to assess their effectiveness.

RESULTS

The logistic model's performance was low when the positive rate was below 10% but stabilized beyond this threshold. Similarly, sample sizes below 1200 yielded poor results, with improvement seen above this threshold. For robustness, the optimal cut-offs for positive rate and sample size were identified as 15% and 1500, respectively. SMOTE and ADASYN oversampling significantly improved classification performance in datasets with low positive rates and small sample sizes.

CONCLUSIONS

The study identifies a positive rate of 15% and a sample size of 1500 as optimal cut-offs for stable logistic model performance. For datasets with low positive rates and small sample sizes, SMOTE and ADASYN are recommended to improve balance and model accuracy.

Machine learning-based risk prediction of acute kidney disease and hospital mortality in older patients

Introduction

Acute kidney injury (AKI) is a prevalent complication in older people, elevating the risks of acute kidney disease (AKD) and mortality. AKD reflects the adverse events developing after AKI. We aimed to develop and validate machine learning models for predicting the occurrence of AKD, AKI and mortality in older patients.

Methods

We retrospectively reviewed the medical records of older patients (aged 65 years and above). To explore the trajectory of kidney dysfunction, patients were categorized into four groups: no kidney disease, AKI recovery, AKD without AKI, or AKD with AKI. We developed eight machine learning models to predict AKD, AKI, and mortality. The best-performing model was identified based on the area under the receiver operating characteristic curve (AUC) and interpreted using the Shapley additive explanations (SHAP) method.

Results

A total of 22,005 patients were finally included in our study. Among them, 4,434 patients (20.15%) developed AKD, 4,000 (18.18%) occurred AKI, and 866 (3.94%) patients deceased. Light gradient boosting machine (LGBM) outperformed in predicting AKD, AKI, and mortality, and the final lite models with 15 features had AUC values of 0.760, 0.767, and 0.927, respectively. The SHAP method revealed that AKI stage, albumin, lactate dehydrogenase, aspirin and coronary heart disease were the top 5 predictors of AKD. An online prediction website for AKD and mortality was developed based on the final models.

Discussion

The LGBM models provide a valuable tool for early prediction of AKD, AKI, and mortality in older patients, facilitating timely interventions. This study highlights the potential of machine learning in improving older adult care, with the developed online tool offering practical utility for healthcare professionals. Further research should aim at external validation and integration of these models into clinical practice.

Personalized prediction of mortality in patients with acute ischemic stroke using explainable artificial intelligence

BACKGROUND

Research into the acute kidney disease (AKD) after acute ischemic stroke (AIS) is rare, and how clinical features influence its prognosis remain unknown. We aim to employ interpretable machine learning (ML) models to study AIS and clarify its decision-making process in identifying the risk of mortality.

METHODS

We conducted a retrospective cohort study involving AIS patients from January 2020 to June 2021. Patient data were randomly divided into training and test sets. Eight ML algorithms were employed to construct predictive models for mortality. The performance of the best model was evaluated using various metrics. Furthermore, we created an artificial intelligence (AI)-driven web application that leveraged the top ten most crucial features for mortality prediction.

RESULTS

The study cohort consisted of 1633 AIS patients, among whom 257 (15.74%) developed subacute AKD, 173 (10.59%) experienced AKI recovery, and 65 (3.98%) met criteria for both AKI and AKD. The mortality rate stood at 4.84%. The LightGBM model displayed superior performance, boasting an AUROC of 0.96 for mortality prediction. The top five features linked to mortality were ACEI/ARE, renal function trajectories, neutrophil count, diuretics, and serum creatinine. Moreover, we designed a web application using the LightGBM model to estimate mortality risk.

CONCLUSIONS

Complete renal function trajectories, including AKI and AKD, are vital for fitting mortality in AIS patients. An interpretable ML model effectively clarified its decision-making process for identifying AIS patients at risk of mortality. The AI-driven web application has the potential to contribute to the development of personalized early mortality prevention.

Community screening for dementia among older adults in China: a machine learning-based strategy

BACKGROUND

Dementia is a leading cause of disability in people older than 65 years worldwide. However, diagnosing dementia in its earliest symptomatic stages remains challenging. This study combined specific questions from the AD8 scale with comprehensive health-related characteristics, and used machine learning (ML) to construct diagnostic models of cognitive impairment (CI).

METHODS

The study was based on the Shenzhen Healthy Ageing Research (SHARE) project, and we recruited 823 participants aged 65 years and older, who completed a comprehensive health assessment and cognitive function assessments. Permutation importance was used to select features. Five ML models using BalanceCascade were applied to predict CI: a support vector machine (SVM), multilayer perceptron (MLP), AdaBoost, gradient boosting decision tree (GBDT), and logistic regression (LR). An AD8 score ≥ 2 was used to define CI as a baseline. SHapley Additive exPlanations (SHAP) values were used to interpret the results of ML models.

RESULTS

The first and sixth items of AD8, platelets, waist circumference, body mass index, carcinoembryonic antigens, age, serum uric acid, white blood cells, abnormal electrocardiogram, heart rate, and sex were selected as predictive features. Compared to the baseline (AUC = 0.65), the MLP showed the highest performance (AUC: 0.83 ± 0.04), followed by AdaBoost (AUC: 0.80 ± 0.04), SVM (AUC: 0.78 ± 0.04), GBDT (0.76 ± 0.04). Furthermore, the accuracy, sensitivity and specificity of four ML models were higher than the baseline. SHAP summary plots based on MLP showed the most influential feature on model decision for positive CI prediction was female sex, followed by older age and lower waist circumference.

CONCLUSIONS

The diagnostic models of CI applying ML, especially the MLP, were substantially more effective than the traditional AD8 scale with a score of ≥ 2 points. Our findings may provide new ideas for community dementia screening and to promote such screening while minimizing medical and health resources.

Machine Learning-Based Prediction of Suicidality in Adolescents With Allergic Rhinitis: Derivation and Validation in 2 Independent Nationwide Cohorts

BACKGROUND

Given the additional risk of suicide-related behaviors in adolescents with allergic rhinitis (AR), it is important to use the growing field of machine learning (ML) to evaluate this risk.

OBJECTIVE

This study aims to evaluate the validity and usefulness of an ML model for predicting suicide risk in patients with AR.

METHODS

We used data from 2 independent survey studies, Korea Youth Risk Behavior Web-based Survey (KYRBS; n=299,468) for the original data set and Korea National Health and Nutrition Examination Survey (KNHANES; n=833) for the external validation data set, to predict suicide risks of AR in adolescents aged 13 to 18 years, with 3.45% (10,341/299,468) and 1.4% (12/833) of the patients attempting suicide in the KYRBS and KNHANES studies, respectively. The outcome of interest was the suicide attempt risks. We selected various ML-based models with hyperparameter tuning in the discovery and performed an area under the receiver operating characteristic curve (AUROC) analysis in the train, test, and external validation data.

RESULTS

The study data set included 299,468 (KYRBS; original data set) and 833 (KNHANES; external validation data set) patients with AR recruited between 2005 and 2022. The best-performing ML model was the random forest model with a mean AUROC of 84.12% (95% CI 83.98%-84.27%) in the original data set. Applying this result to the external validation data set revealed the best performance among the models, with an AUROC of 89.87% (sensitivity 83.33%, specificity 82.58%, accuracy 82.59%, and balanced accuracy 82.96%). While looking at feature importance, the 5 most important features in predicting suicide attempts in adolescent patients with AR are depression, stress status, academic achievement, age, and alcohol consumption.

CONCLUSIONS

This study emphasizes the potential of ML models in predicting suicide risks in patients with AR, encouraging further application of these models in other conditions to enhance adolescent health and decrease suicide rates.

Machine learning-based approach for predicting low birth weight

BACKGROUND

Low birth weight (LBW) has been linked to infant mortality. Predicting LBW is a valuable preventative tool and predictor of newborn health risks. The current study employed a machine learning model to predict LBW.

METHODS

This study implemented predictive LBW models based on the data obtained from the "Iranian Maternal and Neonatal Network (IMaN Net)" from January 2020 to January 2022. Women with singleton pregnancies above the gestational age of 24 weeks were included. Exclusion criteria included multiple pregnancies and fetal anomalies. A predictive model was built using eight statistical learning models (logistic regression, decision tree classification, random forest classification, deep learning feedforward, extreme gradient boost model, light gradient boost model, support vector machine, and permutation feature classification with k-nearest neighbors). Expert opinion and prior observational cohorts were used to select candidate LBW predictors for all models. The area under the receiver operating characteristic curve (AUROC), accuracy, precision, recall, and F1 score were measured to evaluate their diagnostic performance.

RESULTS

We found 1280 women with a recorded LBW out of 8853 deliveries, for a frequency of 14.5%. Deep learning (AUROC: 0.86), random forest classification (AUROC: 0.79), and extreme gradient boost classification (AUROC: 0.79) all have higher AUROC and perform better than others. When the other performance parameters of the models mentioned above with higher AUROC were compared, the extreme gradient boost model was the best model to predict LBW with an accuracy of 0.79, precision of 0.87, recall of 0.69, and F1 score of 0.77. According to the feature importance rank, gestational age and prior history of LBW were the top critical predictors.

CONCLUSIONS

Although this study found that the extreme gradient boost model performed well in predicting LBW, more research is needed to make a better conclusion on the performance of ML models in predicting LBW.