Predicting mortality among critically ill patients with acute kidney injury treated with renal replacement therapy: Development and validation of new prediction models

Construction and evaluation of a mortality prediction model for patients with acute kidney injury undergoing continuous renal replacement therapy based on machine learning algorithms

BACKGROUND

To construct and evaluate a predictive model for in-hospital mortality among critically ill patients with acute kidney injury (AKI) undergoing continuous renal replacement therapy (CRRT), based on nine machine learning (ML) algorithm.

METHODS

The study retrospectively included patients with AKI who underwent CRRT during their initial hospitalization in the United States using the medical information mart for intensive care (MIMIC) database IV (version 2.0), as well as in the intensive care unit (ICU) of Huzhou Central Hospital. Patients from the MIMIC database were used as the training cohort to construct the models (from 2008 to 2019, n = 1068). Patients from Huzhou Central Hospital were utilized as the external validation cohort to evaluate the models (from June 2019 to December 2022, n = 327). In the training cohort, least absolute shrinkage and selection operator (LASSO) regression with cross-validation was employed to select features for constructing the model and subsequently established nine ML predictive models. The performance of these nine models on the external validation cohort dataset was comprehensively evaluated based on the area under the receiver operating characteristic curve (AUROC) and the optimal model was selected. A static nomogram and a web-based dynamic nomogram were presented, with a comprehensive evaluation from the perspectives of discrimination (AUROC), calibration (calibration curve) and clinical practicability (DCA curves).

RESULTS

Finally, 1395 eligible patients were enrolled, including 1068 patients in the training cohort and 327 patients in the external validation cohort. In the training cohort, LASSO regression with cross-validation was employed to select features and nine models were individually constructed. Compared to the other eight models, the Lasso regularized logistic regression (Lasso-LR) model exhibited the highest AUROC (0.756) and the optimal calibration curve. The DCA curve suggested a certain clinical utility in predicting in-hospital mortality among critically ill patients with AKI undergoing CRRT. Consequently, the Lasso-LR model was the optimal model and it was visualized as a common nomogram (static nomogram) and a web-based dynamic nomogram (https://chsyh2006.shinyapps.io/dynnomapp/). Discrimination, calibration and DCA curves were employed to assess the performance of the nomogram. The AUROC for the training and external validation cohorts in the nomogram model was 0.771 (95%CI: 0.743, 0.799) and 0.756 (95%CI: 0.702, 0.809), respectively. The calibration slope and Brier score for the training cohort were 1.000 and 0.195, while for the external validation cohort, they were 0.849 and 0.197, respectively. The DCA indicated that the model had a certain clinical application value.

CONCLUSIONS

Our study selected the optimal model and visualized it as a static and dynamic nomogram integrating clinical predictors, so that clinicians can personalized predict the in-hospital outcome of critically ill patients with AKI undergoing CRRT upon ICU admission.

Using machine learning to predict the risk of short-term and long-term death in acute kidney injury patients after commencing CRRT

BACKGROUND

The mortality rate and prognosis of short-term and long-term acute kidney injury (AKI) patients who undergo continuous renal replacement therapy (CRRT) are different. Setting up risk stratification tools for both short-term and long-term deaths is highly important for clinicians.

METHOD

A total of 1535 AKI patients receiving CRRT were included in this study, with 1144 from the training set (the Dryad database) and 391 from the validation set (MIMIC IV database). A model for predicting mortality within 10 and 90 days was built using nine different machine learning (ML) algorithms. AUROC, F1-score, accuracy, sensitivity, specificity, precision, and calibration curves were used to assess the predictive performance of various ML models.

RESULTS

A total of 420 (31.1%) deaths occurred within 10 days, and 1080 (68.8%) deaths occurred within 90 days. The random forest (RF) model performed best in both predicting 10-day (AUROC: 0.80, 95% CI: 0.74-0.84; accuracy: 0.72, 95% CI: 0.67-0.76; F1-score: 0.59) and 90-day mortality (AUROC: 0.78, 95% CI: 0.73-0.83; accuracy: 0.73, 95% CI: 0.69-0.78; F1-score: 0.80). The importance of the feature shows that SOFA scores are rated as the most important risk factor for both 10-day and 90-day mortality.

CONCLUSION

Our study, utilizing multiple machine learning models, estimates the risk of short-term and long-term mortality among AKI patients who commence CRRT. The results demonstrated that the prognostic factors for short-term and long-term mortality are different. The RF model has the best prediction performance and has valuable potential for clinical application.

Ensemble Machine Learning for Predicting 90-Day Outcomes and Analyzing Risk Factors in Acute Kidney Injury Requiring Dialysis

Purpose

Our objectives were to (1) employ ensemble machine learning algorithms utilizing real-world clinical data to predict 90-day prognosis, including dialysis dependence and mortality, following the first hospitalized dialysis and (2) identify the significant factors associated with overall outcomes.

Patients and Methods

We identified hospitalized patients with Acute kidney injury requiring dialysis (AKI-D) from a dataset of the Taipei Medical University Clinical Research Database (TMUCRD) from January 2008 to December 2020. The extracted data comprise demographics, comorbidities, medications, and laboratory parameters. Ensemble machine learning models were developed utilizing real-world clinical data through the Google Cloud Platform.

Results

The Study Analyzed 1080 Patients in the Dialysis-Dependent Module, Out of Which 616 Received Regular Dialysis After 90 Days. Our Ensemble Model, Consisting of 25 Feedforward Neural Network Models, Demonstrated the Best Performance with an Auroc of 0.846. We Identified the Baseline Creatinine Value, Assessed at Least 90 Days Before the Initial Dialysis, as the Most Crucial Factor. We selected 2358 patients, 984 of whom were deceased after 90 days, for the survival module. The ensemble model, comprising 15 feedforward neural network models and 10 gradient-boosted decision tree models, achieved superior performance with an AUROC of 0.865. The pre-dialysis creatinine value, tested within 90 days prior to the initial dialysis, was identified as the most significant factor.

Conclusion

Ensemble machine learning models outperform logistic regression models in predicting outcomes of AKI-D, compared to existing literature. Our study, which includes a large sample size from three different hospitals, supports the significance of the creatinine value tested before the first hospitalized dialysis in determining overall prognosis. Healthcare providers could benefit from utilizing our validated prediction model to improve clinical decision-making and enhance patient care for the high-risk population.

Predicting outcomes of acute kidney injury in critically ill patients using machine learning

Acute Kidney Injury (AKI) is a sudden episode of kidney failure that is frequently seen in critically ill patients. AKI has been linked to chronic kidney disease (CKD) and mortality. We developed machine learning-based prediction models to predict outcomes following AKI stage 3 events in the intensive care unit. We conducted a prospective observational study that used the medical records of ICU patients diagnosed with AKI stage 3. A random forest algorithm was used to develop two models that can predict patients who will progress to CKD after three and six months of experiencing AKI stage 3. To predict mortality, two survival prediction models have been presented using random survival forests and survival XGBoost. We evaluated established CKD prediction models using AUCROC, and AUPR curves and compared them with the baseline logistic regression models. The mortality prediction models were evaluated with an external test set, and the C-indices were compared to baseline COXPH. We included 101 critically ill patients who experienced AKI stage 3. To increase the training set for the mortality prediction task, an unlabeled dataset has been added. The RF (AUPR: 0.895 and 0.848) and XGBoost (c-index: 0.8248) models have a better performance than the baseline models in predicting CKD and mortality, respectively Machine learning-based models can assist clinicians in making clinical decisions regarding critically ill patients with severe AKI who are likely to develop CKD following discharge. Additionally, we have shown better performance when unlabeled data are incorporated into the survival analysis task.

Development and validation of clinical prediction models for acute kidney injury recovery at hospital discharge in critically ill adults

PURPOSE

Acute kidney injury (AKI) recovery prediction remains challenging. The purpose of the present study is to develop and validate prediction models for AKI recovery at hospital discharge in critically ill patients with ICU-acquired AKI stage 3 (AKI-3).

METHODS

Models were developed and validated in a development cohort (n = 229) and a matched validation cohort (n = 244) from the multicenter EPaNIC database to create prediction models with the least absolute shrinkage and selection operator (Lasso) machine-learning algorithm. We evaluated the discrimination and calibration of the models and compared their performance with plasma neutrophil gelatinase-associated lipocalin (NGAL) measured on first AKI-3 day (NGAL_AKI3) and reference model that only based on age.

RESULTS

Complete recovery and complete or partial recovery occurred in 33.20% and 51.23% of the validation cohort patients respectively. The prediction model for complete recovery based on age, need for renal replacement therapy (RRT), diagnostic group (cardiac/surgical/trauma/others), and sepsis on admission had an area under the receiver operating characteristics curve (AUROC) of 0.53. The prediction model for complete or partial recovery based on age, need for RRT, platelet count, urea, and white blood cell count had an AUROC of 0.61. NGAL_AKI3 showed AUROCs of 0.55 and 0.53 respectively. In cardiac patients, the models had higher AUROCs of 0.60 and 0.71 than NGAL_AKI3's AUROCs of 0.52 and 0.54. The developed models demonstrated a better performance over the reference models (only based on age) for cardiac surgery patients, but not for patients with sepsis and for a general ICU population.

CONCLUSION

Models to predict AKI recovery upon hospital discharge in critically ill patients with AKI-3 showed poor performance in the general ICU population, similar to the biomarker NGAL. In cardiac surgery patients, discrimination was acceptable, and better than NGAL. These findings demonstrate the difficulty of predicting non-reversible AKI early.

Association between predialysis creatinine and mortality in acute kidney injury patients requiring dialysis

BACKGROUND

Creatinine is widely used to estimate renal function, but this is not practical in critical illness. Low creatinine has been associated with mortality in many clinical settings. However, the associations between predialysis creatinine level, Sepsis-related Organ Failure Assessment (SOFA) score, fluid overload, and mortality in acute kidney injury patients receiving dialysis therapy (AKI-D) has not been fully addressed.

METHODS

We extracted data for AKI-D patients in the eICU and MIMIC databases. We conducted a retrospective observational cohort study using the eICU dataset. The study cohort was divided into the high-creatine group and the low-creatinine group by the median value (4 mg/dL). The baseline patient information included demographic data, laboratory tests, medications, and comorbid conditions. The independent association of creatinine level with 30-day mortality was examined using multivariate logistic regression analysis. In sensitivity analyses, the associations between creatinine, SOFA score, and mortality were analyzed in patients with or without fluid overload. We also carried out an external validity using the MIMIC dataset.

RESULTS

In all 1,600 eICU participants, the 30-day mortality rate was 34.2%. The crude overall mortality rate in the low-creatinine group (44.9%) was significantly higher than that in the high-creatinine group (21.9%; P < 0.001). In the fully adjusted models, the low-creatinine group was associated with a higher risk of 30-day mortality (odds ratio, 1.77; 95% confidence interval, 1.29-2.42; P < 0.001) compared with the high-creatinine group. The low-creatinine group had higher SOFA and nonrenal SOFA scores. In sensitivity analyses, the low-creatinine group had a higher 30-day mortality rate with regard to the BMI or albumin level. Fluid overloaded patients were associated with a significantly worse survival in the low-creatinine group. The results were consistent when assessing the external validity using the MIMIC dataset.

CONCLUSIONS

In patients with AKI-D, lower predialysis creatinine was associated with increased mortality risk. Moreover, the mortality rate was substantially higher in patients with lower predialysis creatinine with concomitant elevation of fluid overload status.

Predicting Mortality Using Machine Learning Algorithms in Patients Who Require Renal Replacement Therapy in the Critical Care Unit

Background: General severity of illness scores are not well calibrated to predict mortality among patients receiving renal replacement therapy (RRT) for acute kidney injury (AKI). We developed machine learning models to make mortality prediction and compared their performance to that of the Sequential Organ Failure Assessment (SOFA) and HEpatic failure, LactatE, NorepInephrine, medical Condition, and Creatinine (HELENICC) scores. Methods: We extracted routinely collected clinical data for AKI patients requiring RRT in the MIMIC and eICU databases. The development models were trained in 80% of the pooled dataset and tested in the rest of the pooled dataset. We compared the area under the receiver operating characteristic curves (AUCs) of four machine learning models (multilayer perceptron [MLP], logistic regression, XGBoost, and random forest [RF]) to that of the SOFA, nonrenal SOFA, and HELENICC scores and assessed calibration, sensitivity, specificity, positive (PPV) and negative (NPV) predicted values, and accuracy. Results: The mortality AUC of machine learning models was highest for XGBoost (0.823; 95% confidence interval [CI], 0.791−0.854) in the testing dataset, and it had the highest accuracy (0.758). The XGBoost model showed no evidence of lack of fit with the Hosmer−Lemeshow test (p > 0.05). Conclusion: XGBoost provided the highest performance of mortality prediction for patients with AKI requiring RRT compared with previous scoring systems.

Application of interpretable machine learning for early prediction of prognosis in acute kidney injury

Background

This study aimed to develop an algorithm using the explainable artificial intelligence (XAI) approaches for the early prediction of mortality in intensive care unit (ICU) patients with acute kidney injury (AKI).

Methods

This study gathered clinical data with AKI patients from the Medical Information Mart for Intensive Care IV (MIMIC-IV) in the US between 2008 and 2019. All the data were further randomly divided into a training cohort and a validation cohort. Seven machine learning methods were used to develop the models for assessing in-hospital mortality. The optimal model was selected based on its accuracy and area under the curve (AUC). The SHapley Additive exPlanation (SHAP) values and Local Interpretable Model-Agnostic Explanations (LIME) algorithm were utilized to interpret the optimal model.

Results

A total of 22,360 patients with AKI were finally enrolled in this study (median age, 69.5 years; female, 42.8%). They were randomly split into a training cohort (16770, 75%) and a validation cohort (5590, 25%). The eXtreme Gradient Boosting (XGBoost) model achieved the best performance with an AUC of 0.890. The SHAP values showed that Glasgow Coma Scale (GCS), blood urea nitrogen, cumulative urine output on Day 1 and age were the top 4 most important variables contributing to the XGBoost model. The LIME algorithm was used to explain the individualized predictions.

Conclusions

Machine-learning models based on clinical features were developed and validated with great performance for the early prediction of a high risk of death in patients with AKI.

Association of Intradialytic Hypotension and Ultrafiltration with AKI-D Outcomes in the Outpatient Dialysis Setting

Identifying modifiable predictors of outcomes for cases of acute kidney injury requiring hemodialysis (AKI-D) will allow better care of patients with AKI-D. All patients with AKI-D discharged to University of Virginia (UVA) outpatient HD units between 1 January 2017 to 31 December 2019 (n = 273) were followed- for up to six months. Dialysis-related parameters were measured during the first 4 weeks of outpatient HD to test the hypothesis that modifiable factors during dialysis are associated with AKI-D outcomes of recovery, End Stage Kidney Disease (ESKD), or death. Patients were 42% female, 67% Caucasian, with mean age 62.8 ± 15.4 years. Median number of dialysis sessions was 11 (6–15), lasting 3.6 ± 0.6 h. At 90 days after starting outpatient HD, 45% recovered, 45% were declared ESKD and 9.9% died, with no significant changes noted between three and six months. Patients who recovered, died or were declared ESKD experienced an average of 9, 10 and 16 intradialytic hypotensive (IDH) episodes, respectively. More frequent IDH episodes were associated with increased risk of ESKD (p = 0.01). A one liter increment in net ultrafiltration was associated with 54% increased ratio of ESKD (p = 0.048). Optimizing dialysis prescription to decrease frequency of IDH episodes and minimize UF, and close monitoring of outpatient dialysis for patients with AKI-D, are crucial and may improve outcomes for these patients.

An explainable supervised machine learning predictor of acute kidney injury after adult deceased donor liver transplantation

Background

Early prediction of acute kidney injury (AKI) after liver transplantation (LT) facilitates timely recognition and intervention. We aimed to build a risk predictor of post-LT AKI via supervised machine learning and visualize the mechanism driving within to assist clinical decision-making.

Methods

Data of 894 cases that underwent liver transplantation from January 2015 to September 2019 were collected, covering demographics, donor characteristics, etiology, peri-operative laboratory results, co-morbidities and medications. The primary outcome was new-onset AKI after LT according to Kidney Disease Improving Global Outcomes guidelines. Predicting performance of five classifiers including logistic regression, support vector machine, random forest, gradient boosting machine (GBM) and adaptive boosting were respectively evaluated by the area under the receiver-operating characteristic curve (AUC), accuracy, F1-score, sensitivity and specificity. Model with the best performance was validated in an independent dataset involving 195 adult LT cases from October 2019 to March 2021. SHapley Additive exPlanations (SHAP) method was applied to evaluate feature importance and explain the predictions made by ML algorithms.

Results

430 AKI cases (55.1%) were diagnosed out of 780 included cases. The GBM model achieved the highest AUC (0.76, CI 0.70 to 0.82), F1-score (0.73, CI 0.66 to 0.79) and sensitivity (0.74, CI 0.66 to 0.8) in the internal validation set, and a comparable AUC (0.75, CI 0.67 to 0.81) in the external validation set. High preoperative indirect bilirubin, low intraoperative urine output, long anesthesia time, low preoperative platelets, and graft steatosis graded NASH CRN 1 and above were revealed by SHAP method the top 5 important variables contributing to the diagnosis of post-LT AKI made by GBM model.

Conclusions

Our GBM-based predictor of post-LT AKI provides a highly interoperable tool across institutions to assist decision-making after LT.

Graphic abstract

Supplementary Information

The online version contains supplementary material available at 10.1186/s12967-021-02990-4.

Interhospital Transfer and Outcomes in Patients with AKI: A Population-Based Cohort Study

BACKGROUND

Patients with AKI may require interhospital transfer to receive RRT. Interhospital transfer may lead to delays in therapy, resulting in poor patient outcomes. There is minimal data comparing outcomes among patients undergoing transfer for RRT versus those who receive RRT at the hospital to which they first present.

METHODS

We conducted a population-based cohort study of all adult patients (≥19 years) who received acute dialysis within 14 days of admission to an acute-care hospital between April 1, 2004 and March 31, 2015. The transferred group included all patients who presented to a hospital without a dialysis program and underwent interhospital transfer (with the start of dialysis ≤3 days of transfer and within 14 days of initial admission). All other patients were considered nontransferred. The primary outcome was time to 90-day all-cause mortality, adjusting for demographics, comorbidities, and measures of acute illness severity. We also assessed chronic dialysis dependence as a secondary outcome, using the Fine and Gray proportional hazards model to account for the competing risks of death. In a secondary post hoc analysis, we assessed these outcomes in a propensity score-matched cohort, matching on age, sex, and prior CKD status.

RESULTS

We identified 27,270 individuals initiating acute RRT within 14 days of a hospital admission, of whom 2113 underwent interhospital transfer. Interhospital transfer was associated with lower rate of mortality (adjusted hazard ratio [aHR], 0.90; 95% CI, 0.84 to 0.97). Chronic dialysis dependence was not significantly different between groups (aHR, 0.98; 95% CI, 0.91 to 1.06). In the propensity score-matched analysis, interhospital transfer remained associated with a lower risk of death (HR, 0.88; 95% CI, 0.80 to 0.96).

CONCLUSIONS

Interhospital transfer for receipt of RRT does not confer higher mortality or worse kidney outcomes.

Risk of incident bleeding after acute kidney injury: A retrospective cohort study

PURPOSE

End-stage kidney disease (ESKD) causes bleeding diathesis; however, whether these findings are extrapolable to acute kidney injury (AKI) remains uncertain. We assessed whether AKI is associated with an increased risk of bleeding.

METHODS

Single-center retrospective cohort study, excluding readmissions, admissions <24 h, ESKD or kidney transplants. The primary outcome was the development of incident bleeding analyzed by multivariate time-dependent Cox models.

RESULTS

In 1001 patients, bleeding occurred in 48% of AKI and 57% of non-AKI patients (p = .007). To identify predictors of incident bleeding, we excluded patients who bled before ICU (n = 488). In bleeding-free patients (n = 513), we observed a trend toward higher risks of bleeding in AKI (22% vs. 16%, p = .06), and a higher risk of bleeding in AKI-requiring dialysis (38% vs. 17%, p = .01). Cirrhosis, AKI-requiring dialysis, anticoagulation, and coronary artery disease were associated with bleeding (HR 3.67, 95%CI:1.33-10.25; HR 2.82, 95%CI:1.26-6.32; HR 2.34, 95%CI:1.45-3.80; and HR 1.84, 95%CI:1.06-3.20, respectively), while SOFA score and sepsis had a protective association (HR 0.92 95%CI:0.84-0.99 and HR 0.55, 95%CI:0.34-0.91, respectively). Incident bleeding was not associated with mortality.

CONCLUSIONS

AKI-requiring dialysis was associated with incident bleeding, independent of anticoagulant administration. Studies are needed to better understand how AKI affects coagulation and clinical outcomes.