Machine learning models for predicting acute kidney injury: a systematic review and critical appraisal

Predictive models of sepsis-associated acute kidney injury based on machine learning: a scoping review

BACKGROUND

With the development of artificial intelligence, the application of machine learning to develop predictive models for sepsis-associated acute kidney injury has made potential breakthroughs in early identification, grading, diagnosis, and prognosis determination.

METHODS

Here, we conducted a systematic search of the PubMed, Cochrane Library, Embase (Ovid), Web of Science, and Scopus databases on April 28, 2023, and screened relevant literature. Then, we comprehensively extracted relevant data related to machine learning algorithms, predictors, and predicted objectives. We subsequently performed a critical evaluation of research quality, data aggregation, and analyses.

RESULTS

We screened 25 studies on predictive models for sepsis-associated acute kidney injury from a total of originally identified 2898 studies. The most commonly used machine learning algorithm is traditional logistic regression, followed by eXtreme gradient boosting. We categorized these predictive models into early identification models (60%), prognostic prediction models (32%), and subtype identification models (8%) according to their predictive purpose. The five most commonly used predictors were serum creatinine levels, lactate levels, age, blood urea nitrogen concentration, and diabetes mellitus. In addition, a single data source, insufficient assessment of clinical utility, lack of model bias assessment, and hyperparameter adjustment may be the main reasons for the low quality of the current research.

CONCLUSIONS

However, studies on the nondeath prognostic outcomes, the long-term clinical outcomes, and the subtype identification models are insufficient. Additionally, the poor quality of the research and the insufficient practicality of the model are problems that need to be addressed urgently.

AKIMLpred: An interpretable machine learning model for predicting acute kidney injury within seven days in critically ill patients based on a prospective cohort study

BACKGROUND

Early recognition and timely intervention for AKI in critically ill patients were crucial to reduce morbidity and mortality. This study aimed to use biomarkers to construct a optimal machine learning model for early prediction of AKI in critically ill patients within seven days.

METHODS

The prospective cohort study enrolled 929 patients altogether who were admitted in ICU including 680 patients in training set (Jiefang Campus) and 249 patients in external testing set (Binjiang Campus). After performing strict inclusion and exclusion criteria, 421 patients were selected in training set for constructing predictive model and 167 patients were selected in external testing for evaluating the predictive performance of resulting model. Urine and blood samples were collected for kidney injury associated biomarkers detection. Baseline clinical information and laboratory data of the study participants were collected. We determined the average prediction efficiency of six machine learning models through 10-fold cross validation.

RESULTS

In total, 78 variables were collected when admission in ICU and 43 variables were statistically significant between AKI and non-AKI cohort. Then, 35 variables were selected as independent features for AKI by univariate logistic regression. Spearman correlation analysis was used to remove two highly correlated variables. Three ranking methods were used to explore the influence of 33 variables for further determining the best combination of variables. The gini importance ranking method was found to be applicable for variables filtering. The predictive performance of AKIMLpred which constructed by the XGBoost algorithm was the best among six machine learning models. When the AKIMLpred included the nine features (NGAL, IGFBP7, sCysC, CAF22, KIM-1, NT-proBNP, IL-6, IL-18 and L-FABP) with the highest influence ranking, its model had the best prediction performance, with an AUC of 0.881 and an accuracy of 0.815 in training set, similarly, with an AUC of 0.889 and an accuracy of 0.846 in validation set. Moreover, the performace was slightly outperformed in testing set with an AUC of 0.902 and an accuracy of 0.846. The SHAP algorithm was used to interpret the prediction results of AKIMLpred. The web-calculator of AKIMLpred was shown for predicting AKI with more convenient(https://www.xsmartanalysis.com/model/list/predict/model/html?mid=8065&symbol=11gk693982SU6AE1ms21). AKIMLpred was better than the optimal model built with only routine tests for predicting AKI in critically ill patients within 7 days.

CONCLUSION

The model AKIMLpred constructed by the XGBoost algorithm with selecting the nine most influential biomarkers in the gini importance ranking method had the best performance in predicting AKI in critically ill patients within 7 days. This data-driven predictive model will help clinicians to make quick and accurate diagnosis.

Development and external validation of deep learning clinical prediction models using variable-length time series data

OBJECTIVES

To compare and externally validate popular deep learning model architectures and data transformation methods for variable-length time series data in 3 clinical tasks (clinical deterioration, severe acute kidney injury [AKI], and suspected infection).

MATERIALS AND METHODS

This multicenter retrospective study included admissions at 2 medical centers that spanned 2007-2022. Distinct datasets were created for each clinical task, with 1 site used for training and the other for testing. Three feature engineering methods (normalization, standardization, and piece-wise linear encoding with decision trees [PLE-DTs]) and 3 architectures (long short-term memory/gated recurrent unit [LSTM/GRU], temporal convolutional network, and time-distributed wrapper with convolutional neural network [TDW-CNN]) were compared in each clinical task. Model discrimination was evaluated using the area under the precision-recall curve (AUPRC) and the area under the receiver operating characteristic curve (AUROC).

RESULTS

The study comprised 373 825 admissions for training and 256 128 admissions for testing. LSTM/GRU models tied with TDW-CNN models with both obtaining the highest mean AUPRC in 2 tasks, and LSTM/GRU had the highest mean AUROC across all tasks (deterioration: 0.81, AKI: 0.92, infection: 0.87). PLE-DT with LSTM/GRU achieved the highest AUPRC in all tasks.

DISCUSSION

When externally validated in 3 clinical tasks, the LSTM/GRU model architecture with PLE-DT transformed data demonstrated the highest AUPRC in all tasks. Multiple models achieved similar performance when evaluated using AUROC.

CONCLUSION

The LSTM architecture performs as well or better than some newer architectures, and PLE-DT may enhance the AUPRC in variable-length time series data for predicting clinical outcomes during external validation.

Development and validation of a machine learning model to predict time to renal replacement therapy in patients with chronic kidney disease

BACKGROUND

Predicting time to renal replacement therapy (RRT) is important in patients at high risk for end-stage kidney disease. We developed and validated machine learning models for predicting the time to RRT and compared its accuracy with conventional prediction methods that uses the rate of estimated glomerular filtration rate (eGFR) decline.

METHODS

Data of adult chronic kidney disease (CKD) patients who underwent hemodialysis at Oita University Hospital from April 2016 to March 2021 were extracted from electronic medical records (N = 135). A new machine learning predictor was compared with the established prediction method that uses the eGFR decline rate and the accuracy of the prediction models was determined using the coefficient of determination (R2). The data were preprocessed and split into training and validation datasets. We created multiple machine learning models using the training data and evaluated their accuracy using validation data. Furthermore, we predicted the time to RRT using a conventional prediction method that uses the eGFR decline rate for patients who had measured eGFR three or more times in two years and evaluated its accuracy.

RESULTS

The least absolute shrinkage and selection operator regression model exhibited moderate accuracy with an R2 of 0.60. By contrast, the conventional prediction method was found to be extremely low with an R2 of -17.1.

CONCLUSIONS

The significance of this study is that it shows that machine learning can predict time to RRT moderately well with continuous values from data at a single time point. This approach outperforms the conventional prediction method that uses eGFR time series data and presents new avenues for CKD treatment.

Acute kidney injury in patients with burns

Burn injury is associated with a high risk of acute kidney injury (AKI) with a prevalence of AKI among patients with burns of 9-50%. Despite an improvement in burn injury survival in the past decade, AKI in patients with burns is associated with an extremely poor short-term and long-term prognosis, with a mortality of >80% among those with severe AKI. Factors that contribute to the development of AKI in patients with burns include haemodynamic alterations, burn-induced systemic inflammation and apoptosis, haemolysis, rhabdomyolysis, smoke inhalation injury, drug nephrotoxicity and sepsis. Early and late AKI after burn injury differ in their aetiologies and outcomes. Sepsis is the main driver of late AKI in patients with burns and late AKI has been associated with higher mortality than early AKI. Prevention of early AKI involves correction of hypovolaemia and avoidance of nephrotoxic drugs (for example, hydroxocobalamin), whereas prevention of late AKI involves prevention and early recognition of sepsis as well as avoidance of nephrotoxins. Treatment of AKI in patients with burns remains supportive, including prevention of fluid overload, treatment of electrolyte disturbance and use of kidney replacement therapy when indicated.

Prediction of Postnatal Growth Failure in Very Low Birth Weight Infants Using a Machine Learning Model

Accurate prediction of postnatal growth failure (PGF) can be beneficial for early intervention and prevention. We aimed to develop a machine learning model to predict PGF at discharge among very low birth weight (VLBW) infants using extreme gradient boosting. A total of 729 VLBW infants, born between 2013 and 2017 in four hospitals, were included. PGF was defined as a decrease in z-score between birth and discharge that was greater than 1.28. Feature selection and addition were performed to improve the accuracy of prediction at four different time points, including 0, 7, 14, and 28 days after birth. A total of 12 features with high contribution at all time points by feature importance were decided upon, and good performance was shown as an area under the receiver operating characteristic curve (AUROC) of 0.78 at 7 days. After adding weight change to the 12 features-which included sex, gestational age, birth weight, small for gestational age, maternal hypertension, respiratory distress syndrome, duration of invasive ventilation, duration of non-invasive ventilation, patent ductus arteriosus, sepsis, use of parenteral nutrition, and reach at full enteral nutrition-the AUROC at 7 days after birth was shown as 0.84. Our prediction model for PGF performed well at early detection. Its potential clinical application as a supplemental tool could be helpful for reducing PGF and improving child health.

Dialysis resource allocation in critical care: the impact of the COVID-19 pandemic and the promise of big data analytics

The COVID-19 pandemic resulted in an unprecedented burden on intensive care units (ICUs). With increased demands and limited supply, critical care resources, including dialysis machines, became scarce, leading to the undertaking of value-based cost-effectiveness analyses and the rationing of resources to deliver patient care of the highest quality. A high proportion of COVID-19 patients admitted to the ICU required dialysis, resulting in a major burden on resources such as dialysis machines, nursing staff, technicians, and consumables such as dialysis filters and solutions and anticoagulation medications. Artificial intelligence (AI)-based big data analytics are now being utilized in multiple data-driven healthcare services, including the optimization of healthcare system utilization. Numerous factors can impact dialysis resource allocation to critically ill patients, especially during public health emergencies, but currently, resource allocation is determined using a small number of traditional factors. Smart analytics that take into account all the relevant healthcare information in the hospital system and patient outcomes can lead to improved resource allocation, cost-effectiveness, and quality of care. In this review, we discuss dialysis resource utilization in critical care, the impact of the COVID-19 pandemic, and how AI can improve resource utilization in future public health emergencies. Research in this area should be an important priority.

Machine learning clinical prediction models for acute kidney injury: the impact of baseline creatinine on prediction efficacy

BACKGROUND

There are many Machine Learning (ML) models which predict acute kidney injury (AKI) for hospitalised patients. While a primary goal of these models is to support clinical decision-making, the adoption of inconsistent methods of estimating baseline serum creatinine (sCr) may result in a poor understanding of these models' effectiveness in clinical practice. Until now, the performance of such models with different baselines has not been compared on a single dataset. Additionally, AKI prediction models are known to have a high rate of false positive (FP) events regardless of baseline methods. This warrants further exploration of FP events to provide insight into potential underlying reasons.

OBJECTIVE

The first aim of this study was to assess the variance in performance of ML models using three methods of baseline sCr on a retrospective dataset. The second aim was to conduct an error analysis to gain insight into the underlying factors contributing to FP events.

MATERIALS AND METHODS

The Intensive Care Unit (ICU) patients of the Medical Information Mart for Intensive Care (MIMIC)-IV dataset was used with the KDIGO (Kidney Disease Improving Global Outcome) definition to identify AKI episodes. Three different methods of estimating baseline sCr were defined as (1) the minimum sCr, (2) the Modification of Diet in Renal Disease (MDRD) equation and the minimum sCr and (3) the MDRD equation and the mean of preadmission sCr. For the first aim of this study, a suite of ML models was developed for each baseline and the performance of the models was assessed. An analysis of variance was performed to assess the significant difference between eXtreme Gradient Boosting (XGB) models across all baselines. To address the second aim, Explainable AI (XAI) methods were used to analyse the XGB errors with Baseline 3.

RESULTS

Regarding the first aim, we observed variances in discriminative metrics and calibration errors of ML models when different baseline methods were adopted. Using Baseline 1 resulted in a 14% reduction in the f1 score for both Baseline 2 and Baseline 3. There was no significant difference observed in the results between Baseline 2 and Baseline 3. For the second aim, the FP cohort was analysed using the XAI methods which led to relabelling data with the mean of sCr in 180 to 0 days pre-ICU as the preferred sCr baseline method. The XGB model using this relabelled data achieved an AUC of 0.85, recall of 0.63, precision of 0.54 and f1 score of 0.58. The cohort size was 31,586 admissions, of which 5,473 (17.32%) had AKI.

CONCLUSION

In the absence of a widely accepted method of baseline sCr, AKI prediction studies need to consider the impact of different baseline methods on the effectiveness of ML models and their potential implications in real-world implementations. The utilisation of XAI methods can be effective in providing insight into the occurrence of prediction errors. This can potentially augment the success rate of ML implementation in routine care.

Early recognition and prevention of acute kidney injury in hospitalised children

Acute kidney injury is common in hospitalised children and is associated with poor patient outcomes. Once acute kidney injury occurs, effective therapies to improve patient outcomes or kidney recovery are scarce. Early identification of children at risk of acute kidney injury or at an early injury stage is essential to prevent progression and mitigate complications. Paediatric acute kidney injury is under-recognised by clinicians, which is a barrier to optimisation of inpatient care and follow-up. Acute kidney injury definitions rely on functional biomarkers (ie, serum creatinine and urine output) that are inadequate, since they do not account for biological variability, analytical issues, or physiological responses to volume depletion. Improved predictive tools and diagnostic biomarkers of kidney injury are needed for earlier detection. Novel strategies, including biomarker-guided care algorithms, machine-learning methods, and electronic alerts tied to clinical decision support tools, could improve paediatric acute kidney injury care. Clinical prediction models should be studied in different paediatric populations and acute kidney injury phenotypes. Research is needed to develop and test prevention strategies for acute kidney injury in hospitalised children, including care bundles and therapeutics.

Systematic review of externally validated machine learning models for predicting acute kidney injury in general hospital patients

Acute kidney injury (AKI) is one of the most common and consequential complications among hospitalized patients. Timely AKI risk prediction may allow simple interventions that can minimize or avoid the harm associated with its development. Given the multifactorial and complex etiology of AKI, machine learning (ML) models may be best placed to process the available health data to generate accurate and timely predictions. Accordingly, we searched the literature for externally validated ML models developed from general hospital populations using the current definition of AKI. Of 889 studies screened, only three were retrieved that fit these criteria. While most models performed well and had a sound methodological approach, the main concerns relate to their development and validation in populations with limited diversity, comparable digital ecosystems, use of a vast number of predictor variables and over-reliance on an easily accessible biomarker of kidney injury. These are potentially critical limitations to their applicability in diverse socioeconomic and cultural settings, prompting a need for simpler, more transportable prediction models which can offer a competitive advantage over the current tools used to predict and diagnose AKI.

Data heterogeneity in federated learning with Electronic Health Records: Case studies of risk prediction for acute kidney injury and sepsis diseases in critical care

With the wider availability of healthcare data such as Electronic Health Records (EHR), more and more data-driven based approaches have been proposed to improve the quality-of-care delivery. Predictive modeling, which aims at building computational models for predicting clinical risk, is a popular research topic in healthcare analytics. However, concerns about privacy of healthcare data may hinder the development of effective predictive models that are generalizable because this often requires rich diverse data from multiple clinical institutions. Recently, federated learning (FL) has demonstrated promise in addressing this concern. However, data heterogeneity from different local participating sites may affect prediction performance of federated models. Due to acute kidney injury (AKI) and sepsis' high prevalence among patients admitted to intensive care units (ICU), the early prediction of these conditions based on AI is an important topic in critical care medicine. In this study, we take AKI and sepsis onset risk prediction in ICU as two examples to explore the impact of data heterogeneity in the FL framework as well as compare performances across frameworks. We built predictive models based on local, pooled, and FL frameworks using EHR data across multiple hospitals. The local framework only used data from each site itself. The pooled framework combined data from all sites. In the FL framework, each local site did not have access to other sites' data. A model was updated locally, and its parameters were shared to a central aggregator, which was used to update the federated model's parameters and then subsequently, shared with each site. We found models built within a FL framework outperformed local counterparts. Then, we analyzed variable importance discrepancies across sites and frameworks. Finally, we explored potential sources of the heterogeneity within the EHR data. The different distributions of demographic profiles, medication use, and site information contributed to data heterogeneity.