Development and Validation of a Personalized Model With Transfer Learning for Acute Kidney Injury Risk Estimation Using Electronic Health Records

Artificial intelligence and machine learning: Definition of terms and current concepts in critical care research

With increasing computing power, artificial intelligence (AI) and machine learning (ML) have prospered, which facilitate the analysis of large datasets, especially those found in critical care. It is important to define these terminologies, to inform a standardized approach to critical care research. This manuscript hopes to clarify these terms with examples from medical literature. Three major components that are required for a successful ML implementation: (i) reliable dataset, (ii) ML algorithm, and (iii) unbiased model evaluation, are discussed. A reliable dataset can be structured or unstructured with limited noise, outliers, and missing values. ML, a subset of AI, is typically focused on supervised or unsupervised learning tasks in which the output is based on inputs and derived from iterative pattern recognition algorithms, while AI is the overall ability of a machine to "think" or mimic human behavior; and to analyze data free from human influence. Even with successful implementation, advanced AI and ML algorithms have faced challenges in adoption into practice, mainly due to their lack of interpretability, which hinders trust, buy-in, and engagement from clinicians. Consequently, traditional algorithms, such as linear and logistic regression, that may have reduced predictive power but are highly interpretable, continue to be widely used.

The Road to Precision Medicine for Acute Kidney Injury

OBJECTIVES

Acute kidney injury (AKI) is a common form of organ dysfunction in the ICU. AKI is associated with adverse short- and long-term outcomes, including high mortality rates, which have not measurably improved over the past decade. This review summarizes the available literature examining the evidence of the need for precision medicine in AKI in critical illness, highlights the current evidence for heterogeneity in the field of AKI, discusses the progress made in advancing precision in AKI, and provides a roadmap for studying precision-guided care in AKI.

DATA SOURCES

Medical literature regarding topics relevant to precision medicine in AKI, including AKI definitions, epidemiology, and outcomes, novel AKI biomarkers, studies of electronic health records (EHRs), clinical trial design, and observational studies of kidney biopsies in patients with AKI.

STUDY SELECTION

English language observational studies, randomized clinical trials, reviews, professional society recommendations, and guidelines on areas related to precision medicine in AKI.

DATA EXTRACTION

Relevant study results, statements, and guidelines were qualitatively assessed and narratively synthesized.

DATA SYNTHESIS

We synthesized relevant study results, professional society recommendations, and guidelines in this discussion.

CONCLUSIONS

AKI is a syndrome that encompasses a wide range of underlying pathologies, and this heterogeneity has hindered the development of novel therapeutics for AKI. Wide-ranging efforts to improve precision in AKI have included the validation of novel biomarkers of AKI, leveraging EHRs for disease classification, and phenotyping of tubular secretory clearance. Ongoing efforts such as the Kidney Precision Medicine Project, identifying subphenotypes in AKI, and optimizing clinical trials and endpoints all have great promise in advancing precision medicine in AKI.

Evaluation of the clinical value of 10 estimating glomerular filtration rate equations and construction of a prediction model for kidney damage in adults from central China

Objectives

This study aimed to evaluate 10 estimating glomerular filtration rate (eGFR) equations in central China population and construct a diagnostic prediction model for assessing the kidney damage severity.

Methods

The concordance of 10 eGFR equations was investigated in healthy individuals from central China, and their clinical effectiveness in diagnosing kidney injury was evaluated. Subsequently, relevant clinical indicators were selected to develop a clinical prediction model for kidney damage.

Results

The overall concordance between CKD-EPIASR-Scr and CKD-EPI2021-Scr was the highest (weightedκ = 0.964) in healthy population. The CG formula, CKD-EPIASR-Scr and CKD-EPI2021-Scr performed better than others in terms of concordance with referenced GFR (rGFR), but had poor ability to distinguish between rGFR < 90 or < 60 mL/min·1.73 m2. This finding was basically consistent across subgroups. Finally, two logistic regression prediction models were constructed based on rGFR < 90 or 60 mL/min·1.73 m2. The area under the curve of receiver operating characteristic values of two prediction models were 0.811 vs 0.846 in training set and 0.812 vs 0.800 in testing set.

Conclusion

The concordance of CKD-EPIASR-Scr and CKD-EPI2021-Scr was the highest in the central China population. The Cockcroft-Gault formula, CKD-EPIASR-Scr, and CKD-EPI2021-Scr more accurately reflected true kidney function, while performed poorly in the staging diagnosis of CKD. The diagnostic prediction models showed the good clinical application performance in identifying mild or moderate kidney injury. These findings lay a solid foundation for future research on renal function assessment and predictive equations.

A joint learning method for incomplete and imbalanced data in electronic health record based on generative adversarial networks

Electronic health records (EHR), present challenges of incomplete and imbalanced data in clinical predictions. Previous studies addressed these two issues with two-step separately, which caused the decrease in the performance of prediction tasks. In this paper, we propose a unified framework to simultaneously addresses the challenges of incomplete and imbalanced data in EHR. Based on the framework, we develop a model called Missing Value Imputation and Imbalanced Learning Generative Adversarial Network (MVIIL-GAN). We use MVIIL-GAN to perform joint learning on the imputation process of high missing rate data and the conditional generation process of EHR data. The joint learning is achieved by introducing two discriminators to distinguish the fake data from the generated data at sample-level and variable-level. MVIIL-GAN integrate the missing values imputation and data generation in one step, improving the consistency of parameter optimization and the performance of prediction tasks. We evaluate our framework using the public dataset MIMIC-IV with high missing rates data and imbalanced data. Experimental results show that MVIIL-GAN outperforms existing methods in prediction performance. The implementation of MVIIL-GAN can be found at https://github.com/Peroxidess/MVIIL-GAN.

A framework for integrating artificial intelligence for clinical care with continuous therapeutic monitoring

The complex relationships between continuously monitored health signals and therapeutic regimens can be modelled via machine learning. However, the clinical implementation of the models will require changes to clinical workflows. Here we outline ClinAIOps ('clinical artificial-intelligence operations'), a framework that integrates continuous therapeutic monitoring and the development of artificial intelligence (AI) for clinical care. ClinAIOps leverages three feedback loops to enable the patient to make treatment adjustments using AI outputs, the clinician to oversee patient progress with AI assistance, and the AI developer to receive continuous feedback from both the patient and the clinician. We lay out the central challenges and opportunities in the deployment of ClinAIOps by means of examples of its application in the management of blood pressure, diabetes and Parkinson's disease. By enabling more frequent and accurate measurements of a patient's health and more timely adjustments to their treatment, ClinAIOps may substantially improve patient outcomes.

Temporal Generalizability of Machine Learning Models for Predicting Postoperative Delirium Using Electronic Health Record Data: Model Development and Validation Study

BACKGROUND

Although machine learning models demonstrate significant potential in predicting postoperative delirium, the advantages of their implementation in real-world settings remain unclear and require a comparison with conventional models in practical applications.

OBJECTIVE

The objective of this study was to validate the temporal generalizability of decision tree ensemble and sparse linear regression models for predicting delirium after surgery compared with that of the traditional logistic regression model.

METHODS

The health record data of patients hospitalized at an advanced emergency and critical care medical center in Kumamoto, Japan, were collected electronically. We developed a decision tree ensemble model using extreme gradient boosting (XGBoost) and a sparse linear regression model using least absolute shrinkage and selection operator (LASSO) regression. To evaluate the predictive performance of the model, we used the area under the receiver operating characteristic curve (AUROC) and the Matthews correlation coefficient (MCC) to measure discrimination and the slope and intercept of the regression between predicted and observed probabilities to measure calibration. The Brier score was evaluated as an overall performance metric. We included 11,863 consecutive patients who underwent surgery with general anesthesia between December 2017 and February 2022. The patients were divided into a derivation cohort before the COVID-19 pandemic and a validation cohort during the COVID-19 pandemic. Postoperative delirium was diagnosed according to the confusion assessment method.

RESULTS

A total of 6497 patients (68.5, SD 14.4 years, women n=2627, 40.4%) were included in the derivation cohort, and 5366 patients (67.8, SD 14.6 years, women n=2105, 39.2%) were included in the validation cohort. Regarding discrimination, the XGBoost model (AUROC 0.87-0.90 and MCC 0.34-0.44) did not significantly outperform the LASSO model (AUROC 0.86-0.89 and MCC 0.34-0.41). The logistic regression model (AUROC 0.84-0.88, MCC 0.33-0.40, slope 1.01-1.19, intercept -0.16 to 0.06, and Brier score 0.06-0.07), with 8 predictors (age, intensive care unit, neurosurgery, emergency admission, anesthesia time, BMI, blood loss during surgery, and use of an ambulance) achieved good predictive performance.

CONCLUSIONS

The XGBoost model did not significantly outperform the LASSO model in predicting postoperative delirium. Furthermore, a parsimonious logistic model with a few important predictors achieved comparable performance to machine learning models in predicting postoperative delirium.

Making the Improbable Possible: Generalizing Models Designed for a Syndrome-Based, Heterogeneous Patient Landscape

Syndromic conditions, such as sepsis, are commonly encountered in the intensive care unit. Although these conditions are easy for clinicians to grasp, these conditions may limit the performance of machine-learning algorithms. Individual hospital practice patterns may limit external generalizability. Data missingness is another barrier to optimal algorithm performance and various strategies exist to mitigate this. Recent advances in data science, such as transfer learning, conformal prediction, and continual learning, may improve generalizability of machine-learning algorithms in critically ill patients. Randomized trials with these approaches are indicated to demonstrate improvements in patient-centered outcomes at this point.

Clinical Risk Prediction Models with Meta-Learning Prototypes of Patient Heterogeneity

Hospitalized patients sometimes have complex health conditions, such as multiple diseases, underlying diseases, and complications. The heterogeneous patient conditions may have various representations. A generalized model ignores the differences among heterogeneous patients, and personalized models, even with transfer learning, are still limited to the small amount of training data and the repeated training process. Meta-learning provides a solution for training similar patients based on few-shot learning; however, cannot address common cross-domain patients. Inspired by prototypical networks [1], we proposed a meta-prototype for Electronic Health Records (EHR), a meta-learning-based model with flexible prototypes representing the heterogeneity in patients. We apply this technique to cardiovascular diseases in MIMIC-III and compare it against a set of benchmark models, and demonstrate its ability to address heterogeneous patient health conditions and improve the model performances from 1.2% to 11.9% on different metrics and prediction tasks.Clinical relevance- Developing an adaptive EHR risk prediction model for outcomes-driven phenotyping of heterogeneous patient health conditions.

Predicting in-hospital outcomes of patients with acute kidney injury

Acute kidney injury (AKI) is prevalent and a leading cause of in-hospital death worldwide. Early prediction of AKI-related clinical events and timely intervention for high-risk patients could improve outcomes. We develop a deep learning model based on a nationwide multicenter cooperative network across China that includes 7,084,339 hospitalized patients, to dynamically predict the risk of in-hospital death (primary outcome) and dialysis (secondary outcome) for patients who developed AKI during hospitalization. A total of 137,084 eligible patients with AKI constitute the analysis set. In the derivation cohort, the area under the receiver operator curve (AUROC) for 24-h, 48-h, 72-h, and 7-day death are 95·05%, 94·23%, 93·53%, and 93·09%, respectively. For dialysis outcome, the AUROC of each time span are 88·32%, 83·31%, 83·20%, and 77·99%, respectively. The predictive performance is consistent in both internal and external validation cohorts. The model can predict important outcomes of patients with AKI, which could be helpful for the early management of AKI.

Factors associated with incidence of acute kidney injury: a Japanese regional population-based cohort study, the Shizuoka study

BACKGROUND

Acute kidney injury (AKI) is a globally critical issue. Most studies about AKI have been conducted in limited settings on perioperative or critically ill patients. As a result, there is little information about the epidemiology and risk factors of AKI in the general population.

METHODS

We conducted a population-based cohort study using the Shizuoka Kokuho Database. We included subjects with records of health checkup results. The observation period for each participant was defined as from the date of insurance enrollment or April 2012, whichever occurred later, until the date of insurance withdrawal or September 2020, whichever was later. Primary outcome was AKI associated with admission based on the ICD-10 code. We described the incidence of AKI and performed a multivariate analysis using potential risk factors selected from comorbidities, medications, and health checkup results.

RESULTS

Of 627,814 subjects, 8044 were diagnosed with AKI (incidence 251 per 100,000 person-years). The AKI group was older, with more males. Most comorbidities and prescribed medications were more common in the AKI group. As novel factors, statins (hazard ratio (HR) 0.84, 95% confidence interval (CI) 0.80-0.89) and physical activity habits (HR 0.79, 95% CI 0.75-0.83) were associated with reduced incidence of AKI. Other variables associated with AKI were approximately consistent with those from previous studies.

CONCLUSIONS

The factors associated with AKI and the incidence of AKI in the general Japanese population are indicated. This study generates the hypothesis that statins and physical activity habits are novel protective factors for AKI.

Transferability and interpretability of the sepsis prediction models in the intensive care unit

BACKGROUND

We aimed to develop an early warning system for real-time sepsis prediction in the ICU by machine learning methods, with tools for interpretative analysis of the predictions. In particular, we focus on the deployment of the system in a target medical center with small historical samples.

METHODS

Light Gradient Boosting Machine (LightGBM) and multilayer perceptron (MLP) were trained on Medical Information Mart for Intensive Care (MIMIC-III) dataset and then finetuned on the private Historical Database of local Ruijin Hospital (HDRJH) using transfer learning technique. The Shapley Additive Explanations (SHAP) analysis was employed to characterize the feature importance in the prediction inference. Ultimately, the performance of the sepsis prediction system was further evaluated in the real-world study in the ICU of the target Ruijin Hospital.

RESULTS

The datasets comprised 6891 patients from MIMIC-III, 453 from HDRJH, and 67 from Ruijin real-world data. The area under the receiver operating characteristic curves (AUCs) for LightGBM and MLP models derived from MIMIC-III were 0.98 - 0.98 and 0.95 - 0.96 respectively on MIMIC-III dataset, and, in comparison, 0.82 - 0.86 and 0.84 - 0.87 respectively on HDRJH, from 1 to  5 h preceding. After transfer learning and ensemble learning, the AUCs of the final ensemble model were enhanced to 0.94 - 0.94 on HDRJH and to 0.86 - 0.9 in the real-world study in the ICU of the target Ruijin Hospital. In addition, the SHAP analysis illustrated the importance of age, antibiotics, net balance, and ventilation for sepsis prediction, making the model interpretable.

CONCLUSIONS

Our machine learning model allows accurate real-time prediction of sepsis within 5-h preceding. Transfer learning can effectively improve the feasibility to deploy the prediction model in the target cohort, and ameliorate the model performance for external validation. SHAP analysis indicates that the role of antibiotic usage and fluid management needs further investigation. We argue that our system and methodology have the potential to improve ICU management by helping medical practitioners identify at-sepsis-risk patients and prepare for timely diagnosis and intervention.

TRIAL REGISTRATION

NCT05088850 (retrospectively registered).

SGLT2i reduces renal injury by improving mitochondrial metabolism and biogenesis

OBJECTIVES

Despite advances in treatment, an effective therapeutic strategy for acute kidney injury (AKI) is still lacking. Considering the widely reported clinical benefits of canagliflozin in the kidneys, we assessed the effects of canagliflozin on AKI.

METHODS

Lipopolysaccharide was used to induce AKI in the presence of canagliflozin.

RESULTS

Canagliflozin treatment reduced blood urea nitrogen and serum creatinine levels and improved the renal tubular structure in mice with lipopolysaccharide-induced septic AKI. Canagliflozin also suppressed the inflammatory response, oxidative stress and tubular cell death in the kidneys during septic AKI. In vitro, canagliflozin supplementation maintained mitochondrial function in lipopolysaccharide-treated HK-2 cells by restoring the mitochondrial membrane potential, inhibiting mitochondrial reactive oxygen species production and normalizing mitochondrial respiratory complex activity. In HK-2 cells, canagliflozin stimulated the adenosine monophosphate-activated protein kinase catalytic subunit alpha 1 (AMPKα1)/peroxisome proliferator-activated receptor-gamma coactivator 1 alpha (PGC1α)/nuclear respiratory factor 1 (NRF1) pathway, thus elevating the number of live and healthy mitochondria following lipopolysaccharide treatment. Inhibition of the AMPKα1/PGC1α/NRF1/mitochondrial biogenesis pathway abolished the protective effects of canagliflozin on renal cell mitochondria and tubular viability. Similarly, the protective effects of canagliflozin on kidney function and tubular structure were abrogated in AMPKα1-knockout mice.

CONCLUSIONS

Canagliflozin could be used to treat septic AKI by activating the AMPKα1/PGC1α/NRF1/mitochondrial biogenesis pathway.

A Simpler Machine Learning Model for Acute Kidney Injury Risk Stratification in Hospitalized Patients

Background: Hospitalization-associated acute kidney injury (AKI), affecting one-in-five inpatients, is associated with increased mortality and major adverse cardiac/kidney endpoints. Early AKI risk stratification may enable closer monitoring and prevention. Given the complexity and resource utilization of existing machine learning models, we aimed to develop a simpler prediction model. Methods: Models were trained and validated to predict risk of AKI using electronic health record (EHR) data available at 24 h of inpatient admission. Input variables included demographics, laboratory values, medications, and comorbidities. Missing values were imputed using multiple imputation by chained equations. Results: 26,410 of 209,300 (12.6%) inpatients developed AKI during admission between 13 July 2012 and 11 July 2018. The area under the receiver operating characteristic curve (AUROC) was 0.86 for Random Forest and 0.85 for LASSO. Based on Youden’s Index, a probability cutoff of >0.15 provided sensitivity and specificity of 0.80 and 0.79, respectively. AKI risk could be successfully predicted in 91% patients who required dialysis. The model predicted AKI an average of 2.3 days before it developed. Conclusions: The proposed simpler machine learning model utilizing data available at 24 h of admission is promising for early AKI risk stratification. It requires external validation and evaluation of effects of risk prediction on clinician behavior and patient outcomes.