Machine learning in predicting graft failure following kidney transplantation: A systematic review of published predictive models

Predicting kidney allograft survival with explainable machine learning

INTRODUCTION

Despite significant progress over the last decades in the survival of kidney allografts, several risk factors remain contributing to worsening kidney function or even loss of transplants. We aimed to evaluate a new machine learning method to identify these variables which may predict the early graft loss in kidney transplant patients and to assess their usefulness for improving clinical decisions.

MATERIAL AND METHODS

A retrospective cohort study was carried out with 627 kidney transplant patients followed at least three months. All these data were pre-processed, and their selected features were used to develop an automatically working a machine learning algorithm; this algorithm was then applied for training and parameterization of the model; and finally, the tested model was then used for the analysis of patients' features that were the most impactful for the prediction of clinical outcomes. Our models were evaluated using the Area Under the Curve (AUC), and the SHapley Additive exPlanations (SHAP) algorithm was used to interpret its predictions.

RESULTS

The final selected model achieved a precision of 0.81, a sensitivity of 0.61, a specificity of 0.89, and an AUC value of 0.84. In our model, serum creatinine levels of kidney transplant patients, evaluated at the hospital discharge, proved to be the most important factor in the decision-making for the allograft loss. Patients with a weight equivalent to a BMI closer to the normal range prior to a kidney transplant are less likely to experience graft loss compared to patients with a BMI below the normal range. The age of patients at transplantation and Polyomavirus (BKPyV) infection had significant impact on clinical outcomes in our model.

CONCLUSIONS

Our algorithm suggests that the main characteristics that impacted early allograft loss were serum creatinine levels at the hospital discharge, as well as the pre-transplant values such as body weight, age of patients, and their BKPyV infection. We propose that machine learning tools can be developed to effectively assist medical decision-making in kidney transplantation.

INSPIRE, a publicly available research dataset for perioperative medicine

We present the INSPIRE dataset, a publicly available research dataset in perioperative medicine, which includes approximately 130,000 surgical operations at an academic institution in South Korea over a ten-year period between 2011 and 2020. This comprehensive dataset includes patient characteristics such as age, sex, American Society of Anesthesiologists physical status classification, diagnosis, surgical procedure code, department, and type of anaesthesia. The dataset also includes vital signs in the operating theatre, general wards, and intensive care units (ICUs), laboratory results from six months before admission to six months after discharge, and medication during hospitalisation. Complications include total hospital and ICU length of stay and in-hospital death. We hope this dataset will inspire collaborative research and development in perioperative medicine and serve as a reproducible external validation dataset to improve surgical outcomes.

Assessment of rainfall-derived inflow and infiltration in sewer systems with machine learning approaches

Rainfall-derived inflow/infiltration (RDII) modelling during heavy rainfall events is essential for sewer flow management. In this study, two machine learning algorithms, random forest (RF) and long short-term memory (LSTM), were developed for sewer flow prediction and RDII estimation based on field monitoring data. The study implemented feature engineering for extracting physically significant features in sewer flow modelling and investigated the importance of the relevant features. The results from two case studies indicated the superior capability of machine learning models in RDII estimation in the combined and separated sewer systems, and LSTM model outperformed the two models. Compared to traditional methods, machine learning models were capable of simulating the temporal variation in RDII processes and improved prediction accuracy for peak flows and RDII volumes in storm events.

LightGBM outperforms other machine learning techniques in predicting graft failure after liver transplantation: Creation of a predictive model through large-scale analysis

BACKGROUND

The incidence of graft failure following liver transplantation (LTx) is consistent. While traditional risk scores for LTx have limited accuracy, the potential of machine learning (ML) in this area remains uncertain, despite its promise in other transplant domains. This study aims to determine ML's predictive limitations in LTx by replicating methods used in previous heart transplant research.

METHODS

This study utilized the UNOS STAR database, selecting 64,384 adult patients who underwent LTx between 2010 and 2020. Gradient boosting models (XGBoost and LightGBM) were used to predict 14, 30, and 90-day graft failure compared to conventional logistic regression model. Models were evaluated using both shuffled and rolling cross-validation (CV) methodologies. Model performance was assessed using the AUC across validation iterations.

RESULTS

In a study comparing predictive models for 14-day, 30-day and 90-day graft survival, LightGBM consistently outperformed other models, achieving the highest AUC of.740,.722, and.700 in shuffled CV methods. However, in rolling CV the accuracy of the model declined across every ML algorithm. The analysis revealed influential factors for graft survival prediction across all models, including total bilirubin, medical condition, recipient age, and donor AST, among others. Several features like donor age and recipient diabetes history were important in two out of three models.

CONCLUSIONS

LightGBM enhances short-term graft survival predictions post-LTx. However, due to changing medical practices and selection criteria, continuous model evaluation is essential. Future studies should focus on temporal variations, clinical implications, and ensure model transparency for broader medical utility.

Predicting mortality and recurrence in colorectal cancer: Comparative assessment of predictive models

Introduction

Colorectal cancer (CRC), also known as colorectal cancer, is a significant disease marked by high fatality rates, ranking as the third leading cause of global mortality. The main objective of this study was to assess the accuracy of predictive models in predicting both mortality events and the probability of disease recurrence.

Method

A retrospective analysis was conducted on a cohort of 284 individuals diagnosed with colorectal cancer between 2001 and 2017. Demographic and clinical data, including gender, disease stage, age at diagnosis, recurrence status, and treatment details, were meticulously recorded. We rigorously evaluated various predictive models, including Decision Trees, Random Forests, Random Survival Forests (RSF), Gradient Boosting, mboost, Deep Learning Neural Network (DLNN), and Cox regression. Performance metrics, such as sensitivity, positive predictive value (PPV), specificity, area under the receiver operating characteristic curve (ROC area), and overall accuracy, were calculated for each model to predict mortality and disease recurrence. The analysis was performed using R version 4.1.3 software and the Python programming language.

Results

For mortality prediction, the mboost model demonstrated the highest sensitivity at 96.9% (95% CI: 0.83-0.99) and an ROC area of 0.88. It also exhibited high specificity at 80% (95% CI: 0.59-0.93), a positive predictive value of 86.1% (95% CI: 0.70-0.95), and an overall accuracy of 89% (95% CI: 0.78-0.96). Random Forests showed perfect sensitivity of 100% (95% CI: 0.85-1) but had low specificity at 0% (95% CI: 0-0.52) and poor overall accuracy (50%). On the other hand, DLNN had the lowest performance metrics for mortality prediction, with a sensitivity of 24% (95% CI: 0.222-0.268), specificity of 75% (95% CI: 0.73-0.77), and a lower positive predictive value of 42% (95% CI: 0.38-0.45). The Gradient Boosting model showed the best performance in predicting recurrence, achieving perfect sensitivity of 100% (95% CI: 0.87-1) and high specificity at 92.9% (95% CI: 0.76-0.99). It also had a high positive predictive value of 93.3% (95% CI: 0.77-0.99). Gradient Boosting, with an ROC area of 96.4%, and mboost, with an ROC area of 75%, demonstrated remarkable performance. DLNN had the lowest performance metrics for recurrence prediction, with sensitivity at 1.75% (95% CI: 0.01-0.02), specificity at 98% (95% CI: 0.97-0.98), and a lower positive predictive value at 52.6% (95% CI: 0.39-0.65).

Conclusion

In summary, the mboost model demonstrated outstanding performance in predicting mortality, achieving exceptional results across various evaluation metrics. Random Forests exhibited perfect sensitivity but showed poor specificity and overall accuracy. The DLNN model displayed the lowest performance metrics for mortality prediction. In terms of recurrence prediction, the Gradient Boosting model outperformed other models with perfect sensitivity, high specificity, and positive predictive value. The DLNN model had the lowest performance metrics for recurrence prediction. Overall, the results emphasize the effectiveness of the mboost and Gradient Boosting models in predicting mortality and recurrence in colorectal cancer patients.

Deceased-Donor Kidney Transplant Outcome Prediction Using Artificial Intelligence to Aid Decision-Making in Kidney Allocation

In kidney transplantation, pairing recipients with the highest longevity with low-risk allografts to optimize graft-donor survival is a complex challenge. Current risk prediction models exhibit limited discriminative and calibration capabilities and have not been compared to modern decision-assisting tools. We aimed to develop a highly accurate risk-stratification index using artificial intelligence (AI) techniques. Using data from the UNOS database (156,749 deceased kidney transplants, 2007-2021), we randomly divided transplants into training (80%) and validation (20%) sets. The primary measure was death-censored graft survival. Four machine learning models were assessed for calibration (integrated Brier score [IBS]) and discrimination (time-dependent concordance [CTD] index), compared with existing models. We conducted decision curve analysis and external validation using UK Transplant data. The Deep Cox mixture model showed the best discriminative performance (area under the curve [AUC] = 0.66, 0.67, and 0.68 at 6, 9, and 12 years post-transplant), with CTD at 0.66. Calibration was adequate (IBS = 0.12), while the kidney donor profile index (KDPI) model had lower CTD (0.59) and AUC (0.60). AI-based D-TOP outperformed the KDPI in evaluating transplant pairs based on graft survival, potentially enhancing deceased donor selection. Advanced computing is poised to influence kidney allocation schemes.

Predicting future occlusion or stenosis of lower extremity bypass grafts using artificial intelligence to simultaneously analyze all flow velocities collected in current and previous ultrasound examinations

Objective

Routine surveillance with duplex ultrasound (DUS) examination is recommended after femoral-popliteal and femoral-tibial-pedal vein bypass grafts with various intervals postoperatively. The presently used methodology to analyze bypass graft DUS examination does not use all the available data and has been shown to have a significant rate for missing impending bypass graft failure. The objective of this research is to investigate recurrent neural networks (RNNs) to predict future bypass graft occlusion or stenosis.

Methods

This study includes DUS examinations of 663 patients who had bypass graft operations done between January 2009 and June 2022. Only examinations without missing values were included. We developed two RNNs (a bidirectional long short-term memory unit and a bidirectional gated recurrent unit) to predict bypass graft occlusion and stenosis based on peak systolic velocities collected in the 2 to 5 previous DUS examinations. We excluded the examinations with missing values and split our data into training and test sets. Then, we applied 10-fold cross-validation on training to optimize the hyperparameters and compared models using the test data.

Results

The bidirectional long short-term memory unit model can gain an overall sensitivity of 0.939, specificity of 0.963, and area under the curve of 0.950 on the prediction of bypass graft occlusion, and an overall sensitivity of 0.915, specificity of 0.909, and area under the curve of 0.912 predicting the development of a future critical stenosis. The results on different bypass types show that the system performs differently on different types. The results on subcohorts based on gender, smoking status, and comorbidities show that the performance on current smokers is lower than the never smoker.

Conclusions

We found that RNNs can gain good sensitivity, specificity, and accuracy for the detection of impending bypass graft occlusion or the future development of a critical bypass graft stenosis using all the available peak systolic velocity data in the present and previous bypass graft DUS examinations. Integrating clinical data, including demographics, social determinants, medication, and other risk factors, together with the DUS examination may result in further improvements.

Clinical Relevance

Detecting bypass graft failure before it occurs is important clinically to prevent amputations, salvage limbs, and save lives. Current methods evaluating screening duplex ultrasound examinations have a significant failure rate for detecting a bypass graft at risk for failure. Artificial intelligence using recurrent neural networks has the potential to improve the detection of at-risk bypass graft before they fail. Additionally, artificial intelligence is in the news and is being applied to many fields. Vascular surgeons need to know its potential to improve vascular outcomes.

Systematic reviews of machine learning in healthcare: a literature review

INTRODUCTION

The increasing availability of data and computing power has made machine learning (ML) a viable approach to faster, more efficient healthcare delivery.

METHODS

A systematic literature review (SLR) of published SLRs evaluating ML applications in healthcare settings published between1 January 2010 and 27 March 2023 was conducted.

RESULTS

In total 220 SLRs covering 10,462 ML algorithms were reviewed. The main application of AI in medicine related to the clinical prediction and disease prognosis in oncology and neurology with the use of imaging data. Accuracy, specificity, and sensitivity were provided in 56%, 28%, and 25% SLRs respectively. Internal and external validation was reported in 53% and less than 1% of the cases respectively. The most common modeling approach was neural networks (2,454 ML algorithms), followed by support vector machine and random forest/decision trees (1,578 and 1,522 ML algorithms, respectively).

EXPERT OPINION

The review indicated considerable reporting gaps in terms of the ML's performance, both internal and external validation. Greater accessibility to healthcare data for developers can ensure the faster adoption of ML algorithms into clinical practice.

NephroCAGE-German-Canadian Consortium on AI for Improved Kidney Transplantation Outcome: Protocol for an Algorithm Development and Validation Study

BACKGROUND

Recent advances in hardware and software enabled the use of artificial intelligence (AI) algorithms for analysis of complex data in a wide range of daily-life use cases. We aim to explore the benefits of applying AI to a specific use case in transplant nephrology: risk prediction for severe posttransplant events. For the first time, we combine multinational real-world transplant data, which require specific legal and technical protection measures.

OBJECTIVE

The German-Canadian NephroCAGE consortium aims to develop and evaluate specific processes, software tools, and methods to (1) combine transplant data of more than 8000 cases over the past decades from leading transplant centers in Germany and Canada, (2) implement specific measures to protect sensitive transplant data, and (3) use multinational data as a foundation for developing high-quality prognostic AI models.

METHODS

To protect sensitive transplant data addressing the first and second objectives, we aim to implement a decentralized NephroCAGE federated learning infrastructure upon a private blockchain. Our NephroCAGE federated learning infrastructure enables a switch of paradigms: instead of pooling sensitive data into a central database for analysis, it enables the transfer of clinical prediction models (CPMs) to clinical sites for local data analyses. Thus, sensitive transplant data reside protected in their original sites while the comparable small algorithms are exchanged instead. For our third objective, we will compare the performance of selected AI algorithms, for example, random forest and extreme gradient boosting, as foundation for CPMs to predict severe short- and long-term posttransplant risks, for example, graft failure or mortality. The CPMs will be trained on donor and recipient data from retrospective cohorts of kidney transplant patients.

RESULTS

We have received initial funding for NephroCAGE in February 2021. All clinical partners have applied for and received ethics approval as of 2022. The process of exploration of clinical transplant database for variable extraction has started at all the centers in 2022. In total, 8120 patient records have been retrieved as of August 2023. The development and validation of CPMs is ongoing as of 2023.

CONCLUSIONS

For the first time, we will (1) combine kidney transplant data from nephrology centers in Germany and Canada, (2) implement federated learning as a foundation to use such real-world transplant data as a basis for the training of CPMs in a privacy-preserving way, and (3) develop a learning software system to investigate population specifics, for example, to understand population heterogeneity, treatment specificities, and individual impact on selected posttransplant outcomes.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/48892.

Characteristics and Admission Preferences of Pediatric Emergency Patients and Their Waiting Time Prediction Using Electronic Medical Record Data: Retrospective Comparative Analysis

BACKGROUND

The growing number of patients visiting pediatric emergency departments could have a detrimental impact on the care provided to children who are triaged as needing urgent attention. Therefore, it has become essential to continuously monitor and analyze the admissions and waiting times of pediatric emergency patients. Despite the significant challenge posed by the shortage of pediatric medical resources in China's health care system, there have been few large-scale studies conducted to analyze visits to the pediatric emergency room.

OBJECTIVE

This study seeks to examine the characteristics and admission patterns of patients in the pediatric emergency department using electronic medical record (EMR) data. Additionally, it aims to develop and assess machine learning models for predicting waiting times for pediatric emergency department visits.

METHODS

This retrospective analysis involved patients who were admitted to the emergency department of Children's Hospital Capital Institute of Pediatrics from January 1, 2021, to December 31, 2021. Clinical data from these admissions were extracted from the electronic medical records, encompassing various variables of interest such as patient demographics, clinical diagnoses, and time stamps of clinical visits. These indicators were collected and compared. Furthermore, we developed and evaluated several computational models for predicting waiting times.

RESULTS

In total, 183,024 eligible admissions from 127,368 pediatric patients were included. During the 12-month study period, pediatric emergency department visits were most frequent among children aged less than 5 years, accounting for 71.26% (130,423/183,024) of the total visits. Additionally, there was a higher proportion of male patients (104,147/183,024, 56.90%) compared with female patients (78,877/183,024, 43.10%). Fever (50,715/183,024, 27.71%), respiratory infection (43,269/183,024, 23.64%), celialgia (9560/183,024, 5.22%), and emesis (6898/183,024, 3.77%) were the leading causes of pediatric emergency room visits. The average daily number of admissions was 501.44, and 18.76% (34,339/183,204) of pediatric emergency department visits resulted in discharge without a prescription or further tests. The median waiting time from registration to seeing a doctor was 27.53 minutes. Prolonged waiting times were observed from April to July, coinciding with an increased number of arrivals, primarily for respiratory diseases. In terms of waiting time prediction, machine learning models, specifically random forest, LightGBM, and XGBoost, outperformed regression methods. On average, these models reduced the root-mean-square error by approximately 17.73% (8.951/50.481) and increased the R2 by approximately 29.33% (0.154/0.525). The SHAP method analysis highlighted that the features "wait.green" and "department" had the most significant influence on waiting times.

CONCLUSIONS

This study offers a contemporary exploration of pediatric emergency room visits, revealing significant variations in admission rates across different periods and uncovering certain admission patterns. The machine learning models, particularly ensemble methods, delivered more dependable waiting time predictions. Patient volume awaiting consultation or treatment and the triage status emerged as crucial factors contributing to prolonged waiting times. Therefore, strategies such as patient diversion to alleviate congestion in emergency departments and optimizing triage systems to reduce average waiting times remain effective approaches to enhance the quality of pediatric health care services in China.

Performance of advanced machine learning algorithms overlogistic regression in predicting hospital readmissions: A meta-analysis

Objectives

Machine learning algorithms are being increasingly used for predicting hospital readmissions. This meta-analysis evaluated the performance of logistic regression (LR) and machine learning (ML) models for the prediction of 30-day hospital readmission among patients in the US.

Methods

Electronic databases (i.e., Medline, PubMed, and Embase) were searched from January 2015 to December 2019. Only studies in the English language were included. Two reviewers performed studies screening, quality appraisal, and data collection. The quality of the studies was assessed using the Quality in Prognosis Studies (QUIPS) tool. Model performance was evaluated using the Area Under the Curve (AUC). A random-effects meta-analysis was performed using STATA 16.

Results

Nine studies were included based on the selection criteria. The most common ML techniques were tree-based methods such as boosting and random forest. Most of the studies had a low risk of bias (8/9). The AUC was greater with ML to predict 30-day all-cause hospital readmission compared with LR [Mean Difference (MD): 0.03; 95% Confidence Interval (CI) 0.01-0.05]. Subgroup analyses found that deep-learning methods had a better performance compared with LR (MD 0.06; 95% CI, 0.04-0.09), followed by neural networks (MD: 0.03; 95% CI, 0.03-0.03), while the AUCs of the tree-based (MD: 0.02; 95% CI -0.00-0.04) and kernel-based (MD: 0.02; 95% CI 0.02 (-0.13-0.16) methods were no different compared to LR. More than half of the studies evaluated heart failure-related rehospitalization (N = 5). For the readmission prediction among heart failure patients, ML performed better compared with LR, with a mean difference in AUC of 0.04 (95% CI, 0.01-0.07). The leave-one-out sensitivity analysis confirmed the robustness of the findings.

Conclusion

Multiple ML methods were used to predict 30-day all-cause hospital readmission. Performance varied across the ML methods, with deep-learning methods showing the best performance over the LR.

Prediction of Outcomes After Heart Transplantation in Pediatric Patients Using National Registry Data: Evaluation of Machine Learning Approaches

BACKGROUND

The prediction of posttransplant health outcomes for pediatric heart transplantation is critical for risk stratification and high-quality posttransplant care.

OBJECTIVE

The purpose of this study was to examine the use of machine learning (ML) models to predict rejection and mortality for pediatric heart transplant recipients.

METHODS

Various ML models were used to predict rejection and mortality at 1, 3, and 5 years after transplantation in pediatric heart transplant recipients using United Network for Organ Sharing data from 1987 to 2019. The variables used for predicting posttransplant outcomes included donor and recipient as well as medical and social factors. We evaluated 7 ML models-extreme gradient boosting (XGBoost), logistic regression, support vector machine, random forest (RF), stochastic gradient descent, multilayer perceptron, and adaptive boosting (AdaBoost)-as well as a deep learning model with 2 hidden layers with 100 neurons and a rectified linear unit (ReLU) activation function followed by batch normalization for each and a classification head with a softmax activation function. We used 10-fold cross-validation to evaluate model performance. Shapley additive explanations (SHAP) values were calculated to estimate the importance of each variable for prediction.

RESULTS

RF and AdaBoost models were the best-performing algorithms for different prediction windows across outcomes. RF outperformed other ML algorithms in predicting 5 of the 6 outcomes (area under the receiver operating characteristic curve [AUROC] 0.664 and 0.706 for 1-year and 3-year rejection, respectively, and AUROC 0.697, 0.758, and 0.763 for 1-year, 3-year, and 5-year mortality, respectively). AdaBoost achieved the best performance for prediction of 5-year rejection (AUROC 0.705).

CONCLUSIONS

This study demonstrates the comparative utility of ML approaches for modeling posttransplant health outcomes using registry data. ML approaches can identify unique risk factors and their complex relationship with outcomes, thereby identifying patients considered to be at risk and informing the transplant community about the potential of these innovative approaches to improve pediatric care after heart transplantation. Future studies are required to translate the information derived from prediction models to optimize counseling, clinical care, and decision-making within pediatric organ transplant centers.

Predicting Time to and Average Quality of Future Offers for Kidney Transplant Candidates Declining a Current Deceased Donor Kidney Offer: A Retrospective Cohort Study

Background

At the time a kidney offer is made by an organ donation organization (ODO), transplant physicians must inform candidates on the pros and cons of accepting or declining the offer. Although physicians have a general idea of expected wait time to kidney transplantation by blood group in their ODO, there are no tools that provide quantitative estimates based on the allocation score used and donor/candidate characteristics. This limits the shared decision-making process at the time of kidney offer as (1) the consequences of declining an offer in terms of wait-time prolongation cannot be provided and (2) the quality of the current offer cannot be compared with that of offers that could be made to the specific candidate in the future. This is especially relevant to older transplant candidates as many ODOs use some form of utility matching in their allocation score.

Objective

We aimed to develop a novel method to provide personalized estimates of wait time to next offer and quality of future offers for kidney transplant candidates if they refused a current deceased donor offer from an ODO.

Design

A retrospective cohort study.

Setting

Administrative data from Transplant Quebec.

Patients

All patients who were actively registered on the kidney transplant wait list at any point between March 29, 2012 and December 13, 2017.

Measurements

The time to next offer was defined as the number of days between the time of the current offer and the next offer if the current one were declined. The quality of the offers was measured with the 10-variable Kidney Donor Risk Index (KDRI) equation.

Methods

Candidate-specific kidney offer arrival was modeled with a marked Poisson process. To derive the lambda parameter for the marked Poisson process for each candidate, the arrival of donors was examined in the 2 years prior to the time of the current offer. The Transplant Quebec allocation score was calculated for each ABO-compatible offer with the characteristics that the candidate presented at the time of the current offer. Offers where the candidate's score was lower than the scores of actual recipients of the second kidneys transplanted were filtered out from the candidate-specific kidney offer arrival. The KDRIs of offers that remained were averaged to provide an estimate of the quality of future offers, to be compared with that of the current offer.

Results

During the study period, there were 848 unique donors and 1696 transplant candidates actively registered. The models provide the following information: average time to next offer, time to which there is a 95% probability of receiving a next offer, average KDRI of future offers. The C-index of the model was 0.72. When compared with providing average group estimates of wait time and KDRI of future offers, the model reduced the root-mean-square error in the predicted time to next offer from 137 to 84 days and that of predicted KDRI of future offers from 0.64 to 0.55. The precision of the model's predictions was higher when observed times to next offer were 5 months or less.

Limitations

The models assume that patients declining an offer remain wait-listed until the next one. The model only updates wait time every year after the time of an offer and not in a continuous fashion.

Conclusion

By providing personalized quantitative estimates of time to and quality of future offers, our new approach can inform the shared decision-making process between transplant candidates and physicians when a kidney offer from a deceased donor is made by an ODO.

Automated screening of potential organ donors using a temporal machine learning model

Organ donation is not meeting demand, and yet 30-60% of potential donors are potentially not identified. Current systems rely on manual identification and referral to an Organ Donation Organization (ODO). We hypothesized that developing an automated screening system based on machine learning could reduce the proportion of missed potentially eligible organ donors. Using routine clinical data and laboratory time-series, we retrospectively developed and tested a neural network model to automatically identify potential organ donors. We first trained a convolutive autoencoder that learned from the longitudinal changes of over 100 types of laboratory results. We then added a deep neural network classifier. This model was compared to a simpler logistic regression model. We observed an AUROC of 0.966 (CI 0.949-0.981) for the neural network and 0.940 (0.908-0.969) for the logistic regression model. At a prespecified cutoff, sensitivity and specificity were similar between both models at 84% and 93%. Accuracy of the neural network model was robust across donor subgroups and remained stable in a prospective simulation, while the logistic regression model performance declined when applied to rarer subgroups and in the prospective simulation. Our findings support using machine learning models to help with the identification of potential organ donors using routinely collected clinical and laboratory data.

Machine learning does not outperform traditional statistical modelling for kidney allograft failure prediction

Machine learning (ML) models have recently shown potential for predicting kidney allograft outcomes. However, their ability to outperform traditional approaches remains poorly investigated. Therefore, using large cohorts of kidney transplant recipients from 14 centers worldwide, we developed ML-based prediction models for kidney allograft survival and compared their prediction performances to those achieved by a validated Cox-Based Prognostication System (CBPS). In a French derivation cohort of 4000 patients, candidate determinants of allograft failure including donor, recipient and transplant-related parameters were used as predictors to develop tree-based models (RSF, RSF-ERT, CIF), Support Vector Machine models (LK-SVM, AK-SVM) and a gradient boosting model (XGBoost). Models were externally validated with cohorts of 2214 patients from Europe, 1537 from North America, and 671 from South America. Among these 8422 kidney transplant recipients, 1081 (12.84%) lost their grafts after a median post-transplant follow-up time of 6.25 years (Inter Quartile Range 4.33-8.73). At seven years post-risk evaluation, the ML models achieved a C-index of 0.788 (95% bootstrap percentile confidence interval 0.736-0.833), 0.779 (0.724-0.825), 0.786 (0.735-0.832), 0.527 (0.456-0.602), 0.704 (0.648-0.759) and 0.767 (0.711-0.815) for RSF, RSF-ERT, CIF, LK-SVM, AK-SVM and XGBoost respectively, compared with 0.808 (0.792-0.829) for the CBPS. In validation cohorts, ML models' discrimination performances were in a similar range of those of the CBPS. Calibrations of the ML models were similar or less accurate than those of the CBPS. Thus, when using a transparent methodological pipeline in validated international cohorts, ML models, despite overall good performances, do not outperform a traditional CBPS in predicting kidney allograft failure. Hence, our current study supports the continued use of traditional statistical approaches for kidney graft prognostication.

Toward generalizing the use of artificial intelligence in nephrology and kidney transplantation

With its robust ability to integrate and learn from large sets of clinical data, artificial intelligence (AI) can now play a role in diagnosis, clinical decision making, and personalized medicine. It is probably the natural progression of traditional statistical techniques. Currently, there are many unmet needs in nephrology and, more particularly, in the kidney transplantation (KT) field. The complexity and increase in the amount of data, and the multitude of nephrology registries worldwide have enabled the explosive use of AI within the field. Nephrologists in many countries are already at the center of experiments and advances in this cutting-edge technology and our aim is to generalize the use of AI among nephrologists worldwide. In this paper, we provide an overview of AI from a medical perspective. We cover the core concepts of AI relevant to the practicing nephrologist in a consistent and simple way to help them get started, and we discuss the technical challenges. Finally, we focus on the KT field: the unmet needs and the potential role that AI can play to fill these gaps, then we summarize the published KT-related studies, including predictive factors used in each study, which will allow researchers to quickly focus on the most relevant issues.

Implementing predictive tools in surgery: A narrative review in the context of orthopaedic surgery

Clinical predictive tools are a topic gaining interest. Many tools are developed each year to predict various outcomes in medicine and surgery. However, the proportion of predictive tools that are implemented in clinical practice is small in comparison to the total number of tools developed. This narrative review presents key principles to guide the translation of predictive tools from academic bodies of work into useful tools that complement clinical practice. Our review identified the following principles: (1) identifying a clinical gap, (2) selecting a target user or population, (3) optimizing predictive tool performance, (4) externally validating predictive tools, (5) marketing and disseminating the tool, (6) navigating the challenges of integrating a tool into existing healthcare systems, and (7) developing an ongoing monitoring and evaluation strategy. Although the review focuses on examples in orthopaedic surgery, the principles can be applied to other disciplines in medicine and surgery.

Machine learning for risk stratification in kidney disease

PURPOSE OF REVIEW

Risk stratification for chronic kidney is becoming increasingly important as a clinical tool for both treatment and prevention measures. The goal of this review is to identify how machine learning tools contribute and facilitate risk stratification in the clinical setting.

RECENT FINDINGS

The two key machine learning paradigms to predictively stratify kidney disease risk are genomics-based and electronic health record based approaches. These methods can provide both quantitative information such as relative risk and qualitative information such as characterizing risk by subphenotype.

SUMMARY

The four key methods to stratify chronic kidney disease risk are genomics, multiomics, supervised and unsupervised machine learning methods. Polygenic risk scores utilize whole genome sequencing data to generate an individual's relative risk compared with the population. Multiomic methods integrate information from multiple biomarkers to generate trajectories and prognostic different outcomes. Supervised machine learning methods can directly utilize the growing compendia of electronic health records such as laboratory results and notes to generate direct risk predictions, while unsupervised machine learning methods can cluster individuals with chronic kidney disease into subphenotypes with differing approaches to care.

Using Artificial Intelligence for Predicting the Duration of Emergency Evacuation During Hospital Fire

OBJECTIVE

A danger threatening hospitals is fire. The most important action following a fire is to urgently evacuate the hospital during the shortest time possible. The aim of this study was to predict the duration of emergency evacuation following hospital fire using machine-learning algorithms.

METHODS

In this study, the real emergency evacuation duration of 190 patients admitted to a hospital was predicted in a simulation based on the following 8 factors: the number of hospital floors, patient preparation and transfer time, distance to the safe location, as well as patient's weight, age, sex, and movement capability. To design and validate the model, we used statistical models of machine learning, including Support Vector Machines Random Forest, Naive Bayes Classifier, and Artificial Neural Network.

RESULTS

Data analysis showed that based on the Area Under the Curve, precision, and sensitivity values of 99.5%, 92.4%, and 92.1%, respectively, the Random Forest model showed a better performance compared to other models for predicting the duration of hospital emergency evacuation during fire.

CONCLUSION

Predicting evacuation duration can provide managers with accurate information and true analyses of these events. Therefore, health policy makers and managers can promote preparedness and responsiveness during fire by predicting evacuation duration and developing appropriate plans using machine learning models.

Artificial intelligence and its impact on the domains of universal health coverage, health emergencies and health promotion: An overview of systematic reviews

•

An overview of systematic reviews on the application of AI including 129 studies.

•

AI use is prominent in Universal Health Coverage, featuring image analysis in neoplasms.

•

Half of the reviews did not evaluate validation procedures nor reporting guidelines.

•

Risk of bias was only included un a third of the reviews.

•

There is not sufficient evidence to transfer AI to actual healthcare delivery.

Background

Artificial intelligence is fueling a new revolution in medicine and in the healthcare sector. Despite the growing evidence on the benefits of artificial intelligence there are several aspects that limit the measure of its impact in people’s health. It is necessary to assess the current status on the application of AI towards the improvement of people’s health in the domains defined by WHO’s Thirteenth General Programme of Work (GPW13) and the European Programme of Work (EPW), to inform about trends, gaps, opportunities, and challenges.

Objective

To perform a systematic overview of systematic reviews on the application of artificial intelligence in the people’s health domains as defined in the GPW13 and provide a comprehensive and updated map on the application specialties of artificial intelligence in terms of methodologies, algorithms, data sources, outcomes, predictors, performance, and methodological quality.

Methods

A systematic search in MEDLINE, EMBASE, Cochrane and IEEEXplore was conducted between January 2015 and June 2021 to collect systematic reviews using a combination of keywords related to the domains of universal health coverage, health emergencies protection, and better health and wellbeing as defined by the WHO’s PGW13 and EPW. Eligibility criteria was based on methodological quality and the inclusion of practical implementation of artificial intelligence. Records were classified and labeled using ICD-11 categories into the domains of the GPW13. Descriptors related to the area of implementation, type of modeling, data entities, outcomes and implementation on care delivery were extracted using a structured form and methodological aspects of the included reviews studies was assessed using the AMSTAR checklist.

Results

The search strategy resulted in the screening of 815 systematic reviews from which 203 were assessed for eligibility and 129 were included in the review. The most predominant domain for artificial intelligence applications was Universal Health Coverage (N = 98) followed by Health Emergencies (N = 16) and Better Health and Wellbeing (N = 15). Neoplasms area on Universal Health Coverage was the disease area featuring most of the applications (21.7 %, N = 28). The reviews featured analytics primarily over both public and private data sources (67.44 %, N = 87). The most used type of data was medical imaging (31.8 %, N = 41) and predictors based on regions of interest and clinical data. The most prominent subdomain of Artificial Intelligence was Machine Learning (43.4 %, N = 56), in which Support Vector Machine method was predominant (20.9 %, N = 27). Regarding the purpose, the application of Artificial Intelligence I is focused on the prediction of the diseases (36.4 %, N = 47). With respect to the validation, more than a half of the reviews (54.3 %, N = 70) did not report a validation procedure and, whenever available, the main performance indicator was the accuracy (28.7 %, N = 37). According to the methodological quality assessment, a third of the reviews (34.9 %, N = 45) implemented methods for analysis the risk of bias and the overall AMSTAR score below was 5 (4.01 ± 1.93) on all the included systematic reviews.

Conclusion

Artificial intelligence is being used for disease modelling, diagnose, classification and prediction in the three domains of GPW13. However, the evidence is often limited to laboratory and the level of adoption is largely unbalanced between ICD-11 categoriesand diseases. Data availability is a determinant factor on the developmental stage of artificial intelligence applications. Most of the reviewed studies show a poor methodological quality and are at high risk of bias, which limits the reproducibility of the results and the reliability of translating these applications to real clinical scenarios. The analyzed papers show results only in laboratory and testing scenarios and not in clinical trials nor case studies, limiting the supporting evidence to transfer artificial intelligence to actual care delivery.

Machine Learning-Based Mortality Prediction of Patients at Risk During Hospital Admission

OBJECTIVES

The ability to predict in-hospital mortality from data available at hospital admission would identify patients at risk and thereby assist hospital-wide patient safety initiatives. Our aim was to use modern machine learning tools to predict in-hospital mortality from standardized data sets available at hospital admission.

METHODS

This was a retrospective, observational study in 3 adult tertiary care hospitals in Western Australia between January 2008 and June 2017. Primary outcome measures were the area under the curve for the receiver operating characteristics curve, the F1 score, and the average precision of the 4 machine learning algorithms used: logistic regression, neural networks, random forests, and gradient boosting trees.

RESULTS

Using our 4 predictive models, in-hospital mortality could be predicted satisfactorily (areas under the curve for neural networks, logistic regression, random forests, and gradient boosting trees: 0.932, 0.936, 0.935, and 0.935, respectively), with moderate F1 scores: 0.378, 0.367, 0.380, and 0.380, respectively. Average precision values were 0.312, 0.321, 0.334, and 0.323, respectively. It remains unknown whether additional features might improve our models; however, this would result in additional efforts for data acquisition in daily clinical practice.

CONCLUSIONS

This study demonstrates that using only a limited, standardized data set in-hospital mortality can be predicted satisfactorily at the time point of hospital admission. More parameters describing patient's health are likely needed to improve our model.

Using Information Available at the Time of Donor Offer to Predict Kidney Transplant Survival Outcomes: A Systematic Review of Prediction Models

Statistical models that can predict graft and patient survival outcomes following kidney transplantation could be of great clinical utility. We sought to appraise existing clinical prediction models for kidney transplant survival outcomes that could guide kidney donor acceptance decision-making. We searched for clinical prediction models for survival outcomes in adult recipients with single kidney-only transplants. Models that require information anticipated to become available only after the time of transplantation were excluded as, by that time, the kidney donor acceptance decision would have already been made. The outcomes of interest were all-cause and death-censored graft failure, and death. We summarised the methodological characteristics of the prediction models, predictive performance and risk of bias. We retrieved 4,026 citations from which 23 articles describing 74 models met the inclusion criteria. Discrimination was moderate for all-cause graft failure (C-statistic: 0.570–0.652; Harrell’s C: 0.580–0.660; AUC: 0.530–0.742), death-censored graft failure (C-statistic: 0.540–0.660; Harrell’s C: 0.590–0.700; AUC: 0.450–0.810) and death (C-statistic: 0.637–0.770; Harrell’s C: 0.570–0.735). Calibration was seldom reported. Risk of bias was high in 49 of the 74 models, primarily due to methods for handling missing data. The currently available prediction models using pre-transplantation information show moderate discrimination and varied calibration. Further model development is needed to improve predictions for the purpose of clinical decision-making.

Systematic Review Registration:https://osf.io/c3ehp/l.

Kidney allocation rules simulator

The greatest challenge of any kidney transplant program lies in finding enough organ donors (in number and quality) for all waitlisted transplant candidates. Unfortunately, we must resign ourselves to a manifestly insufficient supply of organs for the current demand. Furthermore we must be able to predict kidney transplant success at organ allocation if we want to minimize the number of patients who return to an already overcrowded waiting list for transplantation. Therefore, the definition of deceased donors' kidney allocation rules on transplantation must be supported by simulations that allow foreseeing, as much as possible, the consequences of these rules. Here we present the Kidney Allocation Rules Simulator (KARS) application that enables testing different kidney transplant allocation' systems with different donors and transplant candidates' datasets. In this application, it is possible to simulate allocation rules implemented in Portugal, in the United Kingdom, in countries within Eurtotransplant, and a previously suggested color priority system. As inputs, this application needs three data files: a file with transplant candidates' information, a file with candidates' anti-HLA antibodies, and a file with donors' information. As output, we will have a file with donor-recipient pairs selected according to the kidney allocation system simulated. When seeking waste reduction while ensuring a fair distribution of organs from deceased donors, the definition of rules selecting donor-recipient pairs in renal transplantation must be based on evidence supported by data. On the continuously changing process for a better distribution of an increasingly scarce resource must, we must be able to predict transplant outcomes when defining the best allocation rules.

Personalized Prediction of Kidney Function Decline and Network Analysis of the Risk Factors after Kidney Transplantation Using Nationwide Cohort Data

We developed a machine-learning-based model that could predict a decrease in one-year graft function after kidney transplantation, and investigated the risk factors of the decreased function. A total of 4317 cases were included from the Korean Organ Transplant Registry (2014−2019). An XGBoost model was trained to predict the recipient’s one-year estimated glomerular filtration rate (eGFR) below 45 mL/min/1.73 m2 using 112 pre- and peri-transplantation variables. The network of model factors was drawn using inter-factor partial correlations and the statistical significance of each factor. The model with seven features achieved an area under the curve of 0.82, sensitivity of 0.73, and specificity of 0.79. The model prediction was associated with five-year graft and rejection-free survival. Post-transplantation hospitalization >25 days and eGFR ≥ 88.0 were the prominent risk and preventive factors, respectively. Donor age and post-transplantation eGFR < 59.8 were connected to multiple risk factors on the network. Therefore, careful donor−recipient matching in older donors, and avoiding pre-transplantation risk factors, would reduce the risk of graft dysfunction. The model might improve long-term graft outcomes by supporting early detection of graft dysfunction, and proactive risk factor control.

Reducing Kidney Discard With Artificial Intelligence Decision Support: the Need for a Transdisciplinary Systems Approach

PURPOSE OF REVIEW

A transdisciplinary systems approach to the design of an artificial intelligence (AI) decision support system can more effectively address the limitations of AI systems. By incorporating stakeholder input early in the process, the final product is more likely to improve decision-making and effectively reduce kidney discard.

RECENT FINDINGS

Kidney discard is a complex problem that will require increased coordination between transplant stakeholders. An AI decision support system has significant potential, but there are challenges associated with overfitting, poor explainability, and inadequate trust. A transdisciplinary approach provides a holistic perspective that incorporates expertise from engineering, social science, and transplant healthcare. A systems approach leverages techniques for visualizing the system architecture to support solution design from multiple perspectives.

SUMMARY

Developing a systems-based approach to AI decision support involves engaging in a cycle of documenting the system architecture, identifying pain points, developing prototypes, and validating the system. Early efforts have focused on describing process issues to prioritize tasks that would benefit from AI support.

Machine learning support for decision making in kidney transplantation: step-by-step development of a technological solution (Preprint)

Background

Kidney transplantation is the preferred treatment option for patients with end-stage renal disease. To maximize patient and graft survival, the allocation of donor organs to potential recipients requires careful consideration.

Objective

This study aimed to develop an innovative technological solution to enable better prediction of kidney transplant survival for each potential donor-recipient pair.

Methods

We used deidentified data on past organ donors, recipients, and transplant outcomes in the United States from the Scientific Registry of Transplant Recipients. To predict transplant outcomes for potential donor-recipient pairs, we used several survival analysis models, including regression analysis (Cox proportional hazards), random survival forests, and several artificial neural networks (DeepSurv, DeepHit, and recurrent neural network [RNN]). We evaluated the performance of each model in terms of its ability to predict the probability of graft survival after kidney transplantation from deceased donors. Three metrics were used: the C-index, integrated Brier score, and integrated calibration index, along with calibration plots.

Results

On the basis of the C-index metrics, the neural network–based models (DeepSurv, DeepHit, and RNN) had better discriminative ability than the Cox model and random survival forest model (0.650, 0.661, and 0.659 vs 0.646 and 0.644, respectively). The proposed RNN model offered a compromise between the good discriminative ability and calibration and was implemented in a technological solution of technology readiness level 4.

Conclusions

Our technological solution based on the RNN model can effectively predict kidney transplant survival and provide support for medical professionals and candidate recipients in determining the most optimal donor-recipient pair.

Predicting Kidney Discard Using Machine Learning

BACKGROUND

Despite the kidney supply shortage, 18%-20% of deceased donor kidneys are discarded annually in the United States. In 2018, 3569 kidneys were discarded.

METHODS

We compared machine learning (ML) techniques to identify kidneys at risk of discard at the time of match run and after biopsy and machine perfusion results become available. The cohort consisted of adult deceased donor kidneys donated between December 4, 2014, and July 1, 2019. The studied ML models included Random Forests (RF), Adaptive Boosting (AdaBoost), Neural Networks (NNet), Support Vector Machines (SVM), and K-nearest Neighbors (KNN). In addition, a Logistic Regression (LR) model was fitted and used for comparison with the ML models' performance.

RESULTS

RF outperformed other ML models. Of 8036 discarded kidneys in the test dataset, LR correctly classified 3422 kidneys, whereas RF correctly classified 4762 kidneys (area under the receiver operative curve [AUC]: 0.85 versus 0.888, and balanced accuracy: 0.681 versus 0.759). For the kidneys with kidney donor profile index of >85% (6079 total), RF significantly outperformed LR in classifying discard and transplant prediction (AUC: 0.814 versus 0.717, and balanced accuracy: 0.732 versus 0.657). More than 388 kidneys were correctly classified using RF. Including biopsy and machine perfusion variables improved the performance of LR and RF (LR's AUC: 0.888 and balanced accuracy: 0.74 versus RF's AUC: 0.904 and balanced accuracy: 0.775).

CONCLUSIONS

Kidneys that are at risk of discard can be more accurately identified using ML techniques such as RF.

Predicting Kidney Graft Survival Using Machine Learning Methods: Prediction Model Development and Feature Significance Analysis Study

BACKGROUND

Kidney transplantation is the optimal treatment for patients with end-stage renal disease. Short- and long-term kidney graft survival is influenced by a number of donor and recipient factors. Predicting the success of kidney transplantation is important for optimizing kidney allocation.

OBJECTIVE

The aim of this study was to predict the risk of kidney graft failure across three temporal cohorts (within 1 year, within 5 years, and after 5 years following a transplant) based on donor and recipient characteristics. We analyzed a large data set comprising over 50,000 kidney transplants covering an approximate 20-year period.

METHODS

We applied machine learning-based classification algorithms to develop prediction models for the risk of graft failure for three different temporal cohorts. Deep learning-based autoencoders were applied for data dimensionality reduction, which improved the prediction performance. The influence of features on graft survival for each cohort was studied by investigating a new nonoverlapping patient stratification approach.

RESULTS

Our models predicted graft survival with area under the curve scores of 82% within 1 year, 69% within 5 years, and 81% within 17 years. The feature importance analysis elucidated the varying influence of clinical features on graft survival across the three different temporal cohorts.

CONCLUSIONS

In this study, we applied machine learning to develop risk prediction models for graft failure that demonstrated a high level of prediction performance. Acknowledging that these models performed better than those reported in the literature for existing risk prediction tools, future studies will focus on how best to incorporate these prediction models into clinical care algorithms to optimize the long-term health of kidney recipients.

Prediction of cancer incidence rates for the European continent using machine learning models

Cancer is one of the most important and common public health problems on Earth that can occur in many different types. Treatments and precautions are aimed at minimizing the deaths caused by cancer; however, incidence rates continue to rise. Thus, it is important to analyze and estimate incidence rates to support the determination of more effective precautions. In this research, 2018 Cancer Datasheet of World Health Organization (WHO), is used and all countries on the European Continent are considered to analyze and predict the incidence rates until 2020, for Lung cancer, Breast cancer, Colorectal cancer, Prostate cancer and All types of cancer, which have highest incidence and mortality rates. Each cancer type is trained by six machine learning models namely, Linear Regression, Support Vector Regression, Decision Tree, Long-Short Term Memory neural network, Backpropagation neural network, and Radial Basis Function neural network according to gender types separately. Linear regression and support vector regression outperformed the other models with the R2 scores 0.99 and 0.98, respectively, in initial experiments, and then used for prediction of incidence rates of the considered cancer types. The ML models estimated that the maximum rise of incidence rates would be in colorectal cancer for females by 6%.

Development and validation of a risk index to predict kidney graft survival: the kidney transplant risk index

Background

Kidney graft failure risk prediction models assist evidence-based medical decision-making in clinical practice. Our objective was to develop and validate statistical and machine learning predictive models to predict death-censored graft failure following deceased donor kidney transplant, using time-to-event (survival) data in a large national dataset from Australia.

Methods

Data included donor and recipient characteristics (n = 98) of 7,365 deceased donor transplants from January 1st, 2007 to December 31st, 2017 conducted in Australia. Seven variable selection methods were used to identify the most important independent variables included in the model. Predictive models were developed using: survival tree, random survival forest, survival support vector machine and Cox proportional regression. The models were trained using 70% of the data and validated using the rest of the data (30%). The model with best discriminatory power, assessed using concordance index (C-index) was chosen as the best model.

Results

Two models, developed using cox regression and random survival forest, had the highest C-index (0.67) in discriminating death-censored graft failure. The best fitting Cox model used seven independent variables and showed moderate level of prediction accuracy (calibration).

Conclusion

This index displays sufficient robustness to be used in pre-transplant decision making and may perform better than currently available tools.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01319-5.

Application of machine learning models for predicting acute kidney injury following donation after cardiac death liver transplantation

BACKGROUND

Acute kidney injury (AKI) is a common complication after liver transplantation (LT) and is an indicator of poor prognosis. The establishment of a more accurate preoperative prediction model of AKI could help to improve the prognosis of LT. Machine learning algorithms provide a potentially effective approach.

METHODS

A total of 493 patients with donation after cardiac death LT (DCDLT) were enrolled. AKI was defined according to the clinical practice guidelines of kidney disease: improving global outcomes (KDIGO). The clinical data of patients with AKI (AKI group) and without AKI (non-AKI group) were compared. With logistic regression analysis as a conventional model, four predictive machine learning models were developed using the following algorithms: random forest, support vector machine, classical decision tree, and conditional inference tree. The predictive power of these models was then evaluated using the area under the receiver operating characteristic curve (AUC).

RESULTS

The incidence of AKI was 35.7% (176/493) during the follow-up period. Compared with the non-AKI group, the AKI group showed a remarkably lower survival rate (P < 0.001). The random forest model demonstrated the highest prediction accuracy of 0.79 with AUC of 0.850 [95% confidence interval (CI): 0.794-0.905], which was significantly higher than the AUCs of the other machine learning algorithms and logistic regression models (P < 0.001).

CONCLUSIONS

The random forest model based on machine learning algorithms for predicting AKI occurring after DCDLT demonstrated stronger predictive power than other models in our study. This suggests that machine learning methods may provide feasible tools for forecasting AKI after DCDLT.

Application of machine learning in predicting hospital readmissions: a scoping review of the literature

Background

Advances in machine learning (ML) provide great opportunities in the prediction of hospital readmission. This review synthesizes the literature on ML methods and their performance for predicting hospital readmission in the US.

Methods

This review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews (PRISMA-ScR) Statement. The extraction of items was also guided by the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS). Electronic databases PUBMED, MEDLINE, and EMBASE were systematically searched from January 1, 2015, through December 10, 2019. The articles were imported into COVIDENCE online software for title/abstract screening and full-text eligibility. Observational studies using ML techniques for hospital readmissions among US patients were eligible for inclusion. Articles without a full text available in the English language were excluded. A qualitative synthesis included study characteristics, ML algorithms utilized, and model validation, and quantitative analysis assessed model performance. Model performances in terms of Area Under the Curve (AUC) were analyzed using R software. Quality in Prognosis Studies (QUIPS) tool was used to assess the quality of the reviewed studies.

Results

Of 522 citations reviewed, 43 studies met the inclusion criteria. A majority of the studies used electronic health records (24, 56%), followed by population-based data sources (15, 35%) and administrative claims data (4, 9%). The most common algorithms were tree-based methods (23, 53%), neural network (NN) (14, 33%), regularized logistic regression (12, 28%), and support vector machine (SVM) (10, 23%). Most of these studies (37, 85%) were of high quality. A majority of these studies (28, 65%) reported ML algorithms with an AUC above 0.70. There was a range of variability within AUC reported by these studies with a median of 0.68 (IQR: 0.64–0.76; range: 0.50–0.90).

Conclusions

The ML algorithms involving tree-based methods, NN, regularized logistic regression, and SVM are commonly used to predict hospital readmission in the US. Further research is needed to compare the performance of ML algorithms for hospital readmission prediction.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01284-z.

Texture analysis to differentiate anterior cruciate ligament in patients after surgery with platelet-rich plasma

BACKGROUND

Platelet-rich plasma (PRP) has been used to favor anterior cruciate ligament (ACL) healing after reconstruction surgeries. However, clinical data are still inconclusive and subjective about PRP. Thus, we propose a quantitative method to demonstrate that PRP produced morphological structure changes.

METHODS

Thirty-four patients undergoing ACL reconstruction surgery were evaluated and divided into control group (sixteen patients) without PRP application and experiment group (eighteen patients) with intraoperative application of PRP. Magnetic resonance imaging (MRI) scans were performed 3 months after surgery. We used Matlab® and machine learning (ML) in Orange Canvas® to texture analysis (TA) features extraction. Experienced radiologists delimited the regions of interest (RoIs) in the T2-weighted images. Sixty-two texture parameters were extracted, including gray-level co-occurrence matrix and gray level run length. We used the algorithms logistic regression (LR), naive Bayes (NB), and stochastic gradient descent (SGD).

RESULTS

The accuracy of the classification with NB, LR, and SGD was 83.3%, 75%, 75%, respectively. For the area under the curve, NB, LR, and SGD presented values of 91.7%, 94.4%, 75%, respectively. In clinical evaluations, the groups show similar responses in terms of improvement in pain and increase in the IKDC index (International Knee Documentation Committee) and Lysholm score indices differing only in the assessment of flexion, which presents a significant difference for the group treated with PRP.

CONCLUSIONS

Here, we demonstrated quantitatively that patients who received PRP presented texture changes when compared to the control group. Thus, our findings suggest that PRP interferes with morphological parameters of the ACL.

TRIAL REGISTRATION

Protocol no. CAAE 56164316.6.0000.5411.

Predicting breast cancer 5-year survival using machine learning: A systematic review

BACKGROUND

Accurately predicting the survival rate of breast cancer patients is a major issue for cancer researchers. Machine learning (ML) has attracted much attention with the hope that it could provide accurate results, but its modeling methods and prediction performance remain controversial. The aim of this systematic review is to identify and critically appraise current studies regarding the application of ML in predicting the 5-year survival rate of breast cancer.

METHODS

In accordance with the PRISMA guidelines, two researchers independently searched the PubMed (including MEDLINE), Embase, and Web of Science Core databases from inception to November 30, 2020. The search terms included breast neoplasms, survival, machine learning, and specific algorithm names. The included studies related to the use of ML to build a breast cancer survival prediction model and model performance that can be measured with the value of said verification results. The excluded studies in which the modeling process were not explained clearly and had incomplete information. The extracted information included literature information, database information, data preparation and modeling process information, model construction and performance evaluation information, and candidate predictor information.

RESULTS

Thirty-one studies that met the inclusion criteria were included, most of which were published after 2013. The most frequently used ML methods were decision trees (19 studies, 61.3%), artificial neural networks (18 studies, 58.1%), support vector machines (16 studies, 51.6%), and ensemble learning (10 studies, 32.3%). The median sample size was 37256 (range 200 to 659820) patients, and the median predictor was 16 (range 3 to 625). The accuracy of 29 studies ranged from 0.510 to 0.971. The sensitivity of 25 studies ranged from 0.037 to 1. The specificity of 24 studies ranged from 0.008 to 0.993. The AUC of 20 studies ranged from 0.500 to 0.972. The precision of 6 studies ranged from 0.549 to 1. All of the models were internally validated, and only one was externally validated.

CONCLUSIONS

Overall, compared with traditional statistical methods, the performance of ML models does not necessarily show any improvement, and this area of research still faces limitations related to a lack of data preprocessing steps, the excessive differences of sample feature selection, and issues related to validation. Further optimization of the performance of the proposed model is also needed in the future, which requires more standardization and subsequent validation.

Development and validation of routine clinical laboratory data derived marker-based nomograms for the prediction of 5-year graft survival in kidney transplant recipients

Background: To develop and validate predictive nomograms for 5-year graft survival in kidney transplant recipients (KTRs) with easily-available laboratory data derived markers and clinical variables within the first year post-transplant.

Methods: The clinical and routine laboratory data from within the first year post-transplant of 1289 KTRs was collected to generate candidate predictors. Univariate and multivariate Cox analyses and LASSO were conducted to select final predictors. X-tile analysis was applied to identify optimal cutoff values to transform potential continuous factors into category variables and stratify patients. C-index, calibration curve, dynamic time-dependent AUC, decision curve analysis, and Kaplan-Meier curves were used to evaluate models’ predictive accuracy and clinical utility.

Results: Two predictive nomograms were constructed by using 0–6- and 0–12- month laboratory data, and showed good predictive performance with C-indexes of 0.78 and 0.85, respectively, in the training cohort. Calibration curves showed that the prediction probabilities of 5-year graft survival were in concordance with actual observations. Additionally, KTRs could be successfully stratified into three risk groups by nomograms.

Conclusions: These predictive nomograms combining demographic and 0–6- or 0–12- month markers derived from post-transplant laboratory data could serve as useful tools for early identification of 5-year graft survival probability in individual KTRs.

Machine learning-based prediction of health outcomes in pediatric organ transplantation recipients

OBJECTIVES

Prediction of post-transplant health outcomes and identification of key factors remain important issues for pediatric transplant teams and researchers. Outcomes research has generally relied on general linear modeling or similar techniques offering limited predictive validity. Thus far, data-driven modeling and machine learning (ML) approaches have had limited application and success in pediatric transplant outcomes research. The purpose of the current study was to examine ML models predicting post-transplant hospitalization in a sample of pediatric kidney, liver, and heart transplant recipients from a large solid organ transplant program.

MATERIALS AND METHODS

Various logistic regression, naive Bayes, support vector machine, and deep learning (DL) methods were used to predict 1-, 3-, and 5-year post-transplant hospitalization using patient and administrative data from a large pediatric organ transplant center.

RESULTS

DL models generally outperformed traditional ML models across organtypes and prediction windows with area under the receiver operating characteristic curve values ranging from 0.750 to 0.851. Shapley additive explanations (SHAP) were used to increase the interpretability of DL model results. Various medical, patient, and social variables were identified as salient predictors across organ types.

DISCUSSION

Results demonstrate the utility of DL modeling for health outcome prediction with pediatric patients, and its use represents an important development in the prediction of post-transplant outcomes in pediatric transplantation compared to prior research.

CONCLUSION

Results point to DL models as potentially useful tools in decision-support systems assisting physicians and transplant teams in identifying patients at a greater risk for poor post-transplant outcomes.

Evaluation of hepatic steatosis before liver transplantation in ex vivo by volumetric quantitative PDFF-MRI

PURPOSE

Over the last two decades, extended criteria have promoted an increased number of donor livers available for liver transplantation. But posttransplant graft loss is still a major concern. Macrovesicular hepatic steatosis (MHS) is recognized as the most significant prognostic histologic parameter in predicting posttransplant graft loss. We aimed to evaluate the utility of ex vivo volumetric quantitative MRI for quantifying MHS before liver transplantation using proton density fat-fraction (PDFF-MRI) histogram analysis.

METHODS

PDFF-MRI was performed at 3.0T in 40 livers. We obtained histogram parameters of whole-liver volume of interest, including the mean, median, 5th, 10th, 25th, 75th, 90th, and 95th percentile PDFF; skewness; kurtosis; entropy; and volume.

RESULTS

Livers from 40 cadaveric donors were included, and histologic ex vivo fat quantification was available for 33 livers. Ten livers had MHS and 23 had normal fat content. The MHS group had higher mean, median, 5th, 10th, 25th, 75th, 90th, and 95th percentile PDFF, and entropy than the group with normal fat content (P < .05). Median PDFF had greater area under the curve value than other parameters. Mean PDFF showed an excellent correlation with entropy and a moderate correlation with MHS quantification on histology.

CONCLUSIONS

Ex vivo volumetric quantitative PDFF-MRI histogram analysis is a very useful and noninvasive method to detect MHS before liver transplantation. Median PDFF was the best predictor of the presence of MHS. Entropy is a very promising parameter.

Prognostic role of artificial intelligence among patients with hepatocellular cancer: A systematic review

BACKGROUND

Prediction of survival after the treatment of hepatocellular carcinoma (HCC) has been widely investigated, yet remains inadequate. The application of artificial intelligence (AI) is emerging as a valid adjunct to traditional statistics due to the ability to process vast amounts of data and find hidden interconnections between variables. AI and deep learning are increasingly employed in several topics of liver cancer research, including diagnosis, pathology, and prognosis.

AIM

To assess the role of AI in the prediction of survival following HCC treatment.

METHODS

A web-based literature search was performed according to the Preferred Reporting Items for Systemic Reviews and Meta-Analysis guidelines using the keywords “artificial intelligence”, “deep learning” and “hepatocellular carcinoma” (and synonyms). The specific research question was formulated following the patient (patients with HCC), intervention (evaluation of HCC treatment using AI), comparison (evaluation without using AI), and outcome (patient death and/or tumor recurrence) structure. English language articles were retrieved, screened, and reviewed by the authors. The quality of the papers was assessed using the Risk of Bias In Non-randomized Studies of Interventions tool. Data were extracted and collected in a database.

RESULTS

Among the 598 articles screened, nine papers met the inclusion criteria, six of which had low-risk rates of bias. Eight articles were published in the last decade; all came from eastern countries. Patient sample size was extremely heterogenous (n = 11-22926). AI methodologies employed included artificial neural networks (ANN) in six studies, as well as support vector machine, artificial plant optimization, and peritumoral radiomics in the remaining three studies. All the studies testing the role of ANN compared the performance of ANN with traditional statistics. Training cohorts were used to train the neural networks that were then applied to validation cohorts. In all cases, the AI models demonstrated superior predictive performance compared with traditional statistics with significantly improved areas under the curve.

CONCLUSION

AI applied to survival prediction after HCC treatment provided enhanced accuracy compared with conventional linear systems of analysis. Improved transferability and reproducibility will facilitate the widespread use of AI methodologies.

Machine learning for predicting long-term kidney allograft survival: a scoping review

Supervised machine learning (ML) is a class of algorithms that "learn" from existing input-output pairs, which is gaining popularity in pattern recognition for classification and prediction problems. In this scoping review, we examined the use of supervised ML algorithms for the prediction of long-term allograft survival in kidney transplant recipients. Data sources included PubMed, the Cumulative Index to Nursing and Allied Health Literature, and the Institute for Electrical and Electronics Engineers (IEEE) Xplore libraries from inception to November 2019. We screened titles and abstracts and potentially eligible full-text reports to select studies and subsequently abstracted the data. Eleven studies were identified. Decision trees were the most commonly used method (n = 8), followed by artificial neural networks (ANN) (n = 4) and Bayesian belief networks (n = 2). The area under receiver operating curve (AUC) was the most common measure of discrimination (n = 7), followed by sensitivity (n = 5) and specificity (n = 4). Model calibration examining the reliability in risk prediction was performed using either the Pearson r or the Hosmer-Lemeshow test in four studies. One study showed that logistic regression had comparable performance to ANN, while another study demonstrated that ANN performed better in terms of sensitivity, specificity, and accuracy, as compared with a Cox proportional hazards model. We synthesized the evidence related to the comparison of ML techniques with traditional statistical approaches for prediction of long-term allograft survival in patients with a kidney transplant. The methodological and reporting quality of included studies was poor. Our study also demonstrated mixed results in terms of the predictive potential of the models.

Machine learning methods in organ transplantation

PURPOSE OF REVIEW

Machine learning techniques play an important role in organ transplantation. Analysing the main tasks for which they are being applied, together with the advantages and disadvantages of their use, can be of crucial interest for clinical practitioners.

RECENT FINDINGS

In the last 10 years, there has been an explosion of interest in the application of machine-learning techniques to organ transplantation. Several approaches have been proposed in the literature aiming to find universal models by considering multicenter cohorts or from different countries. Moreover, recently, deep learning has also been applied demonstrating a notable ability when dealing with a vast amount of information.

SUMMARY

Organ transplantation can benefit from machine learning in such a way to improve the current procedures for donor--recipient matching or to improve standard scores. However, a correct preprocessing is needed to provide consistent and high quality databases for machine-learning algorithms, aiming to robust and fair approaches to support expert decision-making systems.

Using Artificial Intelligence for Predicting Survival of Individual Grafts in Liver Transplantation: A Systematic Review

The demand for liver transplantation far outstrips the supply of deceased donor organs, and so, listing and allocation decisions aim to maximize utility. Most existing methods for predicting transplant outcomes use basic methods, such as regression modeling, but newer artificial intelligence (AI) techniques have the potential to improve predictive accuracy. The aim was to perform a systematic review of studies predicting graft outcomes following deceased donor liver transplantation using AI techniques and to compare these findings to linear regression and standard predictive modeling: donor risk index (DRI), Model for End-Stage Liver Disease (MELD), and Survival Outcome Following Liver Transplantation (SOFT). After reviewing available article databases, a total of 52 articles were reviewed for inclusion. Of these articles, 9 met the inclusion criteria, which reported outcomes from 18,771 liver transplants. Artificial neural networks (ANNs) were the most commonly used methodology, being reported in 7 studies. Only 2 studies directly compared machine learning (ML) techniques to liver scoring modalities (i.e., DRI, SOFT, and balance of risk [BAR]). Both studies showed better prediction of individual organ survival with the optimal ANN model, reporting an area under the receiver operating characteristic curve (AUROC) 0.82 compared with BAR (0.62) and SOFT (0.57), and the other ANN model gave an AUC ROC of 0.84 compared with a DRI (0.68) and SOFT (0.64). AI techniques can provide high accuracy in predicting graft survival based on donors and recipient variables. When compared with the standard techniques, AI methods are dynamic and are able to be trained and validated within every population. However, the high accuracy of AI may come at a cost of losing explainability (to patients and clinicians) on how the technology works.

A systematic review of machine learning models for predicting outcomes of stroke with structured data

Background and purpose

Machine learning (ML) has attracted much attention with the hope that it could make use of large, routinely collected datasets and deliver accurate personalised prognosis. The aim of this systematic review is to identify and critically appraise the reporting and developing of ML models for predicting outcomes after stroke.

Methods

We searched PubMed and Web of Science from 1990 to March 2019, using previously published search filters for stroke, ML, and prediction models. We focused on structured clinical data, excluding image and text analysis. This review was registered with PROSPERO (CRD42019127154).

Results

Eighteen studies were eligible for inclusion. Most studies reported less than half of the terms in the reporting quality checklist. The most frequently predicted stroke outcomes were mortality (7 studies) and functional outcome (5 studies). The most commonly used ML methods were random forests (9 studies), support vector machines (8 studies), decision trees (6 studies), and neural networks (6 studies). The median sample size was 475 (range 70–3184), with a median of 22 predictors (range 4–152) considered. All studies evaluated discrimination with thirteen using area under the ROC curve whilst calibration was assessed in three. Two studies performed external validation. None described the final model sufficiently well to reproduce it.

Conclusions

The use of ML for predicting stroke outcomes is increasing. However, few met basic reporting standards for clinical prediction tools and none made their models available in a way which could be used or evaluated. Major improvements in ML study conduct and reporting are needed before it can meaningfully be considered for practice.

Variable Responses to Corneal Grafts: Insights from Immunology and Systems Biology

Corneal grafts interact with their hosts via complex immunobiological processes that sometimes lead to graft failure. Prediction of graft failure is often a tedious task due to the genetic and nongenetic heterogeneity of patients. As in other areas of medicine, a reliable prediction method would impact therapeutic decision-making in corneal transplantation. Valuable insights into the clinically observed heterogeneity of host responses to corneal grafts have emerged from multidisciplinary approaches, including genomics analyses, mechanical studies, immunobiology, and theoretical modeling. Here, we review the emerging concepts, tools, and new biomarkers that may allow for the prediction of graft survival.