Predictive models for hospital readmission risk: A systematic review of methods

Practical guide to building machine learning-based clinical prediction models using imbalanced datasets

Clinical prediction models often aim to predict rare, high-risk events, but building such models requires robust understanding of imbalance datasets and their unique study design considerations. This practical guide highlights foundational prediction model principles for surgeon-data scientists and readers who encounter clinical prediction models, from feature engineering and algorithm selection strategies to model evaluation and design techniques specific to imbalanced datasets. We walk through a clinical example using readable code to highlight important considerations and common pitfalls in developing machine learning-based prediction models. We hope this practical guide facilitates developing and critically appraising robust clinical prediction models for the surgical community.

Evaluating Algorithmic Bias in 30-Day Hospital Readmission Models: Retrospective Analysis

BACKGROUND

The adoption of predictive algorithms in health care comes with the potential for algorithmic bias, which could exacerbate existing disparities. Fairness metrics have been proposed to measure algorithmic bias, but their application to real-world tasks is limited.

OBJECTIVE

This study aims to evaluate the algorithmic bias associated with the application of common 30-day hospital readmission models and assess the usefulness and interpretability of selected fairness metrics.

METHODS

We used 10.6 million adult inpatient discharges from Maryland and Florida from 2016 to 2019 in this retrospective study. Models predicting 30-day hospital readmissions were evaluated: LACE Index, modified HOSPITAL score, and modified Centers for Medicare & Medicaid Services (CMS) readmission measure, which were applied as-is (using existing coefficients) and retrained (recalibrated with 50% of the data). Predictive performances and bias measures were evaluated for all, between Black and White populations, and between low- and other-income groups. Bias measures included the parity of false negative rate (FNR), false positive rate (FPR), 0-1 loss, and generalized entropy index. Racial bias represented by FNR and FPR differences was stratified to explore shifts in algorithmic bias in different populations.

RESULTS

The retrained CMS model demonstrated the best predictive performance (area under the curve: 0.74 in Maryland and 0.68-0.70 in Florida), and the modified HOSPITAL score demonstrated the best calibration (Brier score: 0.16-0.19 in Maryland and 0.19-0.21 in Florida). Calibration was better in White (compared to Black) populations and other-income (compared to low-income) groups, and the area under the curve was higher or similar in the Black (compared to White) populations. The retrained CMS and modified HOSPITAL score had the lowest racial and income bias in Maryland. In Florida, both of these models overall had the lowest income bias and the modified HOSPITAL score showed the lowest racial bias. In both states, the White and higher-income populations showed a higher FNR, while the Black and low-income populations resulted in a higher FPR and a higher 0-1 loss. When stratified by hospital and population composition, these models demonstrated heterogeneous algorithmic bias in different contexts and populations.

CONCLUSIONS

Caution must be taken when interpreting fairness measures' face value. A higher FNR or FPR could potentially reflect missed opportunities or wasted resources, but these measures could also reflect health care use patterns and gaps in care. Simply relying on the statistical notions of bias could obscure or underplay the causes of health disparity. The imperfect health data, analytic frameworks, and the underlying health systems must be carefully considered. Fairness measures can serve as a useful routine assessment to detect disparate model performances but are insufficient to inform mechanisms or policy changes. However, such an assessment is an important first step toward data-driven improvement to address existing health disparities.

Multimodal Machine Learning for Prediction of 30-day Readmission Risk in Elderly Population

BACKGROUND

Readmission within 30 days is a prevalent issue among elderly patients, linked to unfavorable health outcomes. Our objective was to develop and validate multimodal machine learning models for predicting 30-day readmission risk in elderly patients discharged from internal medicine departments.

METHODS

This was a retrospective cohort study which included elderly patients aged 75 or older, who were hospitalized at the Hadassah Medical Center internal medicine departments between 2014 and 2020. Three machine learning algorithms were developed and employed to predict 30-day readmission risk. The primary measures were predictive model performance scores, specifically area under the receiver operator curve (AUROC), and average precision.

RESULTS

This study included 19,569 admissions. Of them, 3,258 (16.65%) resulted in 30-day readmission. Our three proposed models demonstrated high accuracy and precision on an unseen test set, with AUROC values of 0.87, 0.89, and 0.93, respectively, and average precision values of 0.76, 0.78, and 0.81. Feature importance analysis revealed that the number of admissions in the past year, history of 30-day readmission, Charlson score, and admission length were the most influential variables. Notably, the natural language processing score, representing the probability of readmission according to a textual-based model trained on social workers assessment letters during hospitalization, ranked among the top 10 contributing factors.

CONCLUSIONS

Leveraging multimodal machine learning offers a promising strategy for identifying elderly patients who are at high risk for 30-day readmission. By identifying these patients, machine learning models may facilitate the effective execution of preventive actions to reduce avoidable readmission incidents.

Machine learning algorithms to predict outcomes in children and adolescents with COVID-19: A systematic review

BACKGROUND AND OBJECTIVES

We aimed to analyze the study designs, modeling approaches, and performance evaluation metrics in studies using machine learning techniques to develop clinical prediction models for children and adolescents with COVID-19.

METHODS

We searched four databases for articles published between 01/01/2020 and 10/25/2023, describing the development of multivariable prediction models using any machine learning technique for predicting several outcomes in children and adolescents who had COVID-19.

RESULTS

We included ten articles, six (60 % [95 % confidence interval (CI) 0.31 - 0.83]) were predictive diagnostic models and four (40% [95 % CI 0.170.69]) were prognostic models. All models were developed to predict a binary outcome (n= 10/10, 100 % [95 % CI 0.72-1]). The most frequently predicted outcome was disease detection (n=3/10, 30% [95 % CI 0.11-0.60]). The most commonly used machine learning models in the studies were tree-based (n=12/33, 36.3% [95 % CI 0.17-0.47]) and neural networks (n=9/27, 33.2% [95% CI 0.15-0.44]).

CONCLUSION

Our review revealed that attention is required to address problems including small sample sizes, inconsistent reporting practices on data preparation, biases in data sources, lack of reporting metrics such as calibration and discrimination, hyperparameters and other aspects that allow reproducibility by other researchers and might improve the methodology.

A machine learning model to predict heart failure readmission: toward optimal feature set

Background

Hospital readmissions for heart failure patients remain high despite efforts to reduce them. Predictive modeling using big data provides opportunities to identify high-risk patients and inform care management. However, large datasets can constrain performance.

Objective

This study aimed to develop a machine learning based prediction model leveraging a nationwide hospitalization database to predict 30-day heart failure readmissions. Another objective of this study is to find the optimal feature set that leads to the highest AUC value in the prediction model.

Material and methods

Heart failure patient data was extracted from the 2020 Nationwide Readmissions Database. A heuristic feature selection process incrementally incorporated predictors into logistic regression and random forest models, which yields a maximum increase in the AUC metric. Discrimination was evaluated through accuracy, sensitivity, specificity and AUC.

Results

A total of 566,019 discharges with heart failure diagnosis were recognized. Readmission rate was 8.9% for same-cause and 20.6% for all-cause diagnoses. Random forest outperformed logistic regression, achieving AUCs of 0.607 and 0.576 for same-cause and all-cause readmissions respectively. Heuristic feature selection resulted in the identification of optimal feature sets including 20 and 22 variables from a pool of 30 and 31 features for the same-cause and all-cause datasets. Key predictors included age, payment method, chronic kidney disease, disposition status, number of ICD-10-CM diagnoses, and post-care encounters.

Conclusion

The proposed model attained discrimination comparable to prior analyses that used smaller datasets. However, reducing the sample enhanced performance, indicating big data complexity. Improved techniques like heuristic feature selection enabled effective leveraging of the nationwide data. This study provides meaningful insights into predictive modeling methodologies and influential features for forecasting heart failure readmissions.

Clinical Predictive Modeling of Heart Failure: Domain Description, Models' Characteristics and Literature Review

This study attempts to identify and briefly describe the current directions in applied and theoretical clinical prediction research. Context-rich chronic heart failure syndrome (CHFS) telemedicine provides the medical foundation for this effort. In the chronic stage of heart failure, there are sudden exacerbations of syndromes with subsequent hospitalizations, which are called acute decompensation of heart failure (ADHF). These decompensations are the subject of diagnostic and prognostic predictions. The primary purpose of ADHF predictions is to clarify the current and future health status of patients and subsequently optimize therapeutic responses. We proposed a simplified discrete-state disease model as an attempt at a typical summarization of a medical subject before starting predictive modeling. The study tries also to structure the essential common characteristics of quantitative models in order to understand the issue in an application context. The last part provides an overview of prediction works in the field of CHFS. These three parts provide the reader with a comprehensive view of quantitative clinical predictive modeling in heart failure telemedicine with an emphasis on several key general aspects. The target community is medical researchers seeking to align their clinical studies with prognostic or diagnostic predictive modeling, as well as other predictive researchers. The study was written by a non-medical expert.

Advancing prognostic precision in pulmonary embolism: A clinical and laboratory-based artificial intelligence approach for enhanced early mortality risk stratification

BACKGROUND

Acute pulmonary embolism (PE) is a critical medical emergency that necessitates prompt identification and intervention. Accurate prognostication of early mortality is vital for recognizing patients at elevated risk for unfavourable outcomes and administering suitable therapy. Machine learning (ML) algorithms hold promise for enhancing the precision of early mortality prediction in PE patients.

OBJECTIVE

To devise an ML algorithm for early mortality prediction in PE patients by employing clinical and laboratory variables.

METHODS

This study utilized diverse oversampling techniques to improve the performance of various machine learning models including ANN, SVM, DT, RF, and AdaBoost for early mortality prediction. Appropriate oversampling methods were chosen for each model based on algorithm characteristics and dataset properties. Predictor variables included four lab tests, eight physiological time series indicators, and two general descriptors. Evaluation used metrics like accuracy, F1_score, precision, recall, Area Under the Curve (AUC) and Receiver Operating Characteristic (ROC) curves, providing a comprehensive view of models' predictive abilities.

RESULTS

The findings indicated that the RF model with random oversampling exhibited superior performance among the five models assessed, achieving elevated accuracy and precision alongside high recall for predicting the death class. The oversampling approaches effectively equalized the sample distribution among the classes and enhanced the models' performance.

CONCLUSIONS

The suggested ML technique can efficiently prognosticate mortality in patients afflicted with acute PE. The RF model with random oversampling can aid healthcare professionals in making well-informed decisions regarding the treatment of patients with acute PE. The study underscores the significance of oversampling methods in managing imbalanced data and emphasizes the potential of ML algorithms in refining early mortality prediction for PE patients.

Early prediction of in-hospital mortality utilizing multivariate predictive modelling of electronic medical records and socio-determinants of health of the first day of hospitalization

BACKGROUND

In France an average of 4% of hospitalized patients die during their hospital stay. To aid medical decision making and the attribution of resources, within a few days of admission the identification of patients at high risk of dying in hospital is essential.

METHODS

We used de-identified routine patient data available in the first 2 days of hospitalization in a French University Hospital (between 2016 and 2018) to build models predicting in-hospital mortality (at ≥ 2 and ≤ 30 days after admission). We tested nine different machine learning algorithms with repeated 10-fold cross-validation. Models were trained with 283 variables including age, sex, socio-determinants of health, laboratory test results, procedures (Classification of Medical Acts), medications (Anatomical Therapeutic Chemical code), hospital department/unit and home address (urban, rural etc.). The models were evaluated using various performance metrics. The dataset contained 123,729 admissions, of which the outcome for 3542 was all-cause in-hospital mortality and 120,187 admissions (no death reported within 30 days) were controls.

RESULTS

The support vector machine, logistic regression and Xgboost algorithms demonstrated high discrimination with a balanced accuracy of 0.81 (95%CI 0.80-0.82), 0.82 (95%CI 0.80-0.83) and 0.83 (95%CI 0.80-0.83) and AUC of 0.90 (95%CI 0.88-0.91), 0.90 (95%CI 0.89-0.91) and 0.90 (95%CI 0.89-0.91) respectively. The most predictive variables for in-hospital mortality in all three models were older age (greater risk), and admission with a confirmed appointment (reduced risk).

CONCLUSION

We propose three highly discriminating machine-learning models that could improve clinical and organizational decision making for adult patients at hospital admission.

A call for better validation of opioid overdose risk algorithms

Clinical decision support (CDS) systems powered by predictive models have the potential to improve the accuracy and efficiency of clinical decision-making. However, without sufficient validation, these systems have the potential to mislead clinicians and harm patients. This is especially true for CDS systems used by opioid prescribers and dispensers, where a flawed prediction can directly harm patients. To prevent these harms, regulators and researchers have proposed guidance for validating predictive models and CDS systems. However, this guidance is not universally followed and is not required by law. We call on CDS developers, deployers, and users to hold these systems to higher standards of clinical and technical validation. We provide a case study on two CDS systems deployed on a national scale in the United States for predicting a patient's risk of adverse opioid-related events: the Stratification Tool for Opioid Risk Mitigation (STORM), used by the Veterans Health Administration, and NarxCare, a commercial system.

Identification of Factors Associated With 30-day Readmissions After Posterior Lumbar Fusion Using Machine Learning and Traditional Models: A National Longitudinal Database Study

STUDY DESIGN

A retrospective cohort study.

OBJECTIVE

To identify the factors associated with readmissions after PLF using machine learning and logistic regression (LR) models.

SUMMARY OF BACKGROUND DATA

Readmissions after posterior lumbar fusion (PLF) place significant health and financial burden on the patient and overall health care system.

MATERIALS AND METHODS

The Optum Clinformatics Data Mart database was used to identify patients who underwent posterior lumbar laminectomy, fusion, and instrumentation between 2004 and 2017. Four machine learning models and a multivariable LR model were used to assess factors most closely associated with 30-day readmission. These models were also evaluated in terms of ability to predict unplanned 30-day readmissions. The top-performing model (Gradient Boosting Machine; GBM) was then compared with the validated LACE index in terms of potential cost savings associated with the implementation of the model.

RESULTS

A total of 18,981 patients were included, of which 3080 (16.2%) were readmitted within 30 days of initial admission. Discharge status, prior admission, and geographic division were most influential for the LR model, whereas discharge status, length of stay, and prior admissions had the greatest relevance for the GBM model. GBM outperformed LR in predicting unplanned 30-day readmission (mean area under the receiver operating characteristic curve 0.865 vs. 0.850, P <0.0001). The use of GBM also achieved a projected 80% decrease in readmission-associated costs relative to those achieved by the LACE index model.

CONCLUSIONS

The factors associated with readmission vary in terms of predictive influence based on standard LR and machine learning models used, highlighting the complementary roles these models have in identifying relevant factors for the prediction of 30-day readmissions. For PLF procedures, GBM yielded the greatest predictive ability and associated cost savings for readmission.

LEVEL OF EVIDENCE

3.

Performance of advanced machine learning algorithms overlogistic regression in predicting hospital readmissions: A meta-analysis

Objectives

Machine learning algorithms are being increasingly used for predicting hospital readmissions. This meta-analysis evaluated the performance of logistic regression (LR) and machine learning (ML) models for the prediction of 30-day hospital readmission among patients in the US.

Methods

Electronic databases (i.e., Medline, PubMed, and Embase) were searched from January 2015 to December 2019. Only studies in the English language were included. Two reviewers performed studies screening, quality appraisal, and data collection. The quality of the studies was assessed using the Quality in Prognosis Studies (QUIPS) tool. Model performance was evaluated using the Area Under the Curve (AUC). A random-effects meta-analysis was performed using STATA 16.

Results

Nine studies were included based on the selection criteria. The most common ML techniques were tree-based methods such as boosting and random forest. Most of the studies had a low risk of bias (8/9). The AUC was greater with ML to predict 30-day all-cause hospital readmission compared with LR [Mean Difference (MD): 0.03; 95% Confidence Interval (CI) 0.01-0.05]. Subgroup analyses found that deep-learning methods had a better performance compared with LR (MD 0.06; 95% CI, 0.04-0.09), followed by neural networks (MD: 0.03; 95% CI, 0.03-0.03), while the AUCs of the tree-based (MD: 0.02; 95% CI -0.00-0.04) and kernel-based (MD: 0.02; 95% CI 0.02 (-0.13-0.16) methods were no different compared to LR. More than half of the studies evaluated heart failure-related rehospitalization (N = 5). For the readmission prediction among heart failure patients, ML performed better compared with LR, with a mean difference in AUC of 0.04 (95% CI, 0.01-0.07). The leave-one-out sensitivity analysis confirmed the robustness of the findings.

Conclusion

Multiple ML methods were used to predict 30-day all-cause hospital readmission. Performance varied across the ML methods, with deep-learning methods showing the best performance over the LR.

Epidemiological Determinants of Patient Non-Conveyance to the Hospital in an Emergency Medical Service Environment

BACKGROUND

The increasing prevalence of comorbidities worldwide has spurred the need for time-effective pre-hospital emergency medical services (EMS). Some pre-hospital emergency calls requesting EMS result in patient non-conveyance. Decisions for non-conveyance are sometimes driven by the patient or the clinician, which may jeopardize the patients' healthcare outcomes. This study aimed to explore the distribution and determinants of patient non-conveyance to hospitals in a Middle Eastern national Ambulance Service that promotes the transportation of all emergency call patients and does not adopt clinician-based non-conveyance decision.

METHODS

Using R Language, descriptive, bivariate, and binary logistic regression analyses were conducted for 334,392 multi-national patient non-conveyance emergency calls from June 2018 to July 2022, from a total of 1,030,228 calls to which a response unit was dispatched.

RESULTS

After data pre-processing, 237,862 cases of patient non-conveyance to hospital were retained, with a monthly average of 41.96% (n = 8799) of the emergency service demands and a standard deviation of 5.49% (n = 2040.63). They predominantly involved South Asians (29.36%, n = 69,849); 64.50% (n = 153,427) were of the age category from 14 to 44 years; 61.22% (n = 145,610) were male; 74.59% (n = 177,424) from the urban setting; and 71.28% (n = 169,552) had received on-scene treatment. Binary logistic regression with full variables and backward methods identified the final models of the determinants of patient non-conveyance decisions with an Akaike information criterion prediction estimator, respectively, of (250,200) and (250,169), indicating no significant difference between both models (Chi-square test; p-value = 0.63).

CONCLUSIONS

Despite exercising a cautious protocol by encouraging patient transportation to hospital, patient non-conveyance seems to be a problem in the healthcare system that strains the pre-hospital medical response teams' resources. Policies and regulations should be adopted to encourage individuals to access other primary care centers when required rather than draining emergency services for non-emergency situations.

Implementation of Prediction Models in the Emergency Department from an Implementation Science Perspective-Determinants, Outcomes, and Real-World Impact: A Scoping Review

STUDY OBJECTIVE

Prediction models offer a promising form of clinical decision support in the complex and fast-paced environment of the emergency department (ED). Despite significant advancements in model development and validation, implementation of such models in routine clinical practice remains elusive. This scoping review aims to survey the current state of prediction model implementation in the ED and to provide insights on contributing factors and outcomes from an implementation science perspective.

METHODS

We searched 4 databases from their inception to May 20, 2022: MEDLINE (through PubMed), Embase, Scopus, and CINAHL. Articles that reported implementation outcomes and/or contextual determinants under the Reach, Effectiveness, Adoption, Implementation Maintenance (RE-AIM)/Practical, Robust, Implementation, and Sustainability Model (PRISM) framework were included. Characteristics of studies, models, and results of the RE-AIM/PRISM domains were summarized narratively.

RESULTS

Thirty-six reports on 31 implementations were included. The most common prediction models implemented were early warning scores. The most common implementation strategies used were training stakeholders, infrastructural changes, and using evaluative or iterative strategies. Only one report examined ED patients' perspectives, whereas the rest were focused on the experience of health care workers or organizational stakeholders. Key determinants of successful implementation include strong stakeholder engagement, codevelopment of workflows and implementation strategies, education, and usability.

CONCLUSION

Examining ED prediction models from an implementation science perspective can provide valuable insights and help guide future implementations.

Leveraging deep survival models to predict quality of care risk in diverse hospital readmissions

Hospital readmissions rate is reportedly high and has caused huge financial burden on health care systems in many countries. It is viewed as an important indicator of health care providers' quality of care. We examine the use of machine learning-based survival analysis to assess quality of care risk in hospital readmissions. This study applies various survival models to explore the risk of hospital readmissions given patient demographics and their respective hospital discharges extracted from a health care claims dataset. We explore advanced feature representation techniques such as BioBERT and Node2Vec to encode high-dimensional diagnosis code features. To our knowledge, this study is the first to apply deep-learning based survival-analysis models for predicting hospital readmission risk agnostic of specific medical conditions and a fixed window for readmission. We found that modeling the time from discharge date to readmission date as a Weibull distribution as in the SparseDeepWeiSurv model yields the best discriminative power and calibration. In addition, embedding representations of the diagnosis codes do not contribute to improvement in model performance. We find dependency of each model's performance on the time point at which it is evaluated. This time dependency of the models' performance on the health care claims data may necessitate a different choice of model in quality of care issue detection at different points in time. We show the effectiveness of deep-learning based survival-analysis models in estimating the quality of care risk in hospital readmissions.

Interpretable machine learning models for hospital readmission prediction: a two-step extracted regression tree approach

BACKGROUND

Advanced machine learning models have received wide attention in assisting medical decision making due to the greater accuracy they can achieve. However, their limited interpretability imposes barriers for practitioners to adopt them. Recent advancements in interpretable machine learning tools allow us to look inside the black box of advanced prediction methods to extract interpretable models while maintaining similar prediction accuracy, but few studies have investigated the specific hospital readmission prediction problem with this spirit.

METHODS

Our goal is to develop a machine-learning (ML) algorithm that can predict 30- and 90- day hospital readmissions as accurately as black box algorithms while providing medically interpretable insights into readmission risk factors. Leveraging a state-of-art interpretable ML model, we use a two-step Extracted Regression Tree approach to achieve this goal. In the first step, we train a black box prediction algorithm. In the second step, we extract a regression tree from the output of the black box algorithm that allows direct interpretation of medically relevant risk factors. We use data from a large teaching hospital in Asia to learn the ML model and verify our two-step approach.

RESULTS

The two-step method can obtain similar prediction performance as the best black box model, such as Neural Networks, measured by three metrics: accuracy, the Area Under the Curve (AUC) and the Area Under the Precision-Recall Curve (AUPRC), while maintaining interpretability. Further, to examine whether the prediction results match the known medical insights (i.e., the model is truly interpretable and produces reasonable results), we show that key readmission risk factors extracted by the two-step approach are consistent with those found in the medical literature.

CONCLUSIONS

The proposed two-step approach yields meaningful prediction results that are both accurate and interpretable. This study suggests a viable means to improve the trust of machine learning based models in clinical practice for predicting readmissions through the two-step approach.

Heart Failure Emergency Readmission Prediction Using Stacking Machine Learning Model

Heart failure is a devastating disease that has high mortality rates and a negative impact on quality of life. Heart failure patients often experience emergency readmission after an initial episode, often due to inadequate management. A timely diagnosis and treatment of underlying issues can significantly reduce the risk of emergency readmissions. The purpose of this project was to predict emergency readmissions of discharged heart failure patients using classical machine learning (ML) models based on Electronic Health Record (EHR) data. The dataset used for this study consisted of 166 clinical biomarkers from 2008 patient records. Three feature selection techniques were studied along with 13 classical ML models using five-fold cross-validation. A stacking ML model was trained using the predictions of the three best-performing models for final classification. The stacking ML model provided an accuracy, precision, recall, specificity, F1-score, and area under the curve (AUC) of 89.41%, 90.10%, 89.41%, 87.83%, 89.28%, and 0.881, respectively. This indicates the effectiveness of the proposed model in predicting emergency readmissions. The healthcare providers can intervene pro-actively to reduce emergency hospital readmission risk and improve patient outcomes and decrease healthcare costs using the proposed model.

Using remotely monitored patient activity patterns after hospital discharge to predict 30 day hospital readmission: a randomized trial

Hospital readmission prediction models often perform poorly, but most only use information collected until the time of hospital discharge. In this clinical trial, we randomly assigned 500 patients discharged from hospital to home to use either a smartphone or wearable device to collect and transmit remote patient monitoring (RPM) data on activity patterns after hospital discharge. Analyses were conducted at the patient-day level using discrete-time survival analysis. Each arm was split into training and testing folds. The training set used fivefold cross-validation and then final model results are from predictions on the test set. A standard model comprised data collected up to the time of discharge including demographics, comorbidities, hospital length of stay, and vitals prior to discharge. An enhanced model consisted of the standard model plus RPM data. Traditional parametric regression models (logit and lasso) were compared to nonparametric machine learning approaches (random forest, gradient boosting, and ensemble). The main outcome was hospital readmission or death within 30 days of discharge. Prediction of 30-day hospital readmission significantly improved when including remotely-monitored patient data on activity patterns after hospital discharge and using nonparametric machine learning approaches. Wearables slightly outperformed smartphones but both had good prediction of 30-day hospital-readmission.

Predicting Hospital Readmissions in a Commercially Insured Population over Varying Time Horizons

BACKGROUND

Reducing hospital readmissions is a federal policy priority, and predictive models of hospital readmissions have proliferated in recent years; however, most such models tend to focus on the 30-day readmission time horizon and do not consider readmission over shorter (or longer) windows.

OBJECTIVES

To evaluate the performance of a predictive model of hospital readmissions over three different readmission timeframes in a commercially insured population.

DESIGN

Retrospective multivariate logistic regression with an 80/20 train/test split.

PARTICIPANTS

A total of 2,213,832 commercially insured inpatient admissions from 2016 to 2017 comprising 782,768 unique patients from the Health Care Cost Institute.

MAIN MEASURES

Outcomes are readmission within 14 days, 15-30 days, and 31-60 days from discharge. Predictor variables span six different domains: index admission, condition history, demographic, utilization history, pharmacy, and environmental controls.

KEY RESULTS

Our model generates C-statistics for holdout samples ranging from 0.618 to 0.915. The model's discriminative power declines with readmission time horizon: discrimination for readmission predictions within 14 days following discharge is higher than for readmissions 15-30 days following discharge, which in turn is higher than predictions 31-60 days following discharge. Additionally, the model's predictive power increases nonlinearly with the inclusion of successive risk factor domains: patient-level measures of utilization and condition history add substantially to the discriminative power of the model, while demographic information, pharmacy utilization, and environmental risk factors add relatively little.

CONCLUSION

It is more difficult to predict distant readmissions than proximal readmissions, and the more information the model uses, the better the predictions. Inclusion of utilization-based risk factors add substantially to the discriminative ability of the model, much more than any other included risk factor domain. Our best-performing models perform well relative to other published readmission prediction models. It is possible that these predictions could have operational utility in targeting readmission prevention interventions among high-risk individuals.

Using Explainable Artificial Intelligence to Predict Potentially Preventable Hospitalizations: A Population-Based Cohort Study in Denmark

BACKGROUND

The increasing aging population and limited health care resources have placed new demands on the healthcare sector. Reducing the number of hospitalizations has become a political priority in many countries, and special focus has been directed at potentially preventable hospitalizations.

OBJECTIVES

We aimed to develop an artificial intelligence (AI) prediction model for potentially preventable hospitalizations in the coming year, and to apply explainable AI to identify predictors of hospitalization and their interaction.

METHODS

We used the Danish CROSS-TRACKS cohort and included citizens in 2016-2017. We predicted potentially preventable hospitalizations within the following year using the citizens' sociodemographic characteristics, clinical characteristics, and health care utilization as predictors. Extreme gradient boosting was used to predict potentially preventable hospitalizations with Shapley additive explanations values serving to explain the impact of each predictor. We reported the area under the receiver operating characteristic curve, the area under the precision-recall curve, and 95% confidence intervals (CI) based on five-fold cross-validation.

RESULTS

The best performing prediction model showed an area under the receiver operating characteristic curve of 0.789 (CI: 0.782-0.795) and an area under the precision-recall curve of 0.232 (CI: 0.219-0.246). The predictors with the highest impact on the prediction model were age, prescription drugs for obstructive airway diseases, antibiotics, and use of municipality services. We found an interaction between age and use of municipality services, suggesting that citizens aged 75+ years receiving municipality services had a lower risk of potentially preventable hospitalization.

CONCLUSION

AI is suitable for predicting potentially preventable hospitalizations. The municipality-based health services seem to have a preventive effect on potentially preventable hospitalizations.

Systematic review identifies the design and methodological conduct of studies on machine learning-based prediction models

BACKGROUND AND OBJECTIVES

We sought to summarize the study design, modelling strategies, and performance measures reported in studies on clinical prediction models developed using machine learning techniques.

METHODS

We search PubMed for articles published between 01/01/2018 and 31/12/2019, describing the development or the development with external validation of a multivariable prediction model using any supervised machine learning technique. No restrictions were made based on study design, data source, or predicted patient-related health outcomes.

RESULTS

We included 152 studies, 58 (38.2% [95% CI 30.8-46.1]) were diagnostic and 94 (61.8% [95% CI 53.9-69.2]) prognostic studies. Most studies reported only the development of prediction models (n = 133, 87.5% [95% CI 81.3-91.8]), focused on binary outcomes (n = 131, 86.2% [95% CI 79.8-90.8), and did not report a sample size calculation (n = 125, 82.2% [95% CI 75.4-87.5]). The most common algorithms used were support vector machine (n = 86/522, 16.5% [95% CI 13.5-19.9]) and random forest (n = 73/522, 14% [95% CI 11.3-17.2]). Values for area under the Receiver Operating Characteristic curve ranged from 0.45 to 1.00. Calibration metrics were often missed (n = 494/522, 94.6% [95% CI 92.4-96.3]).

CONCLUSION

Our review revealed that focus is required on handling of missing values, methods for internal validation, and reporting of calibration to improve the methodological conduct of studies on machine learning-based prediction models.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO, CRD42019161764.

A hybrid machine learning framework to improve prediction of all-cause rehospitalization among elderly patients in Hong Kong

BACKGROUND

Accurately estimating elderly patients' rehospitalisation risk benefits clinical decisions and service planning. However, research in rehospitalisation and repeated hospitalisation yielded only models with modest performance, and the model performance deteriorates rapidly as the prediction timeframe expands beyond 28 days and for older participants.

METHODS

A temporal zero-inflated Poisson (tZIP) regression model was developed and validated retrospectively and prospectively. The data of the electronic health records (EHRs) contain cohorts (aged 60+) in a major public hospital in Hong Kong. Two temporal offset functions accounted for the associations between exposure time and parameters corresponding to the zero-inflated logistic component and the Poisson distribution's expected count. tZIP was externally validated with a retrospective cohort's rehospitalisation events up to 12 months after the discharge date. Subsequently, tZIP was validated prospectively after piloting its implementation at the study hospital. Patients discharged within the pilot period were tagged, and the proposed model's prediction of their rehospitalisation was verified monthly. Using a hybrid machine learning (ML) approach, the tZIP-based risk estimator's marginal effect on 28-day rehospitalisation was further validated, competing with other factors representing different post-acute and clinical statuses.

RESULTS

The tZIP prediction of rehospitalisation from 28 days to 365 days was achieved at above 80% discrimination accuracy retrospectively and prospectively in two out-of-sample cohorts. With a large margin, it outperformed the Cox proportional and linear models built with the same predictors. The hybrid ML revealed that the risk estimator's contribution to 28-day rehospitalisation outweighed other features relevant to service utilisation and clinical status.

CONCLUSIONS

A novel rehospitalisation risk model was introduced, and its risk estimators, whose importance outweighed all other factors of diverse post-acute care and clinical conditions, were derived. The proposed approach relies on four easily accessible variables easily extracted from EHR. Thus, clinicians could visualise patients' rehospitalisation risk from 28 days to 365 days after discharge and screen high-risk older patients for follow-up care at the proper time.

Implementing predictive tools in surgery: A narrative review in the context of orthopaedic surgery

Clinical predictive tools are a topic gaining interest. Many tools are developed each year to predict various outcomes in medicine and surgery. However, the proportion of predictive tools that are implemented in clinical practice is small in comparison to the total number of tools developed. This narrative review presents key principles to guide the translation of predictive tools from academic bodies of work into useful tools that complement clinical practice. Our review identified the following principles: (1) identifying a clinical gap, (2) selecting a target user or population, (3) optimizing predictive tool performance, (4) externally validating predictive tools, (5) marketing and disseminating the tool, (6) navigating the challenges of integrating a tool into existing healthcare systems, and (7) developing an ongoing monitoring and evaluation strategy. Although the review focuses on examples in orthopaedic surgery, the principles can be applied to other disciplines in medicine and surgery.

Machine learning methods to predict 30-day hospital readmission outcome among US adults with pneumonia: analysis of the national readmission database

Background

Hospital readmissions for pneumonia are a growing concern in the US, with significant consequences for costs and quality of care. This study developed the rule-based model and other machine learning (ML) models to predict 30-day readmission risk in patients with pneumonia and compared model performance.

Methods

This population-based study involved patients aged ≥ 18 years hospitalized with pneumonia from January 1, 2016, through November 30, 2016, using the Healthcare Cost and Utilization Project-National Readmission Database (HCUP-NRD). Rule-based algorithms and other ML algorithms, specifically decision trees, random forest, extreme gradient descent boosting (XGBoost), and Least Absolute Shrinkage and Selection Operator (LASSO), were used to model all-cause readmissions 30 days post-discharge from index pneumonia hospitalization. A total of 61 clinically relevant variables were included for ML model development. Models were trained on randomly partitioned 50% of the data and evaluated using the remaining dataset. Model hyperparameters were tuned using the ten-fold cross-validation on the resampled training dataset. The area under the receiver operating curves (AUROC) and area under precision-recall curves (AUPRC) were calculated for the testing set to evaluate the model performance.

Results

Of the 372,293 patients with an index hospital hospitalization for pneumonia, 48,280 (12.97%) were readmitted within 30 days. Judged by AUROC in the testing data, rule-based model (0.6591) significantly outperformed decision tree (0.5783, p value < 0.001), random forest (0.6509, p value < 0.01) and LASSO (0.6087, p value < 0.001), but was less superior than XGBoost (0.6606, p value = 0.015). The AUPRC of the rule-based model in the testing data (0.2146) was higher than the decision tree (0.1560), random forest (0.2052), and LASSO (0.2042), but was similar to XGBoost (0.2147). The top risk-predictive rules captured by the rule-based algorithm were comorbidities, illness severity, disposition locations, payer type, age, and length of stay. These predictive risk factors were also identified by other ML models with high variable importance.

Conclusion

The performance of machine learning models for predicting readmission in pneumonia patients varied. The XGboost was better than the rule-based model based on the AUROC. However, important risk factors for predicting readmission remained consistent across ML models.

Hospital readmissions and emergency department re-presentation of COVID-19 patients: a systematic review

Objective.

To characterize the frequency, causes, and predictors of readmissions of COVID-19 patients after discharge from heath facilities or emergency departments, interventions used to reduce readmissions, and outcomes of COVID-19 patients discharged from such settings.

Methods.

We performed a systematic review for case series and observational studies published between January 2020 and April 2021 in PubMed, Embase, LILACS, and MedRxiv, reporting the frequency, causes, or risk factors for readmission of COVID-19 survivors/patients. We conducted a narrative synthesis and assessed the methodological quality using the JBI critical appraisal checklist.

Results.

We identified 44 studies including data from 10 countries. The overall 30-day median readmission rate was 7.1%. Readmissions varied with the length of follow-up, occurring <10.5%, <14.5%, <21.5%, and <30%, respectively, for 10, 30, 60, and 253 days following discharge. Among those followed up for 30 and 60 days, the median time from discharge to readmission was 3 days and 8–11 days, respectively. The significant risk factor associated with readmission was having shorter length of stay, and the important causes included respiratory or thromboembolic events and chronic illnesses. Emergency department re-presentation was >20% in four studies. Risk factors associated with mortality were male gender, advanced age, and comorbidities.

Conclusions.

Readmission of COVID-19 survivors is frequent, and post-discharge mortality is significant in specific populations. There is an urgent need to further examine underlying reasons for early readmission and to prevent additional readmissions and adverse outcomes in COVID-19 survivors.

Identifying patients at risk of unplanned re-hospitalisation using statewide electronic health records

Preventing unplanned hospitalisations, including readmissions and re-presentations to the emergency department, is an important strategy for addressing the growing demand for hospital care. Significant successes have been reported from interventions put in place by hospitals to reduce their incidence. However, there is limited use of data-driven algorithms in hospital services to identify patients for enrolment into these intervention programs. Here we present the results of a study aiming to develop algorithms deployable at scale as part of a state government’s initiative to address rehospitalizations and which fills several gaps identified in the state-of-the-art literature. To the best of our knowledge, our study involves the largest-ever sample size for developing risk models. Logistic regression, random forests and gradient boosted techniques were explored as model candidates and validated retrospectively on five years of data from 27 hospitals in Queensland, Australia. The models used a range of predictor variables sourced from state-wide Emergency Department(ED), inpatient, hospital-dispensed medications and hospital-requested pathology databases. The investigation leads to several findings: (i) the advantage of looking at a longer patient data history, (ii) ED and inpatient datasets alone can provide useful information for predicting hospitalisation risk and the addition of medications and pathology test results leads to trivial performance improvements, (iii) predicting readmissions to the hospital was slightly easier than predicting re-presentations to ED after an inpatient stay, which was slightly easier again than predicting re-presentations to ED after an EDstay, (iv) a gradient boosted approach (XGBoost) was systematically the most powerful modelling approach across various tests.

An explanatory machine learning framework for studying pandemics: The case of COVID-19 emergency department readmissions

One of the major challenges that confront medical experts during a pandemic is the time required to identify and validate the risk factors of the novel disease and to develop an effective treatment protocol. Traditionally, this process involves numerous clinical trials that may take up to several years, during which strict preventive measures must be in place to control the outbreak and reduce the deaths. Advanced data analytics techniques, however, can be leveraged to guide and speed up this process. In this study, we combine evolutionary search algorithms, deep learning, and advanced model interpretation methods to develop a holistic exploratory-predictive-explanatory machine learning framework that can assist clinical decision-makers in reacting to the challenges of a pandemic in a timely manner. The proposed framework is showcased in studying emergency department (ED) readmissions of COVID-19 patients using ED visits from a real-world electronic health records database. After an exploratory feature selection phase using genetic algorithm, we develop and train a deep artificial neural network to predict early (i.e., 7-day) readmissions (AUC = 0.883). Lastly, a SHAP model is formulated to estimate additive Shapley values (i.e., importance scores) of the features and to interpret the magnitude and direction of their effects. The findings are mostly in line with those reported by lengthy and expensive clinical trial studies.

Explainable Stacking-Based Model for Predicting Hospital Readmission for Diabetic Patients

Artificial intelligence is changing the practice of healthcare. While it is essential to employ such solutions, making them transparent to medical experts is more critical. Most of the previous work presented disease prediction models, but did not explain them. Many healthcare stakeholders do not have a solid foundation in these models. Treating these models as ‘black box’ diminishes confidence in their predictions. The development of explainable artificial intelligence (XAI) methods has enabled us to change the models into a ‘white box’. XAI allows human users to comprehend the results from machine learning algorithms by making them easy to interpret. For instance, the expenditures of healthcare services associated with unplanned readmissions are enormous. This study proposed a stacking-based model to predict 30-day hospital readmission for diabetic patients. We employed Random Under-Sampling to solve the imbalanced class issue, then utilised SelectFromModel for feature selection and constructed a stacking model with base and meta learners. Compared with the different machine learning models, performance analysis showed that our model can better predict readmission than other existing models. This proposed model is also explainable and interpretable. Based on permutation feature importance, the strong predictors were the number of inpatients, the primary diagnosis, discharge to home with home service, and the number of emergencies. The local interpretable model-agnostic explanations method was also employed to demonstrate explainability at the individual level. The findings for the readmission of diabetic patients could be helpful in medical practice and provide valuable recommendations to stakeholders for minimising readmission and reducing public healthcare costs.

Machine learning prediction of postoperative unplanned 30-day hospital readmission in older adult

Background: Although unplanned hospital readmission is an important indicator for monitoring the perioperative quality of hospital care, few published studies of hospital readmission have focused on surgical patient populations, especially in the elderly. We aimed to investigate if machine learning approaches can be used to predict postoperative unplanned 30-day hospital readmission in old surgical patients.

Methods: We extracted demographic, comorbidity, laboratory, surgical, and medication data of elderly patients older than 65 who underwent surgeries under general anesthesia in West China Hospital, Sichuan University from July 2019 to February 2021. Different machine learning approaches were performed to evaluate whether unplanned 30-day hospital readmission can be predicted. Model performance was assessed using the following metrics: AUC, accuracy, precision, recall, and F1 score. Calibration of predictions was performed using Brier Score. A feature ablation analysis was performed, and the change in AUC with the removal of each feature was then assessed to determine feature importance.

Results: A total of 10,535 unique surgeries and 10,358 unique surgical elderly patients were included. The overall 30-day unplanned readmission rate was 3.36%. The AUCs of the six machine learning algorithms predicting postoperative 30-day unplanned readmission ranged from 0.6865 to 0.8654. The RF + XGBoost algorithm overall performed the best with an AUC of 0.8654 (95% CI, 0.8484–0.8824), accuracy of 0.9868 (95% CI, 0.9834–0.9902), precision of 0.3960 (95% CI, 0.3854–0.4066), recall of 0.3184 (95% CI, 0.259–0.3778), and F1 score of 0.4909 (95% CI, 0.3907–0.5911). The Brier scores of the six machine learning algorithms predicting postoperative 30-day unplanned readmission ranged from 0.3721 to 0.0464, with RF + XGBoost showing the best calibration capability. The most five important features of RF + XGBoost were operation duration, white blood cell count, BMI, total bilirubin concentration, and blood glucose concentration.

Conclusion: Machine learning algorithms can accurately predict postoperative unplanned 30-day readmission in elderly surgical patients.

Leveraging Advanced Data Analytics to Predict the Risk of All-Cause Seven-Day Emergency Readmissions

Introduction

Emergency readmissions have been a long-time, multifaceted, unsolved problem. Developing a predictive model calibrated with hospital-specific Electronic Health Record (EHR) data could give higher prediction accuracy and insights into high-risk patients for readmission. Thus, we need to proactively introduce the necessary interventions. This study aims to investigate the relationship between features that consider significant predictors of at-risk patients for seven-day readmission through logistic regression in addition to developing several machine learning models to test the predictability of those attributes using EHR data in a Saudi Arabia-specific ED context.

Methods

Univariate and multivariate logistic regression has been used to identify the most statistically significant features that contributed to classifying readmitted and not readmitted patients. Seven different machine learning models were trained and tested, and a comparison between the best-performing model was conducted in terms of five performance metrics. To construct the prediction model and internally validate it, the processed dataset was split into two sets: 70% for the training set and 30% for the test set or validation set.

Results

XGBoost achieved the highest accuracy (64%) in predicting early seven-day readmissions. Catboost was the second-best predictive model at 61%. XGBoost achieved the highest specificity at 70%, and all the models had a sensitivity of 57% except for XGBoost and Catboost at 32% and 38%, respectively. All predictive attributes, patient age, length of stay (LOS) in minutes, visit time (AM), marital status (married), number of medications, and number of abnormal lab results were significant predictors of early seven-day readmissions while marital status and number of vital-sign instabilities at discharge were not statistically significant predictors of seven-day readmission.

Conclusion

Although XGBoost and Catboost showed good accuracy, none of the models achieved good discriminative ability in terms of sensitivity and specificity. Thus, none can be clinically used for predicting early seven-day readmission. More predictive variables need to be fed into the model, specifically predictors approximate to the day of discharge, in order to optimize the model’s performance.

Decision support through risk cost estimation in 30-day hospital unplanned readmission

Unplanned hospital readmissions mean a significant burden for health systems. Accurately estimating the patient’s readmission risk could help to optimise the discharge decision-making process by smartly ordering patients based on a severity score, thus helping to improve the usage of clinical resources. A great number of heterogeneous factors can influence the readmission risk, which makes it highly difficult to be estimated by a human agent. However, this score could be achieved with the help of AI models, acting as aiding tools for decision support systems. In this paper, we propose a machine learning classification and risk stratification approach to assess the readmission problem and provide a decision support system based on estimated patient risk scores.

Classification of Electrocardiography Hybrid Convolutional Neural Network-Long Short Term Memory with Fully Connected Layer

Electrocardiography (ECG) is a technique for observing and recording the electrical activity of the human heart. The usage of an ECG signal is common among clinical professionals in the collection of time data for the examination of any rhythmic conditions associated with a subject. The investigation was carried out in order to computerize the assignment by exhibiting the issue using encoder-decoder techniques, creating the information that was simply typical of it, and utilising misfortune appropriation to anticipate standard or anomalous information. On a broad variety of applications such as voice recognition and prediction, the long short-term memory (LSTM) fully connected layer (FCL) and the two convolutional neural networks (CNNs) have shown superior performance over deep learning networks (DLNs). DNNs are suitable for making high points for a more divisible region and CNNs are suitable for reducing recurrence types, LSTMs are appropriate for temporary displays, in the same way as CNNs are appropriate for reducing recurrence types. The CNN, LSTM, and DNN algorithms are acceptable for viewing. The complementarity of DNNs, CNNs, and LSTMs was investigated in this research by bringing them all together under the single architectural company. The researchers got the ECG data from the MIT-BIH arrhythmia database as a result of the investigation. Our results demonstrate that the approach proposed may expressively describe ECG series and identify abnormalities via scores that outperform existing supervised and unsupervised methods in both the short term and long term. The LSTM network and FCL additionally demonstrated that the unbalanced datasets associated with the ECG beat detection problem could be consistently resolved and that they were not susceptible to the accuracy of ECG signals. It is recommended that cardiologists employ the unique technique to aid them in performing reliable and impartial interpretation of ECG data in telemedicine settings.

A novel interpretable machine learning system to generate clinical risk scores: An application for predicting early mortality or unplanned readmission in a retrospective cohort study

Risk scores are widely used for clinical decision making and commonly generated from logistic regression models. Machine-learning-based methods may work well for identifying important predictors to create parsimonious scores, but such 'black box' variable selection limits interpretability, and variable importance evaluated from a single model can be biased. We propose a robust and interpretable variable selection approach using the recently developed Shapley variable importance cloud (ShapleyVIC) that accounts for variability in variable importance across models. Our approach evaluates and visualizes overall variable contributions for in-depth inference and transparent variable selection, and filters out non-significant contributors to simplify model building steps. We derive an ensemble variable ranking from variable contributions across models, which is easily integrated with an automated and modularized risk score generator, AutoScore, for convenient implementation. In a study of early death or unplanned readmission after hospital discharge, ShapleyVIC selected 6 variables from 41 candidates to create a well-performing risk score, which had similar performance to a 16-variable model from machine-learning-based ranking. Our work contributes to the recent emphasis on interpretability of prediction models for high-stakes decision making, providing a disciplined solution to detailed assessment of variable importance and transparent development of parsimonious clinical risk scores.

Reproducibility of prediction models in health services research

The field of health services research studies the health care system by examining outcomes relevant to patients and clinicians but also health economists and policy makers. Such outcomes often include health care spending, and utilization of care services. Building accurate prediction models using reproducible research practices for health services research is important for evidence-based decision making. Several systematic reviews have summarized prediction models for outcomes relevant to health services research, but these systematic reviews do not present a thorough assessment of reproducibility and research quality of the prediction modelling studies. In the present commentary, we discuss how recent advances in prediction modelling in other medical fields can be applied to health services research. We also describe the current status of prediction modelling in health services research, and we summarize available methodological guidance for the development, update, external validation and systematic appraisal of prediction models.

Implementation of prediction models in the emergency department from an implementation science perspective—Determinants, outcomes and real-world impact: A scoping review protocol

The number of prediction models developed for use in emergency departments (EDs) have been increasing in recent years to complement traditional triage systems. However, most of these models have only reached the development or validation phase, and few have been implemented in clinical practice. There is a gap in knowledge on the real-world performance of prediction models in the ED and how they can be implemented successfully into routine practice. Existing reviews of prediction models in the ED have also mainly focused on model development and validation. The aim of this scoping review is to summarize the current landscape and understanding of implementation of predictions models in the ED. This scoping review follows the Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist. We will include studies that report implementation outcomes and/or contextual determinants according to the RE-AIM/PRISM framework for prediction models used in EDs. We will include outcomes or contextual determinants studied at any point of time in the implementation process except for effectiveness, where only post-implementation results will be included. Conference abstracts, theses and dissertations, letters to editors, commentaries, non-research documents and non-English full-text articles will be excluded. Four databases (MEDLINE (through PubMed), Embase, Scopus and CINAHL) will be searched from their inception using a combination of search terms related to the population, intervention and outcomes. Two reviewers will independently screen articles for inclusion and any discrepancy resolved with a third reviewer. Results from included studies will be summarized narratively according to the RE-AIM/PRISM outcomes and domains. Where appropriate, a simple descriptive summary of quantitative outcomes may be performed.

Impact of social determinants of health on improving the LACE index for 30-day unplanned readmission prediction

Objective

Early and accurate prediction of patients at risk of readmission is key to reducing costs and improving outcomes. LACE is a widely used score to predict 30-day readmissions. We examine whether adding social determinants of health (SDOH) to LACE can improve its predictive performance.

Methods

This is a retrospective study that included all inpatient encounters in the state of Maryland in 2019. We constructed predictive models by fitting Logistic Regression (LR) on LACE and different sets of SDOH predictors. We used the area under the curve (AUC) to evaluate discrimination and SHapley Additive exPlanations values to assess feature importance.

Results

Our study population included 316 558 patients of whom 35 431 (11.19%) patients were readmitted after 30 days. Readmitted patients had more challenges with individual-level SDOH and were more likely to reside in communities with poor SDOH conditions. Adding a combination of individual and community-level SDOH improved LACE performance from AUC = 0.698 (95% CI [0.695–0.7]; ref) to AUC = 0.708 (95% CI [0.705–0.71]; P < .001). The increase in AUC was highest in black patients (+1.6), patients aged 65 years or older (+1.4), and male patients (+1.4).

Discussion

We demonstrated the value of SDOH in improving the LACE index. Further, the additional predictive value of SDOH on readmission risk varies by subpopulations. Vulnerable populations like black patients and the elderly are likely to benefit more from the inclusion of SDOH in readmission prediction.

Conclusion

These findings provide potential SDOH factors that health systems and policymakers can target to reduce overall readmissions.

Machine learning models for diabetes management in acute care using electronic medical records: A systematic review

BACKGROUND

Machine learning (ML) is a subset of Artificial Intelligence (AI) that is used to predict and potentially prevent adverse patient outcomes. There is increasing interest in the application of these models in digital hospitals to improve clinical decision-making and chronic disease management, particularly for patients with diabetes. The potential of ML models using electronic medical records (EMR) to improve the clinical care of hospitalised patients with diabetes is currently unknown.

OBJECTIVE

The aim was to systematically identify and critically review the published literature examining the development and validation of ML models using EMR data for improving the care of hospitalised adult patients with diabetes.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) guidelines were followed. Four databases were searched (Embase, PubMed, IEEE and Web of Science) for studies published between January 2010 to January 2022. The reference lists of the eligible articles were manually searched. Articles that examined adults and both developed and validated ML models using EMR data were included. Studies conducted in primary care and community care settings were excluded. Studies were independently screened and data was extracted using Covidence® systematic review software. For data extraction and critical appraisal, the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was followed. Risk of bias was assessed using the Prediction model Risk Of Bias Assessment Tool (PROBAST). Quality of reporting was assessed by adherence to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guideline. The IJMEDI checklist was followed to assess quality of ML models and the reproducibility of their outcomes. The external validation methodology of the studies was appraised.

RESULTS

Of the 1317 studies screened, twelve met inclusion criteria. Eight studies developed ML models to predict disglycaemic episodes for hospitalized patients with diabetes, one study developed a ML model to predict total insulin dosage, two studies predicted risk of readmission, and one study improved the prediction of hospital readmission for inpatients with diabetes. All included studies were heterogeneous with regard to ML types, cohort, input predictors, sample size, performance and validation metrics and clinical outcomes. Two studies adhered to the TRIPOD guideline. The methodological reporting of all the studies was evaluated to be at high risk of bias. The quality of ML models in all studies was assessed as poor. Robust external validation was not performed on any of the studies. No models were implemented or evaluated in routine clinical care.

CONCLUSIONS

This review identified a limited number of ML models which were developed to improve inpatient management of diabetes. No ML models were implemented in real hospital settings. Future research needs to enhance the development, reporting and validation steps to enable ML models for integration into routine clinical care.

Factors Related to Pediatric Readmissions of Four Major Diagnostic Categories in Hawai`i

Readmissions are a key quality measure for health care decision making and understanding variables associated with readmissions has become a crucial research area. This study identified patient-level factors that might be associated with pediatric readmissions using a database that included inpatient data from 2008 to 2017 from Hawai`i. Four major diagnostic categories with the most pediatric readmissions in the state were identified: respiratory, digestive, mental, and nervous system diseases and disorders. The associations between readmission and patient-level variables, such as age, sex, race/ethnicity, insurance status, and Charlson Comorbidity Index (CCI), were determined for each diagnosis and for overall readmissions. CCI and insurance were the strongest predictors when all diagnoses were combined. However, for some diagnoses, there was weak or no association between CCI, insurance, and readmission. This suggests that diagnosis-specific analysis of predictors of readmission may be more useful than looking at predictors of readmission for all diagnoses combined. While this study focused on patient variables, future studies should also incorporate how hospital variables may also be related to diagnosis.

Radiological Analysis of COVID-19 Using Computational Intelligence: A Broad Gauge Study

Pulmonary medical image analysis using image processing and deep learning approaches has made remarkable achievements in the diagnosis, prognosis, and severity check of lung diseases. The epidemic of COVID-19 brought out by the novel coronavirus has triggered a critical need for artificial intelligence assistance in diagnosing and controlling the disease to reduce its effects on people and global economies. This study aimed at identifying the various COVID-19 medical imaging analysis models proposed by different researchers and featured their merits and demerits. It gives a detailed discussion on the existing COVID-19 detection methodologies (diagnosis, prognosis, and severity/risk detection) and the challenges encountered for the same. It also highlights the various preprocessing and post-processing methods involved to enhance the detection mechanism. This work also tries to bring out the different unexplored research areas that are available for medical image analysis and how the vast research done for COVID-19 can advance the field. Despite deep learning methods presenting high levels of efficiency, some limitations have been briefly described in the study. Hence, this review can help understand the utilization and pros and cons of deep learning in analyzing medical images.

Predicting Hospital Readmissions from Health Insurance Claims Data: A Modeling Study Targeting Potentially Inappropriate Prescribing

BACKGROUND

 Numerous prediction models for readmissions are developed from hospital data whose predictor variables are based on specific data fields that are often not transferable to other settings. In contrast, routine data from statutory health insurances (in Germany) are highly standardized, ubiquitously available, and would thus allow for automatic identification of readmission risks.

OBJECTIVES

 To develop and internally validate prediction models for readmissions based on potentially inappropriate prescribing (PIP) in six diseases from routine data.

METHODS

 In a large database of German statutory health insurance claims, we detected disease-specific readmissions after index admissions for acute myocardial infarction (AMI), heart failure (HF), a composite of stroke, transient ischemic attack or atrial fibrillation (S/AF), chronic obstructive pulmonary disease (COPD), type-2 diabetes mellitus (DM), and osteoporosis (OS). PIP at the index admission was determined by the STOPP/START criteria (Screening Tool of Older Persons' Prescriptions/Screening Tool to Alert doctors to the Right Treatment) which were candidate variables in regularized prediction models for specific readmission within 90 days. The risks from disease-specific models were combined ("stacked") to predict all-cause readmission within 90 days. Validation performance was measured by the c-statistics.

RESULTS

 While the prevalence of START criteria was higher than for STOPP criteria, more single STOPP criteria were selected into models for specific readmissions. Performance in validation samples was the highest for DM (c-statistics: 0.68 [95% confidence interval (CI): 0.66-0.70]), followed by COPD (c-statistics: 0.65 [95% CI: 0.64-0.67]), S/AF (c-statistics: 0.65 [95% CI: 0.63-0.66]), HF (c-statistics: 0.61 [95% CI: 0.60-0.62]), AMI (c-statistics: 0.58 [95% CI: 0.56-0.60]), and OS (c-statistics: 0.51 [95% CI: 0.47-0.56]). Integrating risks from disease-specific models to a combined model for all-cause readmission yielded a c-statistics of 0.63 [95% CI: 0.63-0.64].

CONCLUSION

 PIP successfully predicted readmissions for most diseases, opening the possibility for interventions to improve these modifiable risk factors. Machine-learning methods appear promising for future modeling of PIP predictors in complex older patients with many underlying diseases.

A review of literature on risk prediction tools for hospital readmissions in older adults

PURPOSE

The purpose of this paper is to identify and analyse the readmission risk prediction tools reported in the literature and their benefits when it comes to healthcare organisations and management.

DESIGN/METHODOLOGY/APPROACH

Readmission risk prediction is a growing topic of interest with the aim of identifying patients in particular those suffering from chronic diseases such as congestive heart failure, chronic obstructive pulmonary disease and diabetes, who are at risk of readmission. Several models have been developed with different levels of predictive ability. A structured and extensive literature search of several databases was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-analysis strategy, and this yielded a total of 48,984 records.

FINDINGS

Forty-three articles were selected for full-text and extensive review after following the screening process and according to the eligibility criteria. About 34 unique readmission risk prediction models were identified, in which their predictive ability ranged from poor to good (c statistic 0.5-0.86). Readmission rates ranged between 3.1 and 74.1% depending on the risk category. This review shows that readmission risk prediction is a complex process and is still relatively new as a concept and poorly understood. It confirms that readmission prediction models hold significant accuracy at identifying patients at higher risk for such an event within specific context.

RESEARCH LIMITATIONS/IMPLICATIONS

Since most prediction models were developed for specific populations, conditions or hospital settings, the generalisability and transferability of the predictions across wider or other contexts may be difficult to achieve. Therefore, the value of prediction models remains limited to hospital management. Future research is indicated in this regard.

ORIGINALITY/VALUE

This review is the first to cover readmission risk prediction tools that have been published in the literature since 2011, thereby providing an assessment of the relevance of this crucial KPI to health organisations and managers.

Derivation and Validation of the Cancer READMIT Score: A Readmission Risk Scoring System for Patients With Solid Tumor Malignancies

PURPOSE

Readmissions for the medical treatment of cancer have traditionally been excluded from readmission measures under the Hospital Readmissions Reduction Program. Patients with cancer often have higher readmission rates and may need heightened support to ensure effective care transitions after hospitalization. Estimating readmission risk before discharge may assist in discharge planning efforts and help promote care coordination at time of discharge.

PATIENTS AND METHODS

We developed and validated a readmission risk scoring system among a cohort of adult cancer patients with solid tumor admitted at a comprehensive cancer center. Multivariate logistic regression analysis was used to develop the model. The model's discriminative capacity was evaluated through a receiver operating characteristic curve analysis. We further compared the performance of the developed score with existing risk scores for 30-day readmission.

RESULTS

The 30-day unplanned readmission rate in the total cohort was 16.0% (n = 1,078 of 6,720). After multivariate analysis, Cancer site, Recent emergency room visit within 30 days, non-English primary language, Anemia defined as hemoglobin < 10 g/dL, > 4 Days length of stay during the index admission, unmarried Marital status, Increased white blood cell count > 11 × 10⁹/L, and distant Tumor spread were significantly associated with risk of unplanned 30-day readmission. The derived score, which we call the Cancer READMIT score, had modest discriminatory performance in predicting readmissions (area under the curve for the model receiver operating characteristic curve = 0.647).

CONCLUSION

The Cancer READMIT score was able to predict 30-day unplanned readmissions to our institution with fairly modest performance. External validation of our derived risk scoring system is recommended.

OUP accepted manuscript

Objective

Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model’s potential to introduce bias.

Materials and Methods

Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist.

Results

We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern.

Discussion

Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed.

Conclusion

The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

The Promise for Reducing Healthcare Cost with Predictive Model: An Analysis with Quantized Evaluation Metric on Readmission

Quality of care data has gained transparency captured through various measurements and reporting. Readmission measure is especially related to unfavorable patient outcomes that directly bends the curve of healthcare cost. Under the Hospital Readmission Reduction Program, payments to hospitals were reduced for those with excessive 30-day rehospitalization rates. These penalties have intensified efforts from hospital stakeholders to implement strategies to reduce readmission rates. One of the key strategies is the deployment of predictive analytics stratified by patient population. The recent research in readmission model is focused on making its prediction more accurate. As cost-saving improvements through artificial intelligent-based health solutions are expected, the broad economic impact of such digital tool remains unknown. Meanwhile, reducing readmission rate is associated with increased operating expenses due to targeted interventions. The increase in operating margin can surpass native readmission cost. In this paper, we propose a quantized evaluation metric to provide a methodological mean in assessing whether a predictive model represents cost-effective way of delivering healthcare. Herein, we evaluate the impact machine learning has had on transitional care and readmission with proposed metric. The final model was estimated to produce net healthcare savings at over $1 million given a 50% rate of successfully preventing a readmission.

Association Rules Mining for Hospital Readmission: A Case Study

As an indicator of healthcare quality and performance, hospital readmission incurs major costs for healthcare systems worldwide. Understanding the relationships between readmission factors, such as input features and readmission length, is challenging following intricate hospital readmission procedures. This study discovered the significant correlation between potential readmission factors (threshold of various settings for readmission length) and basic demographic variables. Association rule mining (ARM), particularly the Apriori algorithm, was utilised to extract the hidden input variable patterns and relationships among admitted patients by generating supervised learning rules. The mined rules were categorised into two outcomes to comprehend readmission data; (i) the rules associated with various readmission length and (ii) several expert-validated variables related to basic demographics (gender, race, and age group). The extracted rules proved useful to facilitate decision-making and resource preparation to minimise patient readmission.

Identifying Children at Readmission Risk: At-Admission versus Traditional At-Discharge Readmission Prediction Model

The timing of 30-day pediatric readmissions is skewed with approximately 40% of the incidents occurring within the first week of hospital discharges. The skewed readmission time distribution coupled with delay in health information exchange among healthcare providers might offer a limited time to devise a comprehensive intervention plan. However, pediatric readmission studies are thus far limited to the development of the prediction model after hospital discharges. In this study, we proposed a novel pediatric readmission prediction model at the time of hospital admission which can improve the high-risk patient selection process. We also compared proposed models with the standard at-discharge readmission prediction model. Using the Hospital Cost and Utilization Project database, this prognostic study included pediatric hospital discharges in Florida from January 2016 through September 2017. Four machine learning algorithms—logistic regression with backward stepwise selection, decision tree, Support Vector machines (SVM) with the polynomial kernel, and Gradient Boosting—were developed for at-admission and at-discharge models using a recursive feature elimination technique with a repeated cross-validation process. The performance of the at-admission and at-discharge model was measured by the area under the curve. The performance of the at-admission model was comparable with the at-discharge model for all four algorithms. SVM with Polynomial Kernel algorithms outperformed all other algorithms for at-admission and at-discharge models. Important features associated with increased readmission risk varied widely across the type of prediction model and were mostly related to patients’ demographics, social determinates, clinical factors, and hospital characteristics. Proposed at-admission readmission risk decision support model could help hospitals and providers with additional time for intervention planning, particularly for those targeting social determinants of children’s overall health.

[Limits of outpatient rheumatology care]

BACKGROUND

In recent years outpatient rheumatology has developed into one of the most important pillars of rheumatological care. The better diagnostic possibilities, the well-founded training of rheumatologists and the diverse guidelines have led to increasingly more patients being diagnosed as outpatients and can be treated by the most modern options as outpatients. Nevertheless, limits have also been set for outpatient rheumatology.

OBJECTIVE

This article aims to highlight what the limits of outpatient rheumatology currently are and who sets the limits. The presentation of the limitations should help all participants in the healthcare system to avoid false expectations and frustration; however, limits can also be relocated or overcome by appropriate motivation. The long-term target should be a further improvement in the interaction between outpatient and inpatient rheumatology and therefore the rheumatological care of patients.

RESULTS AND DISCUSSION

The medical limits of outpatient rheumatology are relatively clear and predominantly uncontroversial, the unjustified referral to the inpatient section is, for example, relatively rarely a problem. Some theoretically possible outpatient interventions are carried out in the inpatient sector more due to logistical problems. The healthcare political and legal limitations of outpatient rheumatological institutions are greater. Despite further limitations of outpatient rheumatology this discipline and the outpatient activity still represents an exciting, modern and last but not least a very satisfying activity, which is promoted here.

Post Discharge mHealth and Teach-Back Communication Effectiveness on Hospital Readmissions: A Systematic Review

Hospital readmissions pose a threat to the constrained health resources, especially in resource-poor low-and middle-income countries. In such scenarios, appropriate technologies to reduce avoidable readmissions in hospitals require innovative interventions. mHealth and teach-back communication are robust interventions, utilized for the reduction in preventable hospital readmissions. This review was conducted to highlight the effectiveness of mHealth and teach-back communication in hospital readmission reduction with a view to provide the best available evidence on such interventions. Two authors independently searched for appropriate MeSH terms in three databases (PubMed, Wiley, and Google Scholar). After screening the titles and abstracts, shortlisted manuscripts were subjected to quality assessment and analysis. Two authors checked the manuscripts for quality assessment and assigned scores utilizing the QualSyst tool. The average of the scores assigned by the reviewers was calculated to assign a summary quality score (SQS) to each study. Higher scores showed methodological vigor and robustness. Search strategies retrieved a total of 1932 articles after the removal of duplicates. After screening titles and abstracts, 54 articles were shortlisted. The complete reading resulted in the selection of 17 papers published between 2002 and 2019. Most of the studies were interventional and all the studies focused on hospital readmission reduction as the primary or secondary outcome. mHealth and teach-back communication were the two most common interventions that catered for the hospital readmissions. Among mHealth studies (11 out of 17), seven studies showed a significant reduction in hospital readmissions while four did not exhibit any significant reduction. Among the teach-back communication group (6 out of 17), the majority of the studies (5 out of 6) showed a significant reduction in hospital readmissions while one publication did not elicit a significant hospital readmission reduction. mHealth and teach-back communication methods showed positive effects on hospital readmission reduction. These interventions can be utilized in resource-constrained settings, especially low- and middle-income countries, to reduce preventable readmissions.

Machine learning based early mortality prediction in the emergency department

BACKGROUND

It is a great challenge for emergency physicians to early detect the patient's deterioration and prevent unexpected death through a large amount of clinical data, which requires sufficient experience and keen insight.

OBJECTIVE

To evaluate the performance of machine learning models in quantifying the severity of emergency department (ED) patients and identifying high-risk patients.

METHODS

Using routinely-available demographics, vital signs and laboratory tests extracted from electronic health records (EHRs), a framework based on machine learning and feature engineering was proposed for mortality prediction. Patients who had one complete record of vital signs and laboratory tests in ED were included. The following patients were excluded: pediatric patients aged < 18 years, pregnant woman, and patients died or were discharged or hospitalized within 12 h after admission. Based on 76 original features extracted, 9 machine learning models were adopted to validate our proposed framework. Their optimal hyper-parameters were fine-tuned using the grid search method. The prediction results were evaluated on performance metrics (i.e., accuracy, area under the curve (AUC), recall and precision) with repeated 5-fold cross-validation (CV). The time window from patient admission to the prediction was analyzed at 12 h, 24 h, 48 h, and entire stay.

RESULTS

We studied a total of 1114 ED patients with 71.54% (797/1114) survival and 28.46% (317/1114) death in the hospital. The results revealed a more complete time window leads to better prediction performance. Using the entire stay records, the LightGBM model with refined feature engineering demonstrated high discrimination and achieved 93.6% (±0.008) accuracy, 97.6% (±0.003) AUC, 97.1% (±0.008) recall, and 94.2% (±0.006) precision, even if no diagnostic information was utilized.

CONCLUSIONS

This study quantifies the criticality of ED patients and appears to have significant potential as a clinical decision support tool in assisting physicians in their clinical routine. While the model requires validation before use elsewhere, the same methodology could be used to create a strong model for the new hospital.