Risk prediction models for hospital readmission: a systematic review

Current state of artificial intelligence-based algorithms for hospital admission prediction in patients with heart failure: a scoping review. EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2022;3:415-425. [PMID: 36712159 PMCID: PMC9707890 DOI: 10.1093/ehjdh/ztac035]

AIMS

Patients with congestive heart failure (HF) are prone to clinical deterioration leading to hospital admissions, burdening both patients and the healthcare system. Predicting hospital admission in this patient group could enable timely intervention, with subsequent reduction of these admissions. To date, hospital admission prediction remains challenging. Increasing amounts of acquired data and development of artificial intelligence (AI) technology allow for the creation of reliable hospital prediction algorithms for HF patients. This scoping review describes the current literature on strategies and performance of AI-based algorithms for prediction of hospital admission in patients with HF.

METHODS AND RESULTS

PubMed, EMBASE, and the Web of Science were used to search for articles using machine learning (ML) and deep learning methods to predict hospitalization in patients with HF. After eligibility screening, 23 articles were included. Sixteen articles predicted 30-day hospital (re-)admission resulting in an area under the curve (AUC) ranging from 0.61 to 0.79. Six studies predicted hospital admission over longer time periods ranging from 6 months to 3 years, with AUC's ranging from 0.65 to 0.78. One study prospectively evaluated performance of a disposable sensory patch at home after hospitalization which resulted in an AUC of 0.89 for unplanned hospital admission prediction.

CONCLUSION

AI has the potential to enable prediction of hospital admission in HF patients. Improvement of data management, adding new data sources such as telemonitoring data and ML models and prospective and external validation of current models must be performed before clinical applicability is possible.

Machine learning and LACE index for predicting 30-day readmissions after heart failure hospitalization in elderly patients

Machine learning (ML) techniques may improve readmission prediction performance in heart failure (HF) patients. This study aimed to assess the ability of ML algorithms to predict unplanned all-cause 30-day readmissions in HF elderly patients, and to compare them with conventional LACE (Length of hospitalization, Acuity, Comorbidities, Emergency department visits) index. All patients aged ≥ 65 years discharged alive between 2010 and 2019 after a hospitalization for acute HF were included in this retrospective cohort study. We applied MICE (Multivariate Imputation via Chained Equations) method to obtain a balanced, fully valued dataset and LASSO (Least Absolute Shrinkage and Selection Operator) algorithm to get the most significant features. Training (80% of records) and test (20%) cohorts were randomly selected. Study population: 3079 patients, 394 (12.8%) presented at least one readmission within 30 days, and 2685 (87.2%) did not. In the test cohort AUCs (IC95%) of XGBoost, Ada Boost Classifier, Random forest, and Gradient Boosting, and LACE Index were: 0.803 (0.734-0.872), 0.782 (0.711-0.854), 0.776 (0.703-0.848), 0.786 (0.715-0.857), and 0.504 (0.414-0.594), respectively, for predicting readmissions. A SHAP analysis was performed to offer a breakdown of the ML variables associated with readmission. Positive and negative predicting values estimates of the different ML models and LACE index were also provided, for several values of readmission rate prevalence. Among elderly patients, the rate of all-cause unplanned 30-day readmissions after hospitalization due to an acute HF was high. ML models performed better than the conventional LACE index for predicting readmissions. ML models can be proposed as promising tools for the identification of subjects at high risk of hospitalization in this clinical setting, enabling care teams to target interventions for improving overall clinical outcomes.

Implementation Experience with a 30-Day Hospital Readmission Risk Score in a Large, Integrated Health System: A Retrospective Study

BACKGROUND

Driven by quality outcomes and economic incentives, predicting 30-day hospital readmissions remains important for healthcare systems. The Cleveland Clinic Health System (CCHS) implemented an internally validated readmission risk score in the electronic medical record (EMR).

OBJECTIVE

We evaluated the predictive accuracy of the readmission risk score across CCHS hospitals, across primary discharge diagnosis categories, between surgical/medical specialties, and by race and ethnicity.

DESIGN

Retrospective cohort study.

PARTICIPANTS

Adult patients discharged from a CCHS hospital April 2017-September 2020.

MAIN MEASURES

Data was obtained from the CCHS EMR and billing databases. All patients discharged from a CCHS hospital were included except those from Oncology and Labor/Delivery, patients with hospice orders, or patients who died during admission. Discharges were categorized as surgical if from a surgical department or surgery was performed. Primary discharge diagnoses were classified per Agency for Healthcare Research and Quality Clinical Classifications Software Level 1 categories. Discrimination performance predicting 30-day readmission is reported using the c-statistic.

RESULTS

The final cohort included 600,872 discharges from 11 Northeast Ohio and Florida CCHS hospitals. The readmission risk score for the cohort had a c-statistic of 0.6875 with consistent yearly performance. The c-statistic for hospital sites ranged from 0.6762, CI [0.6634, 0.6876], to 0.7023, CI [0.6903, 0.7132]. Medical and surgical discharges showed consistent performance with c-statistics of 0.6923, CI [0.6807, 0.7045], and 0.6802, CI [0.6681, 0.6925], respectively. Primary discharge diagnosis showed variation, with lower performance for congenital anomalies and neoplasms. COVID-19 had a c-statistic of 0.6387. Subgroup analyses showed c-statistics of > 0.65 across race and ethnicity categories.

CONCLUSIONS

The CCHS readmission risk score showed good performance across diverse hospitals, across diagnosis categories, between surgical/medical specialties, and by patient race and ethnicity categories for 3 years after implementation, including during COVID-19. Evaluating clinical decision-making tools post-implementation is crucial to determine their continued relevance, identify opportunities to improve performance, and guide their appropriate use.

Predictive Modeling of Hospital Readmission: Challenges and Solutions

Hospital readmission prediction is a study to learn models from historical medical data to predict probability of a patient returning to hospital in a certain period, e.g. 30 or 90 days, after the discharge. The motivation is to help health providers deliver better treatment and post-discharge strategies, lower the hospital readmission rate, and eventually reduce the medical costs. Due to inherent complexity of diseases and healthcare ecosystems, modeling hospital readmission is facing many challenges. By now, a variety of methods have been developed, but existing literature fails to deliver a complete picture to answer some fundamental questions, such as what are the main challenges and solutions in modeling hospital readmission; what are typical features/models used for readmission prediction; how to achieve meaningful and transparent predictions for decision making; and what are possible conflicts when deploying predictive approaches for real-world usages. In this paper, we systematically review computational models for hospital readmission prediction, and propose a taxonomy of challenges featuring four main categories: (1) data variety and complexity; (2) data imbalance, locality and privacy; (3) model interpretability; and (4) model implementation. The review summarizes methods in each category, and highlights technical solutions proposed to address the challenges. In addition, a review of datasets and resources available for hospital readmission modeling also provides firsthand materials to support researchers and practitioners to design new approaches for effective and efficient hospital readmission prediction.

Dismissal disagreement and discharge delays: Associations of patient-clinician plan of care agreement with discharge outcomes

BACKGROUND

Many hospitalized patients do not understand or agree with their clinicians about their discharge plan. However, the effect of disagreement on discharge outcomes is unknown.

OBJECTIVE

To measure the correlation between patient-clinician care agreement and discharge outcomes.

DESIGN

A prospective cohort study was performed from September 2019 to March 2020 (Rochester, MN, USA).

SETTING AND PARTICIPANTS

Internal medicine patients and their primary clinician (resident, advanced practice clinician or attending) hospitalized from September 2019-March 2020 at Mayo Clinic Hospital. Participants were independently surveyed following hospital day #3 ward rounds regarding the goals of the hospitalization and discharge planning.

MAIN OUTCOME AND MEASURES

Patient-clinician agreement for main diagnosis, patient's main concern, and four domains of discharge planning was assessed. Readiness for hospital discharge, delayed discharge, and 30-day readmission was measured. Then, associations between patient-clinician agreement, delayed discharge, and 30-day readmissions were analyzed using multivariable logistic regression.

RESULTS

Of the 436 patients and clinicians, 17.7% completely agreed about what needs to be accomplished before dismissal, 40.8% agreed regarding discharge date, and 71.1% agreed regarding discharge location. In the multivariable model, patient-clinician agreement scores were not significantly correlated with discharge outcomes. Patient-clinician agreement on discharge location was higher for those discharged to home (81.5%) versus skilled nursing facility (48.5%) or assisted living (42.9%) (p < .0001). The agreement on the expected length of stay was highest for home-goers (45.9%) compared to skilled nursing (32.0%) or assisted living (21.4%) (p = .004).

CONCLUSIONS

Patients and their clinicians frequently disagree about when and where a patient will go after hospitalization, particularly for those discharged to a skilled nursing facility. While disagreement did not predict discharge outcomes, our findings suggest opportunities to improve effective communication and promote shared mental models regarding discharge earlier in the hospital stay.

Predictors and Impact of Postoperative 30-Day Readmission in Glioblastoma

BACKGROUND

Postoperative 30-day readmissions have been shown to negatively affect survival and other important outcomes in patients with glioblastoma (GBM).

OBJECTIVE

To further investigate patient readmission risk factors of primary and recurrent patients with GBM.

METHODS

The authors retrospectively reviewed records of 418 adult patients undergoing 575 craniotomies for histologically confirmed GBM at an academic medical center. Patient demographics, comorbidities, and clinical characteristics were collected and compared by patient readmission status using chi-square and Mann-Whitney U testing. Multivariable logistic regression was performed to identify risk factors that predicted 30-day readmissions.

RESULTS

The cohort included 69 (12%) 30-day readmissions after 575 operations. Readmitted patients experienced significantly lower median overall survival (11.3 vs 16.4 months, P = .014), had a lower mean Karnofsky Performance Scale score (66.9 vs 74.2, P = .005), and had a longer initial length of stay (6.1 vs 5.3 days, P = .007) relative to their nonreadmitted counterparts. Readmitted patients experienced more postoperative deep vein thromboses or pulmonary embolisms (12% vs 4%, P = .006), new motor deficits (29% vs 14%, P = .002), and nonhome discharges (39% vs 22%, P = .005) relative to their nonreadmitted counterparts. Multivariable analysis demonstrated increased odds of 30-day readmission with each 10-point decrease in Karnofsky Performance Scale score (odds ratio [OR] 1.32, P = .002), each single-point increase in 5-factor modified frailty index (OR 1.51, P = .016), and initial presentation with cognitive deficits (OR 2.11, P = .013).

CONCLUSION

Preoperatively available clinical characteristics strongly predicted 30-day readmissions in patients undergoing surgery for GBM. Opportunities may exist to optimize preoperative and postoperative management of at-risk patients with GBM, with downstream improvements in clinical outcomes.

Derivation and validation of a 90-day unplanned hospital readmission score in older patients discharged form a geriatric ward

PURPOSE

To derive and validate a 90-day unplanned hospital readmission (UHR) score based on information available to non-hospital based care providers.

METHODS

Retrospective longitudinal study with cross-validation method. Participants were older adults (≥ 65 years) admitted to a geriatric short-stay department in a general hospital in France. Patients were split into a derivation cohort and a validation cohort. We recorded demographic information, medical history, and concurrent clinical characteristics. The main outcome was 90-day UHR. Data obtained from hospital discharge letters were used in a logistic regression model to construct a predictive score, and to identify risk groups for 90-day UHR.

RESULTS

In total, 750 and 250 aged adults were included in both the derivation and the validation cohorts. Mean age was 87.2 ± 5.2 years, most were women (68.1%). Independent risk factors for 90-day UHR were: use of mobility aids (p = .02), presence of dementia syndrome (p = .02), history of recent hospitalisation (p = .03), and discharge to domiciliary home (p = .005). From these four risk factors, three groups were determined: low-risk group (score < 4), medium-risk group (score between 4 and 6), and high-risk group (score ≥ 6). In the derivation cohort the 90-day UHR rates increased significantly across risk groups (14%, 22%, and 30%, respectively). The 90-day UHR score had the same discriminant power in the derivation cohort (c-statistic = 0.63) as in the validation cohort (c-statistic = 0.63).

CONCLUSIONS

This score makes it possible to identify aged adults at risk of 90-day UHR and to target multidisciplinary interventions to limit UHR for patients discharged from a Geriatric Short-Stay Unit.

Machine learning prediction of postoperative unplanned 30-day hospital readmission in older adult

Background: Although unplanned hospital readmission is an important indicator for monitoring the perioperative quality of hospital care, few published studies of hospital readmission have focused on surgical patient populations, especially in the elderly. We aimed to investigate if machine learning approaches can be used to predict postoperative unplanned 30-day hospital readmission in old surgical patients.

Methods: We extracted demographic, comorbidity, laboratory, surgical, and medication data of elderly patients older than 65 who underwent surgeries under general anesthesia in West China Hospital, Sichuan University from July 2019 to February 2021. Different machine learning approaches were performed to evaluate whether unplanned 30-day hospital readmission can be predicted. Model performance was assessed using the following metrics: AUC, accuracy, precision, recall, and F1 score. Calibration of predictions was performed using Brier Score. A feature ablation analysis was performed, and the change in AUC with the removal of each feature was then assessed to determine feature importance.

Results: A total of 10,535 unique surgeries and 10,358 unique surgical elderly patients were included. The overall 30-day unplanned readmission rate was 3.36%. The AUCs of the six machine learning algorithms predicting postoperative 30-day unplanned readmission ranged from 0.6865 to 0.8654. The RF + XGBoost algorithm overall performed the best with an AUC of 0.8654 (95% CI, 0.8484–0.8824), accuracy of 0.9868 (95% CI, 0.9834–0.9902), precision of 0.3960 (95% CI, 0.3854–0.4066), recall of 0.3184 (95% CI, 0.259–0.3778), and F1 score of 0.4909 (95% CI, 0.3907–0.5911). The Brier scores of the six machine learning algorithms predicting postoperative 30-day unplanned readmission ranged from 0.3721 to 0.0464, with RF + XGBoost showing the best calibration capability. The most five important features of RF + XGBoost were operation duration, white blood cell count, BMI, total bilirubin concentration, and blood glucose concentration.

Conclusion: Machine learning algorithms can accurately predict postoperative unplanned 30-day readmission in elderly surgical patients.

Acute Poisoning Readmissions to an Emergency Department of a Tertiary Hospital: Evaluation through an Active Toxicovigilance Program

The aim of this study is to investigate hospital readmissions during 1 year after acute poisoning cases (APC), analyze the temporal behavior of early readmissions (ER) (in the month after the index episode) and predict possible ER. A descriptive analysis of the patients with APC assisted between 2011 and 2016 in the Emergency Department of Hospital La Paz is presented, and various methods of inferential statistics were applied and confirmed by Bayesian analysis in order to evaluate factors associated with total and early readmissions. Out of the 4693 cases of APC included, 968 (20.6%) presented, at least one readmission and 476 (10.1%) of them were ER. The mean age of APC with readmission was 41 years (12.7 SD), 78.9% had previous psychiatric pathology and 44.7% had a clinical history of alcohol addiction. Accidental poisoning has been a protective factor for readmission (OR 0.50; 0.26–0.96). Type of toxin (“drug of abuse” OR 8.88; 1.17–67.25), history of addiction (OR 1.93; 1.18–3.10) and psychiatric history (OR 3.30; 2.53–4.30) are risk factors for readmissions during the first year. Women showed three or more readmissions in a year. The results of the study allow for identification of the predictors for the different numbers of readmissions in the year after the index APC, as well as for ERs.

Depressive symptoms and malnutrition are associated with other geriatric syndromes and increase risk for 30-Day readmission in hospitalized older adults: a prospective cohort study

BACKGROUND

Readmission in older adults is typically complex with multiple contributing factors. We aim to examine how two prevalent and potentially modifiable geriatric conditions - depressive symptoms and malnutrition - relate to other geriatric syndromes and 30-day readmission in hospitalized older adults.

METHODS

Consecutive admissions of patients ≥ 65 years to a general medical department were recruited over 16 months. Patients were screened for depression, malnutrition, delirium, cognitive impairment, and frailty at admission. Medical records were reviewed for poor oral intake and functional decline during hospitalization. Unplanned readmission within 30-days of discharge was tracked through the hospital's electronic health records and follow-up telephone interviews. We use directed acyclic graphs (DAGs) to depict the relationship of depressive symptoms and malnutrition with geriatric syndromes that constitute covariates of interest and 30-day readmission outcome. Multiple logistic regression was performed for the independent associations of depressive symptoms and malnutrition with 30-day readmission, adjusting for variables based on DAG-identified minimal adjustment set.

RESULTS

We recruited 1619 consecutive admissions, with mean age 76.4 (7.9) years and 51.3% females. 30-day readmission occurred in 331 (22.0%) of 1,507 patients with follow-up data. Depressive symptoms, malnutrition, higher comorbidity burden, hospitalization in the one-year preceding index admission, frailty, delirium, as well as functional decline and poor oral intake during the index admission, were more commonly observed among patients who were readmitted within 30 days of discharge (P < 0.05). Patients with active depressive symptoms were significantly more likely to be frail (OR = 1.62, 95% CI 1.22-2.16), had poor oral intake (OR = 1.35, 95% CI 1.02-1.79) and functional decline during admission (OR = 1.58, 95% CI 1.11-2.23). Malnutrition at admission was significantly associated with frailty (OR = 1.53, 95% CI 1.07-2.19), delirium (OR = 2.33, 95% CI 1.60-3.39) cognitive impairment (OR = 1.88, 95% CI 1.39-2.54) and poor oral intake during hospitalization (OR = 2.70, 95% CI 2.01-3.64). In minimal adjustment set identified by DAG, depressive symptoms (OR = 1.38, 95% CI 1.02-1.86) remained significantly associated with 30-day readmission. The association of malnutrition with 30-day readmission was no longer statistically significant after adjusting for age, ethnicity and depressive symptoms in the minimal adjustment set (OR = 1.40, 95% CI 0.99-1.98).

CONCLUSION

The observed causal associations support screening and targeted interventions for depressive symptoms and malnutrition during admission and in the post-acute period.

Medicare Claim-Based National Institutes of Health Stroke Scale to Predict 30-Day Mortality and Hospital Readmission

BACKGROUND

The Centers for Medicare and Medicaid Services (CMS) penalizes hospitals for higher than expected 30-day mortality rates using methods without accounting for condition severity risk adjustment. For patients with stroke, CMS claims did not quantify stroke severity until recently, when the National Institutes of Health Stroke Scale (NIHSS) reporting began.

OBJECTIVE

Examine the predictive ability of claim-based NIHSS to predict 30-day mortality and 30-day hospital readmission in patients with ischemic stroke.

DESIGN

Retrospective cohort study of Medicare claims data.

PATIENTS

Medicare beneficiaries with ischemic stroke (N=43,241) acute hospitalization between October 2016 and November 2017.

MEASUREMENTS

All-cause 30-day mortality and 30-day hospital readmission. NIHSS score was derived from ICD-10 codes and stratified into the following: minor to moderate, moderate, moderate to severe, and severe categories.

RESULTS

Among 43,241 patients with ischemic stroke with NIHSS from 2,659 US hospitals, 64.6% had minor to moderate stroke, 14.3% had moderate, 12.7% had moderate to severe, and 8.5% had a severe stroke,10.1% died within 30 days, 12.1% were readmitted within 30 days. The NIHSS exhibited stronger discriminant property (C-statistic 0.83, 95% CI: 0.82-0.84) for 30-day mortality compared to Elixhauser (0.74, 95% CI: 0.73-0.75). A monotonic increase in the adjusted 30-day mortality risk occurred relative to minor to moderate stroke category: hazard ratio [HR]=2.92 (95% CI=2.59-3.29) for moderate stroke, HR=5.49 (95% CI=4.90-6.15) for moderate to severe stroke, and HR=7.82 (95% CI=6.95-8.80) for severe stroke. After accounting for competing risk of mortality, there was a significantly higher readmission risk in the moderate stroke (HR=1.11, 95% CI=1.03-1.20), but significantly lower readmission risk in the severe stroke (HR=0.84, 95% CI=0.74-0.95) categories.

LIMITATION

Timing of NIHSS reporting during hospitalization is unknown.

CONCLUSIONS

Medicare claim-based NIHSS is significantly associated with 30-day mortality in Medicare patients with ischemic stroke and significantly improves discriminant property relative to the Elixhauser comorbidity index.

A Machine Learning Approach for Predicting Readmission Charges Billed by Hospitals

Background

The Centers for Medicare and Medicaid Services projects that health care costs will continue to grow over the next few years. Rising readmission costs contribute significantly to increasing health care costs. Multiple areas of health care, including readmissions, have benefited from the application of various machine learning algorithms in several ways.

Objective

We aimed to identify suitable models for predicting readmission charges billed by hospitals. Our literature review revealed that this application of machine learning is underexplored. We used various predictive methods, ranging from glass-box models (such as regularization techniques) to black-box models (such as deep learning–based models).

Methods

We defined readmissions as readmission with the same major diagnostic category (RSDC) and all-cause readmission category (RADC). For these readmission categories, 576,701 and 1,091,580 individuals, respectively, were identified from the Nationwide Readmission Database of the Healthcare Cost and Utilization Project by the Agency for Healthcare Research and Quality for 2013. Linear regression, lasso regression, elastic net, ridge regression, eXtreme gradient boosting (XGBoost), and a deep learning model based on multilayer perceptron (MLP) were the 6 machine learning algorithms we tested for RSDC and RADC through 10-fold cross-validation.

Results

Our preliminary analysis using a data-driven approach revealed that within RADC, the subsequent readmission charge billed per patient was higher than the previous charge for 541,090 individuals, and this number was 319,233 for RSDC. The top 3 major diagnostic categories (MDCs) for such instances were the same for RADC and RSDC. The average readmission charge billed was higher than the previous charge for 21 of the MDCs in the case of RSDC, whereas it was only for 13 of the MDCs in RADC. We recommend XGBoost and the deep learning model based on MLP for predicting readmission charges. The following performance metrics were obtained for XGBoost: (1) RADC (mean absolute percentage error [MAPE]=3.121%; root mean squared error [RMSE]=0.414; mean absolute error [MAE]=0.317; root relative squared error [RRSE]=0.410; relative absolute error [RAE]=0.399; normalized RMSE [NRMSE]=0.040; mean absolute deviation [MAD]=0.031) and (2) RSDC (MAPE=3.171%; RMSE=0.421; MAE=0.321; RRSE=0.407; RAE=0.393; NRMSE=0.041; MAD=0.031). The performance obtained for MLP-based deep neural networks are as follows: (1) RADC (MAPE=3.103%; RMSE=0.413; MAE=0.316; RRSE=0.410; RAE=0.397; NRMSE=0.040; MAD=0.031) and (2) RSDC (MAPE=3.202%; RMSE=0.427; MAE=0.326; RRSE=0.413; RAE=0.399; NRMSE=0.041; MAD=0.032). Repeated measures ANOVA revealed that the mean RMSE differed significantly across models with P<.001. Post hoc tests using the Bonferroni correction method indicated that the mean RMSE of the deep learning/XGBoost models was statistically significantly (P<.001) lower than that of all other models, namely linear regression/elastic net/lasso/ridge regression.

Conclusions

Models built using XGBoost and MLP are suitable for predicting readmission charges billed by hospitals. The MDCs allow models to accurately predict hospital readmission charges.

Decision support through risk cost estimation in 30-day hospital unplanned readmission

Unplanned hospital readmissions mean a significant burden for health systems. Accurately estimating the patient's readmission risk could help to optimise the discharge decision-making process by smartly ordering patients based on a severity score, thus helping to improve the usage of clinical resources. A great number of heterogeneous factors can influence the readmission risk, which makes it highly difficult to be estimated by a human agent. However, this score could be achieved with the help of AI models, acting as aiding tools for decision support systems. In this paper, we propose a machine learning classification and risk stratification approach to assess the readmission problem and provide a decision support system based on estimated patient risk scores.

Semiparametric Survival Analysis of 30-Day Hospital Readmissions with Bayesian Additive Regression Kernel Model

In this paper, we introduce a kernel-based nonlinear Bayesian model for a right-censored survival outcome data set. Our kernel-based approach provides a flexible nonparametric modeling framework to explore nonlinear relationships between predictors with right-censored survival outcome data. Our proposed kernel-based model is shown to provide excellent predictive performance via several simulation studies and real-life examples. Unplanned hospital readmissions greatly impair patients’ quality of life and have imposed a significant economic burden on American society. In this paper, we focus our application on predicting 30-day readmissions of patients. Our survival Bayesian additive regression kernel model (survival BARK or sBARK) improves the timeliness of readmission preventive intervention through a data-driven approach.

A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models

OBJECTIVE

Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model's potential to introduce bias.

MATERIALS AND METHODS

Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist.

RESULTS

We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern.

DISCUSSION

Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed.

CONCLUSION

The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

The Impact of Principal Diagnosis on Readmission Risk among Patients Hospitalized for Community-Acquired Pneumonia

Coding variation distorts performance/outcome statistics not eliminated by risk adjustment. Among 1596 community-acquired pneumonia patients hospitalized from 1998 to 2012 identified using an evidence-based algorithm, the authors measured the association of principal diagnosis (PD) with 30-day readmission, stratified by Pneumonia Severity Index risk class. The 152 readmitted patients were more ill (Pneumonia Severity Index class V 38.8% versus 25.8%) and less likely to have a pneumonia PD (52.6% versus 69.9%). Among patients with PDs of pneumonia, respiratory failure, sepsis, and aspiration, mortality/readmission rates were 3.9/8.5%, 28.8/14.0%, 24.7/19.6%, and 9.0/15.0%, respectively. The nonpneumonia PDs were associated with a greater risk of adjusted 30-day readmission: respiratory failure odds ratio (OR) 1.89 (95% confidence interval [CI], 1.13-3.15), sepsis OR 2.54 (95% CI, 1.52-4.26), and possibly aspiration OR 1.73 (95% CI, 0.88-3.41). With increasing use of alternative PDs among pneumonia patients, quality reporting must account for variations in condition coding practices. Rigorous risk adjustment does not eliminate the need for accurate, consistent case definition in producing valid quality measures.

Bounce-Back: Predicting Acute Readmission From Inpatient Rehabilitation for Patients With Stroke

OBJECTIVE

The aim of the study was to identify demographic, medical, and functional risk factors for discharge to an acute hospital before completion of an inpatient rehabilitation program and 7- and 30-day readmissions after completion of an inpatient rehabilitation program.

DESIGN

This cohort study included 138,063 fee-for-service Medicare beneficiaries with a primary diagnosis of new onset stroke discharged from an inpatient rehabilitation facility from June 2009 to December 2011. Multivariate models examined readmission outcomes and included data from 6 mos before onset of the stroke to 30 days after discharge from the inpatient rehabilitation facility.

RESULTS

In the acute discharge model (n = 9870), comorbidities and complications added risk, and the longer the stroke onset to admission to inpatient rehabilitation facility, the more likely discharge to the acute hospital. In the 7-day (n = 4755) and 30-day (n = 9861) readmission models, patients who were more complex with comorbidities, were black, or had managed care Medicare were more likely to have a readmission. Functional status played a role in all three models.

CONCLUSIONS

Results suggest that certain demographic, medical, and functional characteristics are associated differentially with rehospitalization after completion inpatient rehabilitation. The strongest model was the discharge to the acute hospital model with concordance statistic (c-statistic) of 0.87.

Readmission Prediction for Patients with Heterogeneous Medical History: A Trajectory-Based Deep Learning Approach

Hospital readmission refers to the situation where a patient is re-hospitalized with the same primary diagnosis within a specific time interval after discharge. Hospital readmission causes $26 billion preventable expenses to the U.S. health systems annually and often indicates suboptimal patient care. To alleviate those severe financial and health consequences, it is crucial to proactively predict patients’ readmission risk. Such prediction is challenging because the evolution of patients’ medical history is dynamic and complex. The state-of-the-art studies apply statistical models which use static predictors in a period, failing to consider patients’ heterogeneous medical history. Our approach – Trajectory-BAsed DEep Learning (TADEL) – is motivated to tackle the deficiencies of the existing approaches by capturing dynamic medical history. We evaluate TADEL on a five-year national Medicare claims dataset including 3.6 million patients per year over all hospitals in the United States, reaching an F1 score of 87.3% and an AUC of 88.4%. Our approach significantly outperforms all the state-of-the-art methods. Our findings suggest that health status factors and insurance coverage are important predictors for readmission. This study contributes to IS literature and analytical methodology by formulating the trajectory-based readmission prediction problem and developing a novel deep-learning-based readmission risk prediction framework. From a health IT perspective, this research delivers implementable methods to assess patients’ readmission risk and take early interventions to avoid potential negative consequences.

Re-admission following discharge from a Geriatric Evaluation and Management Unit: identification of risk factors

ObjectiveTo establish independent factors that influence the likelihood of re-admission within 30 days of discharge from a Geriatric Evaluation and Management Unit.MethodsAn observational prospective cohort design using clinical data extracted from the medical charts of eligible patients discharged from a tertiary public hospital Geriatric Evaluation and Management Unit between July 2017 and April 2019. Binary logistic regression was undertaken to determine variables that increased the likelihood of hospital re-admission (dependent variable).ResultsA total of 367 patients were eligible for inclusion, with 69 patients re-admitted within 30 days of discharge. Univariate analysis demonstrated significant differences between groups (re-admission vs non-re-admission) with respect to Charlson Comorbidity Index (CCI) (7.4 [2.4] vs 6.3 [2.2], P = 0.001), Clinical Frailty Scale (CFS) (5.6 [1.1] vs 5.2 [1.34], P = 0.02), and documented malnourishment (36.2% vs 23.6%, P = 0.04). All three variables remained significant when entered into the regression model (X2 = 25.095, P < 0.001). A higher score for the CFS (OR 1.3; 95% CI 1.03-1.64; P = 0.03) and CCI (OR 1.2; 95% CI 1.06-1.33; P = 0.004), and documented malnourishment (OR 1.92; 95% CI 1.06-3.47; P = 0.03) were all independent factors that increased the likelihood of patient re-admission within 30 days of discharge.ConclusionsThis study supports the formal inclusion of the CCI and CFS into routine practice in Geriatric Evaluation and Management Units. The inclusion of the measures can help inform future discharge planning practices. Clinicians should use malnourishment status, CCI and CFS to identify at risk patients and target discharge planning interventions accordingly.

Reproducibility of prediction models in health services research

The field of health services research studies the health care system by examining outcomes relevant to patients and clinicians but also health economists and policy makers. Such outcomes often include health care spending, and utilization of care services. Building accurate prediction models using reproducible research practices for health services research is important for evidence-based decision making. Several systematic reviews have summarized prediction models for outcomes relevant to health services research, but these systematic reviews do not present a thorough assessment of reproducibility and research quality of the prediction modelling studies. In the present commentary, we discuss how recent advances in prediction modelling in other medical fields can be applied to health services research. We also describe the current status of prediction modelling in health services research, and we summarize available methodological guidance for the development, update, external validation and systematic appraisal of prediction models.

Development and validation of a social vulnerabilities survey for medical inpatients

OBJECTIVES

Our objective was to validate a Social Vulnerabilities Survey that was developed to identify patient barriers in the following domains: (1) salience or priority of health; (2) social support; (3) transportation; and (4) finances.

DESIGN

Cross-sectional psychometric study.Questions for one domain (health salience) were developed de novo while questions for the other domains were derived from national surveys and/or previously validated questionnaires. We tested construct (ie, convergent and discriminative) validity for these new questions through hypothesis testing of correlations between question responses and patient characteristics. Exploratory factor analysis was conducted to determine structural validity of the survey as a whole.

SETTING

Patients admitted to the inpatient internal medicine service at a tertiary care hospital in Calgary, Canada.

PARTICIPANTS

A total of 406 patients were included in the study.

RESULTS

The mean age of respondents was 55.5 (SD 18.6) years, with the majority being men (55.4%). In feasibility testing of the first 107 patients, the Social Vulnerabilities Survey was felt to be acceptable, comprehensive and met face validity. Hypothesis testing of the health salience questions revealed that the majority of observed correlations were exactly as predicted. Exploratory factor analysis of the global survey revealed the presence of five factors (eigenvalue >1): social support, health salience, drug insurance, transportation barriers and drug costs. All but four questions loaded to these five factors.

CONCLUSIONS

The Social Vulnerabilities Survey has face, construct and structural validity. It can be used to measure modifiable social vulnerabilities, such that their effects on health outcomes can be explored and understood.

Forecasting Hospital Readmissions with Machine Learning

Hospital readmissions are regarded as a compounding economic factor for healthcare systems. In fact, the readmission rate is used in many countries as an indicator of the quality of services provided by a health institution. The ability to forecast patients’ readmissions allows for timely intervention and better post-discharge strategies, preventing future life-threatening events, and reducing medical costs to either the patient or the healthcare system. In this paper, four machine learning models are used to forecast readmissions: support vector machines with a linear kernel, support vector machines with an RBF kernel, balanced random forests, and weighted random forests. The dataset consists of 11,172 actual records of hospitalizations obtained from the General Hospital of Komotini “Sismanogleio” with a total of 24 independent variables. Each record is composed of administrative, medical-clinical, and operational variables. The experimental results indicate that the balanced random forest model outperforms the competition, reaching a sensitivity of 0.70 and an AUC value of 0.78.

Predictivity of the comorbidity indices for geriatric syndromes

Background

The aging population and increasing chronic diseases make a tremendous burden on the health care system. The study evaluated the relationship between comorbidity indices and common geriatric syndromes.

Methods

A total of 366 patients who were hospitalized in a university geriatric inpatient service were included in the study. Sociodemographic characteristics, laboratory findings, and comprehensive geriatric assessment(CGA) parameters were recorded. Malnutrition, urinary incontinence, frailty, polypharmacy, falls, orthostatic hypotension, depression, and cognitive performance were evaluated. Comorbidities were ranked using the Charlson Comorbidity Index(CCI), Elixhauser Comorbidity Index(ECM), Geriatric Index of Comorbidity(GIC), and Medicine Comorbidity Index(MCI). Because, the CCI is a valid and reliable tool used in different clinical settings and diseases, patients with CCI score higher than four was accepted as multimorbid. Additionally, the relationship between geriatric syndromes and comorbidity indices was assessed with regression analysis.

Results

Patients’ mean age was 76.2 ± 7.25 years(67.8% female). The age and sex of multimorbid patients according to the CCI were not different compared to others. The multimorbid group had a higher rate of dementia and polypharmacy among geriatric syndromes. All four indices were associated with frailty and polypharmacy(p < 0.05). CCI and ECM scores were related to dementia, polypharmacy, and frailty. Moreover, CCI was also associated with separately slow walking speed and low muscle strength. On the other hand, unlike CCI, ECM was associated with malnutrition.

Conclusions

In the study comparing the four comorbidity indices, it is revealed that none of the indices is sufficient to use alone in geriatric practice. New indices should be developed considering the complexity of the geriatric cases and the limitations of the existing indices.

Analysis of Readmissions in a Mobile Integrated Health Transitional Care Program Using Root Cause Analysis and Common Cause Analysis

ABSTRACT

Mobile integrated health and community paramedicine (MIH-CP) programs are gaining popularity in the United States as a strategy to address the barriers to healthcare access and appropriate utilization. After one year of operation, leadership of Baltimore City's MIH-CP program was interested in understanding the circumstances surrounding readmission for enrolled patients and to incorporate quality improvement tools to direct program development. Retrospective chart review was performed to determine preventable versus unpreventable readmissions with a hypothesis that deficits in social determinants of health would play a more significant role in preventable readmissions. In the studied population, at least one root cause that can be considered a social determinant of health was present in 75.8% of preventable readmissions versus only 15.2% of unpreventable readmissions. Root Cause Analysis highlighted health literacy, functional status, and behavioral health issues among the root causes that most heavily influence preventable readmissions. Common Cause Analysis results suggest our MIH-CP program should focus its resources on mitigating poor health literacy and functional status. This project's findings successfully directed leadership of the city's MIH-CP program to modify program processes and advocate for the use of these quality improvement tools for other MIH-CP programs.

Identifying prognostic factors for clinical outcomes and costs in four high-volume surgical treatments using routinely collected hospital data

Identifying prognostic factors (PFs) is often costly and labor-intensive. Routinely collected hospital data provide opportunities to identify clinically relevant PFs and construct accurate prognostic models without additional data-collection costs. This multicenter (66 hospitals) study reports on associations various patient-level variables have with outcomes and costs. Outcomes were in-hospital mortality, intensive care unit (ICU) admission, length of stay, 30-day readmission, 30-day reintervention and in-hospital costs. Candidate PFs were age, sex, Elixhauser Comorbidity Score, prior hospitalizations, prior days spent in hospital, and socio-economic status. Included patients dealt with either colorectal carcinoma (CRC, n = 10,254), urinary bladder carcinoma (UBC, n = 17,385), acute percutaneous coronary intervention (aPCI, n = 25,818), or total knee arthroplasty (TKA, n = 39,214). Prior hospitalization significantly increased readmission risk in all treatments (OR between 2.15 and 25.50), whereas prior days spent in hospital decreased this risk (OR between 0.55 and 0.95). In CRC patients, women had lower risk of in-hospital mortality (OR 0.64), ICU admittance (OR 0.68) and 30-day reintervention (OR 0.70). Prior hospitalization was the strongest PF for higher costs across all treatments (31–64% costs increase/hospitalization). Prognostic model performance (c-statistic) ranged 0.67–0.92, with Brier scores below 0.08. R-squared ranged from 0.06–0.19 for LoS and 0.19–0.38 for costs. Identified PFs should be considered as building blocks for treatment-specific prognostic models and information for monitoring patients after surgery. Researchers and clinicians might benefit from gaining a better insight into the drivers behind (costs) prognosis.

Applicability of predictive models for 30-day unplanned hospital readmission risk in paediatrics: a systematic review

OBJECTIVES

To summarise multivariable predictive models for 30-day unplanned hospital readmissions (UHRs) in paediatrics, describe their performance and completeness in reporting, and determine their potential for application in practice.

DESIGN

Systematic review.

DATA SOURCE

CINAHL, Embase and PubMed up to 7 October 2021.

ELIGIBILITY CRITERIA

English or German language studies aiming to develop or validate a multivariable predictive model for 30-day paediatric UHRs related to all-cause, surgical conditions or general medical conditions were included.

DATA EXTRACTION AND SYNTHESIS

Study characteristics, risk factors significant for predicting readmissions and information about performance measures (eg, c-statistic) were extracted. Reporting quality was addressed by the 'Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis' (TRIPOD) adherence form. The study quality was assessed by applying six domains of potential biases. Due to expected heterogeneity among the studies, the data were qualitatively synthesised.

RESULTS

Based on 28 studies, 37 predictive models were identified, which could potentially be used for determining individual 30-day UHR risk in paediatrics. The number of study participants ranged from 190 children to 1.4 million encounters. The two most common significant risk factors were comorbidity and (postoperative) length of stay. 23 models showed a c-statistic above 0.7 and are primarily applicable at discharge. The median TRIPOD adherence of the models was 59% (P₂₅-P₇₅, 55%-69%), ranging from a minimum of 33% to a maximum of 81%. Overall, the quality of many studies was moderate to low in all six domains.

CONCLUSION

Predictive models may be useful in identifying paediatric patients at increased risk of readmission. To support the application of predictive models, more attention should be placed on completeness in reporting, particularly for those items that may be relevant for implementation in practice.

Does a screening checklist for complex health and social care needs have potential clinical usefulness for predicting unplanned hospital readmissions in intensive care survivors: development and prospective cohort study

OBJECTIVES

Intensive care (ICU) survivors are at high risk of long-term physical and psychosocial problems. Unplanned hospital readmission rates are high, but the best way to triage patients for interventions is uncertain. We aimed to develop and evaluate a screening checklist to help predict subsequent readmissions or deaths.

DESIGN

A checklist for complex health and social care needs (CHSCNs) was developed based on previous research, comprising six items: multimorbidity; polypharmacy; frequent previous hospitalisations; mental health issues; fragile social circumstances and impaired activities of daily living. Patients were considered to have CHSCNs if two or more were present. We prospectively screened all ICU discharges for CHSCNs for 12 months.

SETTING

ICU, Royal Infirmary, Edinburgh, UK.

PARTICIPANTS

ICU survivors over a 12-month period (1 June 2018 and 31 May 2019).

INTERVENTIONS

None.

OUTCOME MEASURE

Readmission or death in the community within 3 months postindex hospital discharge.

RESULTS

Of 1174 ICU survivors, 937 were discharged alive from the hospital. Of these 253 (27%) were classified as having CHSCNs. In total 28% (266/937) patients were readmitted (N=238) or died (N=28) within 3 months. Among CHSCNs patients 45% (n=115) patients were readmitted (N=105) or died (N=10). Patients without CHSCNs had a 22% readmission (N=133) or death (N=18) rate. The checklist had: sensitivity 43% (95% CI 37% to 49%), specificity 79% (95% CI 76% to 82%), positive predictive value 45% (95% CI 41% to 51%), and negative predictive value 78% (95% CI 76% to 80%). Relative risk of readmission/death for patients with CHSCNs was 2.06 (95% CI 1.69 to 2.50), indicating a pretest to post-test probability change of 28%-45%. The checklist demonstrated high inter-rater reliability (percentage agreement ≥87% for all domains; overall kappa, 0.84).

CONCLUSIONS

Early evaluation of a screening checklist for CHSCNs at ICU discharge suggests potential clinical usefulness, but this requires further evaluation as part of a care pathway.

Intravenous antibiotics at the index emergency department visit as an independent risk factor for hospital admission at the return visit within 72 hours

Introduction

Although infection was the most common symptom in patients returning to the ED, whether intravenous antibiotic administration at the index visit could serve as an indicator of patients with infectious diseases at high risk for hospital admission after returning to the ED within a short period of time remains unclear. The study aimed to investigate the potential risk factors for hospital admission in patients returning to the ED within 72 hours with a final diagnosis of infectious diseases.

Material and methods

This retrospective cohort study analyzed return visits to the ED from January to December 2019. Adult patients aged >20 years who had a return visit to the ED within 72 hours with an infectious disease were included herein. In total, 715 eligible patients were classified into the intravenous antibiotics and non-intravenous antibiotics group (reference group). The outcome studied was hospital admission to general ward and intensive care unit (ICU) at the return visits.

Results

Patients receiving intravenous antibiotics at index visits had significantly higher risk—approximately two times—for hospital admission at the return visits than those did not (adjusted odds ratio = 2.47, 95% CI = 1.34–4.57, p = 0.004). For every 10 years increase in age, the likelihood for hospital admission increased by 38%. Other factors included abnormal respiratory rate and high C-reactive protein levels.

Conclusions

Intravenous antibiotic administration at the index visit was an independent risk factor for hospital admission at return visits in patients with an infection disease. Physicians should consider carefully before discharging patients receiving intravenous antibiotics.

Heterogeneity in Longitudinal Healthcare Utilisation by Older Adults: A Latent Transition Analysis of the Irish Longitudinal Study on Ageing

BACKGROUND

Older adults likely exhibit considerable differences in healthcare need and usage. Identifying differences in healthcare utilisation both between and within individuals over time may support future service development.

OBJECTIVES

To characterise temporal changes in healthcare utilisation among a nationally representative sample of community-dwelling older adults.

METHODS

A latent transition analysis of the first three waves of The Irish Longitudinal Study on Ageing (TILDA) (N = 6128) was conducted.

RESULTS

Three latent classes of healthcare utilisation were identified, 'primary care only'; 'primary care and outpatient visits' and 'multiple utilisation'. The classes were invariant across all three waves. Transition probabilities indicated dynamic changes over time, particularly for the 'primary care and outpatient visits' and 'multiple utilisation' statuses.

DISCUSSION

Older adults exhibit temporal changes in healthcare utilisation which may reflect changes in healthcare need and disease progression. Further research is required to identify the factors which influence movement between healthcare utilisation patterns.

Development and validation of an interpretable machine learning scoring tool for estimating time to emergency readmissions

BACKGROUND

Emergency readmission poses an additional burden on both patients and healthcare systems. Risk stratification is the first step of transitional care interventions targeted at reducing readmission. To accurately predict the short- and intermediate-term risks of readmission and provide information for further temporal risk stratification, we developed and validated an interpretable machine learning risk scoring system.

METHODS

In this retrospective study, all emergency admission episodes from January 1st 2009 to December 31st 2016 at a tertiary hospital in Singapore were assessed. The primary outcome was time to emergency readmission within 90 days post discharge. The Score for Emergency ReAdmission Prediction (SERAP) tool was derived via an interpretable machine learning-based system for time-to-event outcomes. SERAP is six-variable survival score, and takes the number of emergency admissions last year, age, history of malignancy, history of renal diseases, serum creatinine level, and serum albumin level during index admission into consideration.

FINDINGS

A total of 293,589 ED admission episodes were finally included in the whole cohort. Among them, 203,748 episodes were included in the training cohort, 50,937 episodes in the validation cohort, and 38,904 in the testing cohort. Readmission within 90 days was documented in 80,213 (27.3%) episodes, with a median time to emergency readmission of 22 days (Interquartile range: 8-47). For different time points, the readmission rates observed in the whole cohort were 6.7% at 7 days, 10.6% at 14 days, 13.6% at 21 days, 16.4% at 30 days, and 23.0% at 60 days. In the testing cohort, the SERAP achieved an integrated area under the curve of 0.737 (95% confidence interval: 0.730-0.743). For a specific 30-day readmission prediction, SERAP outperformed the LACE index (Length of stay, Acuity of admission, Charlson comorbidity index, and Emergency department visits in past six months) and the HOSPITAL score (Hemoglobin at discharge, discharge from an Oncology service, Sodium level at discharge, Procedure during the index admission, Index Type of admission, number of Admissions during the last 12 months, and Length of stay). Besides 30-day readmission, SERAP can predict readmission rates at any time point during the 90-day period.

INTERPRETATION

Better performance in risk prediction was achieved by the SERAP than other existing scores, and accurate information about time to emergency readmission was generated for further temporal risk stratification and clinical decision-making. In the future, external validation studies are needed to evaluate the SERAP at different settings and assess their real-world performance.

FUNDING

This study was supported by the Singapore National Medical Research Council under the PULSES Center Grant, and Duke-NUS Medical School.

Scoring systems for the management of oncological hepato-pancreato-biliary patients

Oncological scoring systems in surgery are used as evidence-based decision aids to best support management through assessing prognosis, effectiveness and recurrence. Currently, the use of scoring systems in the hepato-pancreato-biliary (HPB) field is limited as concerns over precision and applicability prevent their widespread clinical implementation. The aim of this review was to discuss clinically useful oncological scoring systems for surgical management of HPB patients. A narrative review was conducted to appraise oncological HPB scoring systems. Original research articles of established and novel scoring systems were searched using Google Scholar, PubMed, Cochrane, and Ovid Medline. Selected models were determined by authors. This review discusses nine scoring systems in cancers of the liver (CLIP, BCLC, ALBI Grade, RETREAT, Fong's score), pancreas (Genç's score, mGPS), and biliary tract (TMHSS, MEGNA). Eight models used exclusively objective measurements to compute their scores while one used a mixture of both subjective and objective inputs. Seven models evaluated their scoring performance in external populations, with reported discriminatory c-statistic ranging from 0.58 to 0.82. Selection of model variables was most frequently determined using a combination of univariate and multivariate analysis. Calibration, another determinant of model accuracy, was poorly reported amongst nine scoring systems. A diverse range of HPB surgical scoring systems may facilitate evidence-based decisions on patient management and treatment. Future scoring systems need to be developed using heterogenous patient cohorts with improved stratification, with future trends integrating machine learning and genetics to improve outcome prediction.

Risk Stratification Model for Predicting Coronary Care Unit Readmission

Background

Use of statistical models for assessing the clinical risk of readmission to medical and surgical intensive care units is well established. However, models for predicting risk of coronary care unit (CCU) readmission are rarely reported. Therefore, this study investigated the characteristics and outcomes of patients readmitted to CCU to identify risk factors for CCU readmission and to establish a scoring system for identifying patients at high risk for CCU readmission.

Methods

Medical data were collected for 27,841 patients with a history of readmission to the CCU of a single multi-center healthcare provider in Taiwan during 2001-2019. Characteristics and outcomes were compared between a readmission group and a non-readmission group. Data were segmented at a 9:1 ratio for model building and validation.

Results

The number of patients with a CCU readmission history after transfer to a standard care ward was 1,790 (6.4%). The eleven factors that had the strongest associations with CCU readmission were used to develop and validate a CCU readmission risk scoring and prediction model. When the model was used to predict CCU readmission, the receiver-operating curve characteristic was 0.7038 for risk score model group and 0.7181 for the validation group. A CCU readmission risk score was assigned to each patient. The patients were then stratified by risk score into low risk (0–12), moderate risk (13–31) and high risk (32–40) cohorts check scores, which showed that CCU readmission risk significantly differed among the three groups.

Conclusions

This study developed a model for estimating CCU readmission risk. By using the proposed model, clinicians can improve CCU patient outcomes and medical care quality.

AI Models for Predicting Readmission of Pneumonia Patients within 30 Days after Discharge

A model with capability for precisely predicting readmission is a target being pursued worldwide. The objective of this study is to design predictive models using artificial intelligence methods and data retrieved from the National Health Insurance Research Database of Taiwan for identifying high-risk pneumonia patients with 30-day all-cause readmissions. An integrated genetic algorithm (GA) and support vector machine (SVM), namely IGS, were used to design predictive models optimized with three objective functions. In IGS, GA was used for selecting salient features and optimal SVM parameters, while SVM was used for constructing the models. For comparison, logistic regression (LR) and deep neural network (DNN) were also applied for model construction. The IGS model with AUC used as the objective function achieved an accuracy, sensitivity, specificity, and area under ROC curve (AUC) of 70.11%, 73.46%, 69.26%, and 0.7758, respectively, outperforming the models designed with LR (65.77%, 78.44%, 62.54%, and 0.7689, respectively) and DNN (61.50%, 79.34%, 56.95%, and 0.7547, respectively), as well as previously reported models constructed using thedata of electronic health records with an AUC of 0.71–0.74. It can be used for automatically detecting pneumonia patients with a risk of all-cause readmissions within 30 days after discharge so as to administer suitable interventions to reduce readmission and healthcare costs.

Human Factors Considerations in Transitions in Care Clinical Decision Support System Implementation Studies

Objective: Review transitions in care clinical decision support system (CDSS) implementation studies and describe human factors considerations in users, design, alert types, intervention timing, and implementation outcomes. Methods: Literature review in PubMed guided by subject matter experts. Results: Twelve articles were included. Targeted users included physicians, nurses, pharmacists, or interdisciplinary teams. Alerts were deployed via email, cloud-based software, or the EHR in inpatient and/or outpatient settings. Outcome measures varied across articles, with mixed performance. There were six readmissions-focused, two prescribing, one laboratory, two prescribing and laboratory, and one discharge disposition CDSS. Few articles reported statistically significant differences in outcomes, and many reported alert fatigue. Discussion and Conclusion: Despite the increasing prevalence of CDSS for transitions in care, few articles describe implementation processes and outcomes, and evidence of clinical practice improvement is mixed. Future studies should utilize implementation science frameworks and incorporate appropriate implementation outcomes in addition to traditional clinical outcomes like readmission rates.

Predicting Clinical Events Based on Raw Text: From Bag-of-Words to Attention-Based Transformers

Identifying which patients are at higher risks of dying or being re-admitted often happens to be resource- and life- saving, thus is a very important and challenging task for healthcare text analytics. While many successful approaches exist to predict such clinical events based on categorical and numerical variables, a large amount of health records exists in the format of raw text such as clinical notes or discharge summaries. However, the text-analytics models applied to free-form natural language found in those notes are lagging behind the break-throughs happening in the other domains and remain to be primarily based on older bag-of-words technologies. As a result, they rarely reach the accuracy level acceptable for the clinicians. In spite of their success in other domains, the superiority of deep neural approaches over classical bags of words for this task has not yet been convincingly demonstrated. Also, while some successful experiments have been reported, the most recent break-throughs due to the pre-trained language models have not yet made their ways into the medical domain. Using a publicly available healthcare dataset, we have explored several classification models to predict patients' re-admission or a fatality based on their discharge summaries and established that 1) The performance of the neural models used in our experiments convincingly exceeds those based on bag-of-words by several percentage points as measured by the standard metrics. 2) This allows us to achieve the accuracy typically acceptable by the clinicians as of practical use (area under the ROC curve above 0.70) for the majority of our prediction targets. 3) While the pre-trained attention-based transformer performed only on par with the model that averages word embeddings when applied to full length discharge summaries, the transformer still handles shorter text segments substantially better, at times with the margin of 0.04 in the area under the ROC curve. Thus, our findings extend the success of pre-trained language models reported in other domains to the task of clinical event prediction, and likely to other text-classification tasks in the healthcare analytics domain. 4) We suggest several models to overcome the transformers' major drawback (their input size limitation), and confirm that this is crucial to achieve their top performance. Our modifications are domain agnostic, and thus can be applied in other applications where the text inputs exceed 200 words. 5) We have successfully demonstrated how non-text attributes (such as patient age, demographics, type of admission etc.) can be combined with text to gain additional improvements for several prediction targets. We include extensive ablation studies showing the impact of the training size, and highlighting the tradeoffs between the performance and the resources needed.

Decongestion, kidney injury and prognosis in patients with acute heart failure

BACKGROUND

In patients with acute heart failure (AHF), the development of worsening renal function with appropriate decongestion is thought to be a benign functional change and not associated with poor prognosis. We investigated whether the benefit of decongestion outweighs the risk of concurrent kidney tubular damage and leads to better outcomes.

METHODS

We retrospectively analyzed data from the AKINESIS study, which enrolled AHF patients requiring intravenous diuretic therapy. Urine neutrophil gelatinase-associated lipocalin (uNGAL) and B-type natriuretic peptide (BNP) were serially measured during the hospitalization. Decongestion was defined as ≥30% BNP decrease at discharge compared to admission. Univariable and multivariable Cox models were assessed for one-year mortality.

RESULTS

Among 736 patients, 53% had ≥30% BNP decrease at discharge. Levels of uNGAL and BNP at each collection time point had positive but weak correlations (r ≤ 0.133). Patients without decongestion and with higher discharge uNGAL values had worse one-year mortality, while those with decongestion had better outcomes regardless of uNGAL values (p for interaction 0.018). This interaction was also significant when the change in BNP was analyzed as a continuous variable (p < 0.001). Although higher peak and discharge uNGAL were associated with mortality in univariable analysis, only ≥30% BNP decrease was a significant predictor after multivariable adjustment.

CONCLUSIONS

Among AHF patients treated with diuretic therapy, decongestion was generally not associated with kidney tubular damage assessed by uNGAL. Kidney tubular damage with adequate decongestion does not impact outcomes; however, kidney injury without adequate decongestion is associated with a worse prognosis.

The Potential For Bias In Machine Learning And Opportunities For Health Insurers To Address It

As the use of machine learning algorithms in health care continues to expand, there are growing concerns about equity, fairness, and bias in the ways in which machine learning models are developed and used in clinical and business decisions. We present a guide to the data ecosystem used by health insurers to highlight where bias can arise along machine learning pipelines. We suggest mechanisms for identifying and dealing with bias and discuss challenges and opportunities to increase fairness through analytics in the health insurance industry.

Coronavirus Disease 2019 and Hospital Readmissions: Patient Characteristics and Socioeconomic Factors Associated With Readmissions in an Urban Safety-Net Hospital System

BACKGROUND

It is not yet known whether socioeconomic factors (ie, social determinants of health) are associated with readmission following hospitalization for coronavirus disease 2019 (COVID-19).

METHODS

We conducted a retrospective cohort study of 6191 adult patients hospitalized with COVID-19 in a large New York City safety-net hospital system between March 1 and June 1, 2020. Associations between 30-day readmission and selected demographic characteristics, socioeconomic factors, prior health care utilization, and relevant features of the index hospitalization were analyzed using a multivariable generalized estimating equation model.

RESULTS

The readmission rate was 7.3%, with a median of 7 days between discharge and readmission. The following were risk factors for readmission: age 65 and older [adjusted odds ratio (aOR): 1.32; 95% confidence interval (CI): 1.13-1.55], history of homelessness, (aOR: 2.03 95% CI: 1.49-2.77), baseline coronary artery disease (aOR: 1.68; 95% CI: 1.34-2.10), congestive heart failure (aOR: 1.34; 95% CI: 1.20-1.49), cancer (aOR: 1.68; 95% CI: 1.26-2.24), chronic kidney disease (aOR: 1.74; 95% CI: 1.46-2.07). Patients' sex, race/ethnicity, insurance, and presence of obesity were not associated with increased odds of readmission. A longer length of stay (aOR: 0.98; 95% CI: 0.97-1.00) and use of noninvasive supplemental oxygen (aOR: 0.68; 95% CI: 0.56-0.83) was associated with lower odds of readmission. Upon readmission, 18.4% of patients required intensive care, and 13.7% expired.

CONCLUSION

We have found some factors associated with increased odds of readmission among patients hospitalized with COVID-19. Awareness of these risk factors, including patients' social determinants of health, may ultimately help to reduce readmission rates.

Real-Time Risk Tool for Pharmacy Interventions

Background

Adverse drug events (ADEs) result in excess hospitalizations. Thorough admission medication histories (AMHs) may prevent ADEs; however, the resources required oftentimes outweigh what is available in large hospital settings. Previous risk prediction models embedded into the Electronic Medical Record (EMR) have been used at hospitals to aid in targeting delivery of scarce resources.

Objective

To determine if an AMH scoring tool used to allocate resources can decrease 30-day hospital readmissions.

Design Setting and Participants

Propensity-matched cohort study, Medicine/Surgery patients in large academic safety-net hospital.

Intervention or Exposure

Pharmacy-conducted AMHs identified by risk model versus standard of care AMH.

Main Outcomes and Measures

A total of 30-day hospital readmissions and inpatient ADE prevention.

Results

The model screened 87 240 hospitalizations between June 2017 and June 2019 and 4027 patients per group were included. There were significantly less 30 day readmissions among high-risk identified patients that received a pharmacy-conducted AMH compared to controls (11% vs 15%; P = 0.004) and no significant difference in readmission rates for low-risk patients. While there was significantly higher documentation of major ADE prevention in the pharmacy-led AMH group versus control (1656 vs 12; P < 0.001), there was no difference in electronically-detected inpatient ADEs between groups.

Conclusions

A risk tool embedded into the EMR can be used to identify patients whom pharmacy teams can easily target for AMHs. This study showed significant reductions in readmissions for patients identified as high-risk. However, the same benefit in readmissions was not seen in those identified at low-risk, which supports allocating resources to those that will benefit the most.

Predictive Model for ICU Readmission Based on Discharge Summaries Using Machine Learning and Natural Language Processing

Predicting ICU readmission risk will help physicians make decisions regarding discharge. We used discharge summaries to predict ICU 30-day readmission risk using text mining and machine learning (ML) with data from the Medical Information Mart for Intensive Care III (MIMIC-III). We used Natural Language Processing (NLP) and the Bag-of-Words approach on discharge summaries to build a Document-Term-Matrix with 3000 features. We compared the performance of support vector machines with the radial basis function kernel (SVM-RBF), adaptive boosting (AdaBoost), quadratic discriminant analysis (QDA), least absolute shrinkage and selection operator (LASSO), and Ridge Regression. A total of 4000 patients were used for model training and 6000 were used for validation. Using the bag-of-words determined by NLP, the area under the receiver operating characteristic (AUROC) curve was 0.71, 0.68, 0.65, 0.69, and 0.65 correspondingly for SVM-RBF, AdaBoost, QDA, LASSO, and Ridge Regression. We then used the SVM-RBF model for feature selection by incrementally adding features to the model from 1 to 3000 bag-of-words. Through this exhaustive search approach, only 825 features (words) were dominant. Using those selected features, we trained and validated all ML models. The AUROC curve was 0.74, 0.69, 0.67, 0.70, and 0.71 respectively for SVM-RBF, AdaBoost, QDA, LASSO, and Ridge Regression. Overall, this technique could predict ICU readmission relatively well.

Digital Biomarkers for Supporting Transitional Care Decisions: Protocol for a Transnational Feasibility Study

Background

Virtual Health and Wellbeing Living Lab Infrastructure is a Horizon 2020 project that aims to harmonize Living Lab procedures and facilitate access to European health and well-being research infrastructures. In this context, this study presents a joint research activity that will be conducted within Virtual Health and Wellbeing Living Lab Infrastructure in the transitional care domain to test and validate the harmonized Living Lab procedures and infrastructures. The collection of data from various sources (information and communications technology and clinical and patient-reported outcome measures) demonstrated the capacity to assess risk and support decisions during care transitions, but there is no harmonized way of combining this information.

Objective

This study primarily aims to evaluate the feasibility and benefit of collecting multichannel data across Living Labs on the topic of transitional care and to harmonize data processes and collection. In addition, the authors aim to investigate the collection and use of digital biomarkers and explore initial patterns in the data that demonstrate the potential to predict transition outcomes, such as readmissions and adverse events.

Methods

The current research protocol presents a multicenter, prospective, observational cohort study that will consist of three phases, running consecutively in multiple sites: a cocreation phase, a testing and simulation phase, and a transnational pilot phase. The cocreation phase aims to build a common understanding among different sites, investigate the differences in hospitalization discharge management among countries, and the willingness of different stakeholders to use technological solutions in the transitional care process. The testing and simulation phase aims to explore ways of integrating observation of a patient’s clinical condition, patient involvement, and discharge education in transitional care. The objective of the simulation phase is to evaluate the feasibility and the barriers faced by health care professionals in assessing transition readiness.

Results

The cocreation phase will be completed by April 2022. The testing and simulation phase will begin in September 2022 and will partially overlap with the deployment of the transnational pilot phase that will start in the same month. The data collection of the transnational pilots will be finalized by the end of June 2023. Data processing is expected to be completed by March 2024. The results will consist of guidelines and implementation pathways for large-scale studies and an analysis for identifying initial patterns in the acquired data.

Conclusions

The knowledge acquired through this research will lead to harmonized procedures and data collection for Living Labs that support transitions in care.

International Registered Report Identifier (IRRID)

PRR1-10.2196/34573

Machine learning for predicting readmission risk among the frail: Explainable AI for healthcare

Healthcare costs due to unplanned readmissions are high and negatively affect health and wellness of patients. Hospital readmission is an undesirable outcome for elderly patients. Here, we present readmission risk prediction using five machine learning approaches for predicting 30-day unplanned readmission for elderly patients (age ≥ 50 years). We use a comprehensive and curated set of variables that include frailty, comorbidities, high-risk medications, demographics, hospital, and insurance utilization to build these models. We conduct a large-scale study with electronic health record (her) data with over 145,000 observations from 76,000 patients. Findings indicate that the category boost (CatBoost) model outperforms other models with a mean area under the curve (AUC) of 0.79. We find that prior readmissions, discharge to a rehabilitation facility, length of stay, comorbidities, and frailty indicators were all strong predictors of 30-day readmission. We present in-depth insights using Shapley additive explanations (SHAP), the state of the art in machine learning explainability.

A review of literature on risk prediction tools for hospital readmissions in older adults

PURPOSE

The purpose of this paper is to identify and analyse the readmission risk prediction tools reported in the literature and their benefits when it comes to healthcare organisations and management.

DESIGN/METHODOLOGY/APPROACH

Readmission risk prediction is a growing topic of interest with the aim of identifying patients in particular those suffering from chronic diseases such as congestive heart failure, chronic obstructive pulmonary disease and diabetes, who are at risk of readmission. Several models have been developed with different levels of predictive ability. A structured and extensive literature search of several databases was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-analysis strategy, and this yielded a total of 48,984 records.

FINDINGS

Forty-three articles were selected for full-text and extensive review after following the screening process and according to the eligibility criteria. About 34 unique readmission risk prediction models were identified, in which their predictive ability ranged from poor to good (c statistic 0.5-0.86). Readmission rates ranged between 3.1 and 74.1% depending on the risk category. This review shows that readmission risk prediction is a complex process and is still relatively new as a concept and poorly understood. It confirms that readmission prediction models hold significant accuracy at identifying patients at higher risk for such an event within specific context.

RESEARCH LIMITATIONS/IMPLICATIONS

Since most prediction models were developed for specific populations, conditions or hospital settings, the generalisability and transferability of the predictions across wider or other contexts may be difficult to achieve. Therefore, the value of prediction models remains limited to hospital management. Future research is indicated in this regard.

ORIGINALITY/VALUE

This review is the first to cover readmission risk prediction tools that have been published in the literature since 2011, thereby providing an assessment of the relevance of this crucial KPI to health organisations and managers.

UNPLANNED REPRESENTATION TO HOSPITAL BY PATIENTS WITH DIABETES: DEVELOPMENT AND PILOT FEASIBILITY TESTING OF A SCREENING TOOL

BackgroundUnplanned representation of patients with diabetes recently discharged from emergency department or in-patient hospitals is a common but complex problem worldwide. This study set out to examine the feasibility of a risk screening interview and whether component characteristics may be associated with unplanned representation of patients with diabetes to a tertiary metropolitan hospital.MethodsA screening interview comprised of demographic, social and clinical characteristics was developed and piloted using prospective cross-sectional survey design. A convenience sample of 55 patients was recruited and screened. Outcomes were the occurrence of unplanned representation to hospital within 28 or 90 days of hospital discharge from the index presentation.ResultsThe screening interview was shown to be broadly feasible and acceptable for use by staff and patients, with identified areas for modification. Seventeen participants (30.9%) experienced unplanned representation within 90 days of hospital discharge; for 13 participants (23.6%) this occurred within 28 days. Characteristics linked with unplanned representation to hospital were identified.ConclusionsPreliminary data indicated the feasibility of tool use and informed refinement for future testing of the ability of the screening interview to predict those patients with diabetes at high risk of unplanned representation to hospital to enhance effective care planning.

The association between geriatric treatment and 30-day readmission risk among medical inpatients aged ≥75 years with multimorbidity

Background

Readmission to hospital is frequent among older patients and reported as a post-discharge adverse outcome. The effect of treatment in a geriatric ward for acutely admitted older patients on mortality and function is well established, but less is known about the possible influence of such treatment on the risk of readmission, particularly in the oldest and most vulnerable patients. Our aim was to assess the risk for early readmission for multimorbid patients > 75 years treated in a geriatric ward compared to medical wards and to identify risk factors for 30-day readmissions.

Methods

Prospective cohort study of patients acutely admitted to a medical department at a Norwegian regional hospital. Eligible patients were community-dwelling, multimorbid, receiving home care services, and aged 75+. Patients were consecutively included in the period from 1 April to 31 October 2012. Clinical data were retrieved from the referral letter and medical records.

Results

We included 227 patients with a mean (SD) age of 86.0 (5.7) years, 134 (59%) were female and 59 (26%) were readmitted within 30 days after discharge. We found no statistically significant difference in readmission rate between patients treated in a geriatric ward versus other medical wards. In adjusted Cox proportional hazards regression analyses, lower age (hazard ratio (95% confidence interval) 0.95 (0.91–0.99) per year), female gender (2.17 (1.15–4.00)) and higher MMSE score (1.03 (1.00–1.06) per point) were significant risk factors for readmission.

Conclusions

Lower age, female gender and higher cognitive function were the main risk factors for 30-day readmission to hospital among old patients with multimorbidity. We found no impact of geriatric care on the readmission rate.

Post-hospital falls incidence and risk factors among older adults: a systematic review and meta-analysis

BACKGROUND

Post-hospital falls constitute a significant health concern for older adults who have been recently discharged from the hospital.

OBJECTIVES

To systematically summarise existing evidence on the incidence and risk factors for post-hospital falls among older adults.

METHODS

A systematic review and meta-analysis was conducted. Six electronic databases were searched to identify cohort studies investigating the incidence and risk factors for post-hospital falls in older adults. The incidence and risk factors for post-hospital falls were extracted. The meta-analysis was used to calculate pooled incidences and 95% confidence intervals (CI). The meta-regression and subgroup meta-analysis were conducted to explore sources of heterogeneity in incidence proportions across the eligible studies. A qualitative synthesis was performed for the post-hospital falls risk factors.

RESULTS

Eighteen studies from eight countries (n = 9,080,568) were included. The pooled incidence proportion of any and recurrent post-hospital falls was 14% (95% CI: 13%-15%) and 10% (95% CI: 5%-14%), respectively. Follow-up period, study quality, study country, setting, percentage of female subjects, percentage of subjects with previous falls and the primary data collection method for falls significantly contributed to the 64.8% of the heterogeneity in incidence proportions. Twenty-six risk factors for post-hospital falls were identified in the eligible studies, where biological factors were the most commonly identified factors. The highest risks were reported for previous falls, previous fractures, delirium and neurological diseases.

CONCLUSION

The findings of this study suggested future post-hospital falls prevention should prioritise the needs of older adults with the dominant risk factors. Further investigations into the period-specific incidence and socioeconomic and environmental risk factors for post-hospital falls are also required.

The predictive utility of functional status at discharge: a population-level cohort analysis

Background

Functional status is a patient-important, patient-centered measurement. The utility of functional status measures to inform post-discharge patient needs is unknown. We sought to examine the utility of routinely collected functional status measures gathered from older hospitalized patients to predict a panel of post-discharge outcomes.

Methods

In this population-based retrospective cohort study, Adults 65+ discharged from an acute hospitalization between 4 November 2008 and 18 March 2016 in Ontario, Canada and received an assessment of functional status at discharge using the Health Outcomes for Better Information and Care tool were included. Multivariable regression analysis was used to determine the relationship between functional status and emergency department (ED) re-presentation, hospital readmission, long term care facility (LTCF) admission or wait listing (‘LTCF readiness’), and death at 180 days from discharge.

Results

A total of 80 020 discharges were included. 38 928 (48.6%) re-presented to the ED, 24 222 (30.3%) were re-admitted, 5 037 (6.3%) were LTCF ready, and 9 047 (11.3%) died at 180 days. Beyond age, diminished functional status at discharge was the factor most associated with LTCF readiness (adjusted Odds Ratio [OR] 4.11 for those who are completely dependent for activities of daily living compared to those who are independent; 95% Confidence Interval [CI]: 3.70-4.57) and death (OR 3.99; 95% CI: 3.67-4.35). Functional status also had a graded relationship with each outcome and improved the discriminability of the models predicting death and LTCF readiness (p<0.01) but not ED re-presentation or hospital re-admission.

Conclusion

Routinely collected functional status at discharge meaningfully improves the prediction of long term care home readiness and death. The routine assessment of functional status can inform post-discharge care and planning for older adults.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-021-02652-6.