Readmissions to Intensive Care: A Prospective Multicenter Study in Australia and New Zealand

Improving Intensive Care Unit Early Readmission Prediction Using Optimized and Explainable Machine Learning

It is of great interest to develop and introduce new techniques to automatically and efficiently analyze the enormous amount of data generated in today's hospitals, using state-of-the-art artificial intelligence methods. Patients readmitted to the ICU in the same hospital stay have a higher risk of mortality, morbidity, longer length of stay, and increased cost. The methodology proposed to predict ICU readmission could improve the patients' care. The objective of this work is to explore and evaluate the potential improvement of existing models for predicting early ICU patient readmission by using optimized artificial intelligence algorithms and explainability techniques. In this work, XGBoost is used as a predictor model, combined with Bayesian techniques to optimize it. The results obtained predicted early ICU readmission (AUROC of 0.92 ± 0.03) improves state-of-the-art consulted works (whose AUROC oscillate between 0.66 and 0.78). Moreover, we explain the internal functioning of the model by using Shapley Additive Explanation-based techniques, allowing us to understand the model internal performance and to obtain useful information, as patient-specific information, the thresholds from which a feature begins to be critical for a certain group of patients, and the feature importance ranking.

Development and validation of an interpretable 3 day intensive care unit readmission prediction model using explainable boosting machines

Background

Intensive care unit (ICU) readmissions are associated with mortality and poor outcomes. To improve discharge decisions, machine learning (ML) could help to identify patients at risk of ICU readmission. However, as many models are black boxes, dangerous properties may remain unnoticed. Widely used post hoc explanation methods also have inherent limitations. Few studies are evaluating inherently interpretable ML models for health care and involve clinicians in inspecting the trained model.

Methods

An inherently interpretable model for the prediction of 3 day ICU readmission was developed. We used explainable boosting machines that learn modular risk functions and which have already been shown to be suitable for the health care domain. We created a retrospective cohort of 15,589 ICU stays and 169 variables collected between 2006 and 2019 from the University Hospital Münster. A team of physicians inspected the model, checked the plausibility of each risk function, and removed problematic ones. We collected qualitative feedback during this process and analyzed the reasons for removing risk functions. The performance of the final explainable boosting machine was compared with a validated clinical score and three commonly used ML models. External validation was performed on the widely used Medical Information Mart for Intensive Care version IV database.

Results

The developed explainable boosting machine used 67 features and showed an area under the precision-recall curve of 0.119 ± 0.020 and an area under the receiver operating characteristic curve of 0.680 ± 0.025. It performed on par with state-of-the-art gradient boosting machines (0.123 ± 0.016, 0.665 ± 0.036) and outperformed the Simplified Acute Physiology Score II (0.084 ± 0.025, 0.607 ± 0.019), logistic regression (0.092 ± 0.026, 0.587 ± 0.016), and recurrent neural networks (0.095 ± 0.008, 0.594 ± 0.027). External validation confirmed that explainable boosting machines (0.221 ± 0.023, 0.760 ± 0.010) performed similarly to gradient boosting machines (0.232 ± 0.029, 0.772 ± 0.018). Evaluation of the model inspection showed that explainable boosting machines can be useful to detect and remove problematic risk functions.

Conclusions

We developed an inherently interpretable ML model for 3 day ICU readmission prediction that reached the state-of-the-art performance of black box models. Our results suggest that for low- to medium-dimensional datasets that are common in health care, it is feasible to develop ML models that allow a high level of human control without sacrificing performance.

Explainable Machine Learning on AmsterdamUMCdb for ICU Discharge Decision Support: Uniting Intensivists and Data Scientists

Unexpected ICU readmission is associated with longer length of stay and increased mortality. To prevent ICU readmission and death after ICU discharge, our team of intensivists and data scientists aimed to use AmsterdamUMCdb to develop an explainable machine learning–based real-time bedside decision support tool.

Physician and Nurse Research in Multidisciplinary Intensive Care Units

BACKGROUND

Although clinical care is multidisciplinary, intensive care unit research commonly focuses on single-discipline themes. We sought to characterize intensive care unit research conducted by physicians and nurses.

METHODS

One hundred randomly selected reports of clinical studies published in critical care medical and nursing journals were reviewed.

RESULTS

Of the 100 articles reviewed, 50 were published in medical journals and 50 were published in nursing journals. Only 1 medical study (2%) used qualitative methods, compared with 9 nursing studies (18%) (P = .02). The distribution of quantitative study designs differed between medical and nursing journals (P < .001), with medical journals having a predominance of cohort studies (29 articles [58%]). Compared with medical journal articles, nursing journal articles had significantly fewer authors (median [interquartile range], 5 [3-6] vs 8 [6-10]; P < .001) and study participants (94 [51-237] vs 375 [86-4183]; P < .001) and a significantly lower proportion of male study participants (55% [26%-65%] vs 60% [51%-65%]; P = .02). Studies published in medical journals were much more likely than those published in nursing journals to exclusively involve patients as participants (47 [94%] vs 25 [50%]; P < .001). Coauthorship between physicians and nurses was evident in 14 articles (14%), with infrequent inclusion of authors from other health care disciplines.

CONCLUSIONS

Physician research and nurse research differ in several important aspects and tend to occur within silos. Increased interprofessional collaboration is possible and worthwhile.

Transition of care at discharge from the Intensive Care Unit: a scoping review

OBJECTIVE

to map the available evidence on the components of the transition of care, practices, strategies, and tools used in the discharge from the Intensive Care Unit (ICU) to the Inpatient Unit (IU) and its impact on the outcomes of adult patients.

METHOD

a scoping review using search strategies in six relevant health databases.

RESULTS

37 articles were included, in which 30 practices, strategies or tools were identified for organizing and executing the transfer process, with positive or negative impacts, related to factors intrinsic to the Intensive Care Unit and the Inpatient Unit and cross-sectional factors regarding the staff. The analysis of hospital readmission and mortality outcomes was prevalent in the included studies, in which trends and potential protective actions for a successful care transition are found; however, they still lack more robust evidence and consensus in the literature.

CONCLUSION

transition of care components and practices were identified, in addition to factors intrinsic to the patient, associated with worse outcomes after discharge from the Intensive Care Unit. Discharges at night or on weekends were associated with increased rates of readmission and mortality; however, the association of other practices with the patient's outcome is still inconclusive.

Modified Early Warning Score as a predictor of intensive care unit readmission within 48 hours: a retrospective observational study

OBJECTIVE

To evaluate the hypothesis that the Modified Early Warning Score (MEWS) at the time of intensive care unit discharge is associated with readmission and to identify the MEWS that most reliably predicts intensive care unit readmission within 48 hours of discharge.

METHODS

This was a retrospective observational study of the MEWSs of discharged patients from the intensive care unit. We compared the demographics, severity scores, critical illness characteristics, and MEWSs of readmitted and non-readmitted patients, identified factors associated with readmission in a logistic regression model, constructed a Receiver Operating Characteristic (ROC) curve of the MEWS in predicting the probability of readmission, and presented the optimum criterion with the highest sensitivity and specificity.

RESULTS

The readmission rate was 2.6%, and the MEWS was a significant predictor of readmission, along with intensive care unit length of stay > 10 days and tracheostomy. The ROC curve of the MEWS in predicting the readmission probability had an AUC of 0.82, and a MEWS > 6 carried a sensitivity of 0.78 (95%CI 0.66 - 0.9) and specificity of 0.9 (95%CI 0.87 - 0.93).

CONCLUSION

The MEWS is associated with intensive care unit readmission, and a score > 6 has excellent accuracy as a prognostic predictor.

Benchmarking Deep Learning Architectures for Predicting Readmission to the ICU and Describing Patients-at-Risk

To compare different deep learning architectures for predicting the risk of readmission within 30 days of discharge from the intensive care unit (ICU). The interpretability of attention-based models is leveraged to describe patients-at-risk. Several deep learning architectures making use of attention mechanisms, recurrent layers, neural ordinary differential equations (ODEs), and medical concept embeddings with time-aware attention were trained using publicly available electronic medical record data (MIMIC-III) associated with 45,298 ICU stays for 33,150 patients. Bayesian inference was used to compute the posterior over weights of an attention-based model. Odds ratios associated with an increased risk of readmission were computed for static variables. Diagnoses, procedures, medications, and vital signs were ranked according to the associated risk of readmission. A recurrent neural network, with time dynamics of code embeddings computed by neural ODEs, achieved the highest average precision of 0.331 (AUROC: 0.739, F₁-Score: 0.372). Predictive accuracy was comparable across neural network architectures. Groups of patients at risk included those suffering from infectious complications, with chronic or progressive conditions, and for whom standard medical care was not suitable. Attention-based networks may be preferable to recurrent networks if an interpretable model is required, at only marginal cost in predictive accuracy.

Risk prediction models for intensive care unit readmission: A systematic review of methodology and applicability

OBJECTIVE

We conducted a systematic review of primary models to predict intensive care unit (ICU) readmission.

REVIEW METHODS

We searched MEDLINE, PubMed, Scopus, and Embase for studies on the development of ICU readmission prediction models that are published until January 2017. Data were extracted on the source of data, participants, outcomes, candidate predictors, sample size, missing data, methods for model development, and measures of model performance and model evaluation. The quality and applicability of the included studies were assessed using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies.

RESULTS

We identified five studies describing the development of the primary prediction models of ICU readmission. Studies ranged in size from 343 to 704,963 patients with the mean age of 58.0-68.9 years. The proportion of readmission ranged from 2.5% to 9.6%. The discriminative ability of prediction models measured by area under the receiver operating characteristic curve was 0.66-0.81. None of the studies performed external validations. The quality scores ranged from 42 to 54 out of 62, and the applicability scores from 24 to 32 out of 38.

CONCLUSION

We identified five prediction models for ICU readmission. However, owing to the numerous methodological and reporting deficiencies in the included studies, physicians using these models should interpret the predictions with precautions until an external validation study shows the acceptable level of calibration and accuracy of these models.

Who Are High Users of Hospitals in Canada? Findings From a Population-Based Study

Background

Dying people and older people have often been thought of as high users of hospitals, but current population-based evidence is needed to confirm or refute this claim.

Purpose

Quantitative population-based study designed to identify and describe hospital patients who are high users.

Methods

Data for all 2014–2015 Canadian hospital patients (excluding Quebec) were analyzed to identify and describe high users through descriptive-comparative and regression analysis tests.

Results

Only a small proportion of patients are high users in relation to multiple admissions or 30+ inpatient days of care, and with considerable diversity among them and relatively few of these advanced in age or dying in hospital.

Conclusions

Relatively few patients are high users of hospitals. These people are most often under age 65, so they have the potential to be ill and high users for many years. Flagging would enable individualized care planning to reduce illness exacerbations or slow disease progression and address other risk factors for long or repeat hospitalizations.

Early Discharge From Intensive Care After Cardiac Surgery is Feasible With an Adequate Fast Track, Stepdown Unit: Waikato Experience

BACKGROUND

Enhanced recovery programs within cardiothoracic surgery are a well described benefit to patient postoperative outcomes. We describe our Australasian unit's experience of a day zero discharge enhanced recovery unit from the intensive care department.

METHODS

A retrospective study was conducted on a prospectively maintained database at Waikato Cardiothoracic Unit from September 2014 till October 2017 with 1,739 patients undergoing cardiac surgery. Twenty-two (22) patients were excluded as deaths either intraoperative or in the intensive care unit (ICU) and therefore never discharged. Total population of the study was 1,717 patients. The primary endpoint of this study was to determine if there is no survival disadvantage for the day zero discharge unit compared to standard treatment in ICU at follow-up. The secondary endpoint of the study was to highlight the association between pre and postoperative variables and the impact on discharge from the ICU.

RESULTS

One hundred sixty-eight (168) patients were discharged to the enhanced recovery unit (ERU) day zero. Mean number of hours spent in ICU for the day zero cohort was 7.18 (±1.59. Mean Age 62.5 (±11.22), M:F 4.25:1. Patients were more likely to be discharged day zero if they had a lower EuroSCORE II 1.57 (±1.67) and lower preoperative creatinine 89.4 (±27.5). Those admitted to the ERU on day zero postoperatively were more likely to be discharged with a lower creatinine level, a higher haemoglobin level and have less readmissions per 30days (p<0.05). Survival analysis demonstrated that the patients who were discharged early from ICU had significantly better follow-up survival compared to those who were discharged after 24hours (p<0.05).

CONCLUSIONS

A fast track unit increases the efficiency of an ICU and cardiac surgical department. With the advancements of cardiac surgery a higher number of patients will be suitable for a fast track method. Our unit has demonstrated that a day zero fast track unit in New Zealand can perform with adequate patient safety with no increased risk of mortality and with low rates of failure of the day zero discharge fast track therapy.

[Functional status as an independent risk factor in elderly patients admitted to an Intensive Care Unit]

OBJECTIVE

To assess the association of previous functional status in elderly patients admitted to the ICU, estimated by the Barthel and Short Form-Late Life Function and Disability instrument scales, and the relationship with prognosis and functional capacity at hospital discharge.

MATERIAL AND METHODS

Observational prospective study of ICU-admitted patients older than 74 years, with a length of stay greater than 48hours. Demographic data, social background, comorbidities, disability questionnaire (Barthel, Short Form-Late Life Function and Disability instrument), main diagnosis and severity (SAPS 3) on ICU admission were recorded. Factors associated with mortality or poor functional status at hospital discharge (Barthel Index less than 35) were established by multivariate analysis.

RESULTS

During the study period, 219 elderly patients were admitted in ICU, of whom 129 (15%) had an ICU length of stay greater than 48hours. The median age was 80 years (77-83), with 52% women. Main diagnoses on admission included ischaemic heart disease (19%), another medical diagnosis (38%), and surgical procedure (43%). A Barthel score <36 (median 95, 85-100) was observed in 3% of the patients on admission. The median ICU length of stay was 5 days (4-8). ICU mortality was 6% (hospital mortality: 10%). On hospital discharge, 7% had severe dependence (Barthel <36). In this population, factors independently associated with mortality or poor functional status at hospital discharge were the pre-admission functional status, based on Short Form-Late Life Function and Disability instrument (OR 0.95, 95% CI, 0.91 to 0.98), and the severity on admission assessed by SAPS 3 (OR 1.10, 95% CI, 1.02 to 1.18), p=.0007.

CONCLUSIONS

In elderly patients requiring ICU admission, a higher SAPS 3 score and functional impairment on admission were associated with mortality or severe dependence upon discharge.

Preventable readmission to intensive care unit in critically ill cancer patients

BACKGROUND

Readmission to intensive care unit (ICU) after discharge to ward has been reported to be associated with increased hospital mortality and longer length of stay (LOS). The objective of this study was to investigate whether ICU readmission are preventable in critically ill cancer patients.

METHODS

Data of patients who readmitted to intensive care unit (ICU) at National Cancer Center/Cancer Hospital of Chinese Academy of Medical Sciences (CAMS) and Peking Union Medical College (PUMC) between January 2013 and November 2016 were retrospectively collected and reviewed.

RESULTS

A total of 39 patients were included in the final analysis, and the overall readmission rate between 2013 and 2016 was 1.32% (39/2,961). Of 39 patients, 32 (82.1%) patients were judged as unpreventable and 7 (17.9%) patients were preventable. There were no significant differences in duration of mechanical ventilation, ICU LOS, hospital LOS, ICU mortality and in-hospital mortality between patients who were unpreventable and preventable. For 24 early readmission patients, 7 (29.2%) patients were preventable and 17 (70.8%) patients were unpreventable. Patients who were late readmission were all unpreventable. There was a trend that patients who were preventable had longer 1-year survival compared with patients who were unpreventable (100% vs. 66.8%, log rank=1.668, P=0.196).

CONCLUSION

Most readmission patients were unpreventable, and all preventable readmissions occurred in early period after discharge to ward. There were no significant differences in short term outcomes and 1-year survival in critically ill cancer patients whose readmissions were preventable or not.

Prediction of early unplanned intensive care unit readmission in a UK tertiary care hospital: a cross-sectional machine learning approach

OBJECTIVES

Unplanned readmissions to the intensive care unit (ICU) are highly undesirable, increasing variance in care, making resource planning difficult and potentially increasing length of stay and mortality in some settings. Identifying patients who are likely to suffer unplanned ICU readmission could reduce the frequency of this adverse event.

SETTING

A single academic, tertiary care hospital in the UK.

PARTICIPANTS

A set of 3326 ICU episodes collected between October 2014 and August 2016. All records were of patients who visited an ICU at some point during their stay. We excluded patients who were ≤16 years of age; visited ICUs other than the general and neurosciences ICU; were missing crucial electronic patient record measurements; or had indeterminate ICU discharge outcomes or very early or extremely late discharge times. After exclusion, 2018 outcome-labelled episodes remained.

PRIMARY AND SECONDARY OUTCOME MEASURES

Area under the receiver operating characteristic curve (AUROC) for prediction of unplanned ICU readmission or in-hospital death within 48 hours of first ICU discharge.

RESULTS

In 10-fold cross-validation, an ensemble predictor was trained on data from both the target hospital and the Medical Information Mart for Intensive Care (MIMIC-III) database and tested on the target hospital's data. This predictor discriminated between patients with the unplanned ICU readmission or death outcome and those without this outcome, attaining mean AUROC of 0.7095 (SE 0.0260), superior to the purpose-built Stability and Workload Index for Transfer (SWIFT) score (AUROC=0.6082, SE 0.0249; p=0.014, pairwise t-test).

CONCLUSIONS

Despite the inherent difficulties, we demonstrate that a novel machine learning algorithm based on transfer learning could achieve good discrimination, over and above that of the treating clinicians or the value added by the SWIFT score. Accurate prediction of unplanned readmission could be used to target resources more efficiently.