Delirium prediction in the intensive care unit: comparison of two delirium prediction models

Interpretable machine learning model for early prediction of delirium in elderly patients following intensive care unit admission: a derivation and validation study

Background and objective

Delirium is the most common neuropsychological complication among older adults admitted to the intensive care unit (ICU) and is often associated with a poor prognosis. This study aimed to construct and validate an interpretable machine learning (ML) for early delirium prediction in older ICU patients.

Methods

This was a retrospective observational cohort study and patient data were extracted from the Medical Information Mart for Intensive Care-IV database. Feature variables associated with delirium, including predisposing factors, disease-related factors, and iatrogenic and environmental factors, were selected using least absolute shrinkage and selection operator regression, and prediction models were built using logistic regression, decision trees, support vector machines, extreme gradient boosting (XGBoost), k-nearest neighbors and naive Bayes methods. Multiple metrics were used for evaluation of performance of the models, including the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, recall, F1 score, calibration plot, and decision curve analysis. SHapley Additive exPlanations (SHAP) were used to improve the interpretability of the final model.

Results

Nine thousand seven hundred forty-eight adults aged 65 years or older were included for analysis. Twenty-six features were selected to construct ML prediction models. Among the models compared, the XGBoost model demonstrated the best performance including the highest AUC (0.836), accuracy (0.765), sensitivity (0.713), recall (0.713), and F1 score (0.725) in the training set. It also exhibited excellent discrimination with AUC of 0.810, good calibration, and had the highest net benefit in the validation cohort. The SHAP summary analysis showed that Glasgow Coma Scale, mechanical ventilation, and sedation were the top three risk features for outcome prediction. The SHAP dependency plot and SHAP force analysis interpreted the model at both the factor level and individual level, respectively.

Conclusion

ML is a reliable tool for predicting the risk of critical delirium in elderly patients. By combining XGBoost and SHAP, it can provide clear explanations for personalized risk prediction and more intuitive understanding of the effect of key features in the model. The establishment of such a model would facilitate the early risk assessment and prompt intervention for delirium.

Precision-based approaches to delirium in critical illness: A narrative review

Delirium occurs in critical illness and is associated with poor clinical outcomes, having a longstanding impact on survivors. Understanding the complexity of delirium in critical illness and its deleterious outcome has expanded since early reports. Delirium is a culmination of predisposing and precipitating risk factors that result in a transition to delirium. Known risks range from advanced age, frailty, medication exposure or withdrawal, sedation depth, and sepsis. Because of its multifactorial nature, different clinical phenotypes, and potential neurobiological causes, a precise approach to reducing delirium in critical illness requires a broad understanding of its complexity. Refinement in the categorization of delirium subtypes or phenotypes (i.e., psychomotor classifications) requires attention. Recent advances in the association of clinical phenotypes with clinical outcomes expand our understanding and highlight potentially modifiable targets. Several delirium biomarkers in critical care have been examined, with disrupted functional connectivity being precise in detecting delirium. Recent advances reinforce delirium as an acute, and partially modifiable, brain dysfunction, and place emphasis on the importance of mechanistic pathways including cholinergic activity and glucose metabolism. Pharmacologic agents have been assessed in randomized controlled prevention and treatment trials, with a disappointing lack of efficacy. Antipsychotics remain widely used after "negative" trials, yet may have a role in specific subtypes. However, antipsychotics do not appear to improve clinical outcomes. Alpha-2 agonists perhaps hold greater potential for current use and future investigation. The role of thiamine appears promising, yet requires evidence. Looking forward, clinical pharmacists should prioritize the mitigation of predisposing and precipitating risk factors as able. Future research is needed within individual delirium psychomotor subtypes and clinical phenotypes to identify modifiable targets that hold the potential to improve not only delirium duration and severity, but long-term outcomes including cognitive impairment.

Delirium severity and outcomes of critically ill COVID-19 patients

OBJECTIVE

To investigate the impact of delirium severity in critically ill COVID-19 patients and its association with outcomes.

METHODS

This prospective cohort study was performed in two tertiary intensive care units in Rio de Janeiro, Brazil. COVID-19 patients were evaluated daily during the first 7 days of intensive care unit stay using the Richmond Agitation Sedation Scale, Confusion Assessment Method for Intensive Care Unit (CAM-ICU) and Confusion Method Assessment for Intensive Care Unit-7 (CAM-ICU-7). Delirium severity was correlated with outcomes and one-year mortality.

RESULTS

Among the 277 COVID-19 patients included, delirium occurred in 101 (36.5%) during the first 7 days of intensive care unit stay, and it was associated with a higher length of intensive care unit stay in days (IQR 13 [7 - 25] versus 6 [4 - 12]; p < 0.001), higher hospital mortality (25.74% versus 5.11%; p < 0.001) and additional higher one-year mortality (5.3% versus 0.6%, p < 0.001). Delirium was classified by CAM-ICU-7 in terms of severity, and higher scores were associated with higher in-hospital mortality (17.86% versus 34.38% versus 38.46%, 95%CI, p value < 0.001). Severe delirium was associated with a higher risk of progression to coma (OR 7.1; 95%CI 1.9 - 31.0; p = 0.005) and to mechanical ventilation (OR 11.09; 95%CI 2.8 - 58.5; p = 0.002) in the multivariate analysis, adjusted by severity and frailty.

CONCLUSION

In patients admitted with COVID-19 in the intensive care unit, delirium was an independent risk factor for the worst prognosis, including mortality. The delirium severity assessed by the CAM-ICU-7 during the first week in the intensive care unit was associated with poor outcomes, including progression to coma and to mechanical ventilation.

External validity of an automated delirium prediction model (DEMO) and comparison to the manual VMS-questions: a retrospective cohort study

BACKGROUND

It is estimated that one-third of delirium cases in hospitals could be prevented with appropriate interventions. In Dutch hospitals a manual instrument (VMS-questions) is used to identify patients at-risk for delirium. Delirium Model (DEMO) is an automated model which could support delirium prevention more efficiently. However, it has not been validated beyond the hospital it was developed in.

AIM

To externally validate the DEMO and compare its performance to the VMS-questions.

METHOD

A retrospective cohort study between July and December 2018 was conducted. Delirium cases were identified through a chart review, and the VMS-questions were extracted from the electronic health records. The DEMO was validated in patients ≥ 60 years, and a comparison with the VMS-questions was made in patients ≥ 70 years.

RESULTS

In total 1,345 admissions were included. The DEMO predicted 59 out of 75 delirium cases (sensitivity 0.79, 95% CI = 0.68-0.87; specificity 0.75, 95% CI = 0.72-0.77). Compared to the VMS-questions, the DEMO showed a lower specificity (0.64 vs. 0.72; p < 0.001) and a comparable sensitivity (0.83 vs. 0.80; p = 0.56). The VMS-questions were missing in 20% of admissions, in which the DEMO correctly predicted 10 of 12 delirium cases.

CONCLUSION

The DEMO showed acceptable performance for delirium prediction. Overall the DEMO predicted more delirium cases because the VMS-questions were missing in 20% of admissions. This study shows that automated instruments such as DEMO could play a key role in the efficient and timely deployment of measures to prevent delirium.

Deep Learning-Based Recurrent Delirium Prediction in Critically Ill Patients

OBJECTIVES

To predict impending delirium in ICU patients using recurrent deep learning.

DESIGN

Retrospective cohort study.

SETTING

Fifteen medical-surgical ICUs across Alberta, Canada, between January 1, 2014, and January 24, 2020.

PATIENTS

Forty-three thousand five hundred ten ICU admissions from 38,426 patients.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We used ICU and administrative health data to train deep learning models to predict delirium episodes in the next two 12-hour periods (0-12 and 12-24 hr), starting at 24 hours after ICU admission, and to generate new predictions every 12 hours. We used a comprehensive set of 3,643 features, capturing patient history, early ICU admission information (first 24 hr), and the temporal dynamics of various clinical variables throughout the ICU admission. Our deep learning architecture consisted of a feature embedding, a recurrent, and a prediction module. Our best model based on gated recurrent units yielded a sensitivity of 0.810, a specificity of 0.848, a precision (positive predictive value) of 0.704, and an area under the receiver operating characteristic curve (AUROC) of 0.909 in the hold-out test set for the 0-12-hour prediction horizon. For the 12-24-hour prediction horizon, the same model achieved a sensitivity of 0.791, a specificity of 0.807, a precision of 0.637, and an AUROC of 0.895 in the test set.

CONCLUSIONS

Our delirium prediction model achieved strong performance by applying deep learning to a dataset that is at least one order of magnitude larger than those used in previous studies. Another novel aspect of our study is the temporal nature of our features and predictions. Our model enables accurate prediction of impending delirium in the ICU, which can potentially lead to early intervention, more efficient allocation of ICU resources, and improved patient outcomes.

Survey of Guideline Compliance and Attitude Toward Symptom Management in Japanese Intensive Care Units

Introduction

The Clinical Practice Guideline for the Management of Pain, Agitation, and Delirium in Adult Patients in the Intensive Care Unit (ICU) was revised in 2018 to include sleep disruption and immobility. Inadequate management of these symptoms can lead to negative consequences. A 2019 survey in Japan found that the guideline was recognized but needed to be consistently implemented.

Objective

This study aimed to examine compliance with the guideline for symptom management of pain, agitation, delirium, and sleep in Japanese ICUs.

Methods

This study included all ICUs in Japan and asked one representative from each unit to respond to the web survey from January 2022 to February 2022.

Results

Of a potential 643 units, 125 respondents from the ICU were included in the analysis (19.4% response rate). Compared to the guideline's recommendations, (a) pain assessment was performed in 86.3% of patients who could self-report, and in 72.0% of those who could not self-report; (b) agitation and sedation assessment was performed in 99% of patients; (c) only 66.1% of nurses reported assessing sleep quality on the units, and 9.1% performed the subjective sleep quality assessment; (d) the use of the recommended risk factor of the delirium assessment tool was low (9.6%). Additionally, according to the survey respondents, contrary to the guideline, many units administered medications to prevent and treat delirium, and approximately 30% used multiple non-drug interventions. The data are expressed as numbers and percentages. Some datasets were incomplete due to missing values.

Conclusion

Most units used drugs for delirium prevention and treatment, and only a few used non-drug interventions. There is a need to popularize the assessment of sleep and delirium risk factors and use non-drug interventions to promote patient-centered care in the future.

Comparison of Prognostic Accuracy of 3 Delirium Prediction Models

BACKGROUND

Delirium is a severe complication in critical care patients. Accurate prediction could facilitate determination of which patients are at risk. In the past decade, several delirium prediction models have been developed.

OBJECTIVES

To compare the prognostic accuracy of the PRE-DELIRIC, E-PRE-DELIRIC, and Lanzhou models, and to investigate the difference in prognostic accuracy of the PRE-DELIRIC model between patients receiving and patients not receiving mechanical ventilation.

METHODS

This retrospective study involved adult patients admitted to the intensive care unit during a 2-year period. Delirium was assessed by using the Confusion Assessment Method for the Intensive Care Unit or any administered dose of haloperidol or quetiapine. Model discrimination was assessed by calculating the area under the receiver operating characteristic curve (AUC); values were compared using the DeLong test.

RESULTS

The study enrolled 1353 patients. The AUC values were calculated as 0.716 (95% CI, 0.688-0.745), 0.681 (95% CI, 0.650-0.712), and 0.660 (95% CI, 0.629-0.691) for the PRE-DELIRIC, E-PRE-DELIRIC, and Lanzhou models, respectively. The difference in model discrimination was statistically significant for comparison of the PRE-DELIRIC with the E-PRE-DELIRIC (AUC difference, 0.035; P = .02) and Lanzhou models (AUC difference, 0.056; P < .001). In the PRE-DELIRIC model, the AUC was 0.711 (95% CI, 0.680-0.743) for patients receiving mechanical ventilation and 0.664 (95% CI, 0.586-0.742) for those not receiving it (difference, 0.047; P = .27).

CONCLUSION

Statistically significant differences in prognostic accuracy were found between delirium prediction models. The PRE-DELIRIC model was the best-performing model and can be used in patients receiving or not receiving mechanical ventilation.

Establishment of a model for predicting the outcome of induced labor in full-term pregnancy based on machine learning algorithm

To evaluate and establish a prediction model of the outcome of induced labor based on machine learning algorithm. This was a cross-sectional design. The subjects were divided into primipara and multipara, and the risk factors for the outcomes of induced labor were assessed by multifactor logistic regression analysis. The outcome model of labor induced with oxytocin (OT) was constructed based on the four machine learning algorithms, including AdaBoost, logistic regression, naive Bayes classifier, and support vector machine. Factors, such as accuracy, recall, precision, F1 value, and receiver operating characteristic curve, were used to evaluate the prediction performance of the model, and the clinical application of the model was verified. A total of 907 participants were included in this study. Logistic regression algorithm obtained better results in both primipara and multipara groups compared to the other three models. The accuracy of the model for the prediction of "successful induction of labor" was 94.24% and 96.55%, and that of "failed induction of labor" was 65.00% and 66.67% in the primipara and the multipara groups, respectively. This study established a prediction model of OT-induced labor based on the Logistic regression algorithm, with rapid response, high accuracy, and strong extrapolation, which was critical for obstetric clinical nursing.

External validation and comparison of two delirium prediction models in patients admitted to the cardiac intensive care unit

Background

No data is available on delirium prediction models in the cardiac intensive care unit (CICU), although preexisting delirium prediction models [PREdiction of DELIRium in ICu patients (PRE-DELIRIC) and Early PREdiction of DELIRium in ICu patients (E-PRE-DELIRIC)] were developed and validated based on a population admitted to the general intensive care unit (ICU). Therefore, we externally validated the usefulness of the PRE-DELIRIC and E-PRE-DELIRIC models and compared their predictive performance in patients admitted to the CICU.

Methods

A total of 2,724 patients admitted to the CICU were enrolled between September 2012 and December 2018. Delirium was defined as at least one positive Confusion Assessment Method for the ICU (CAM-ICU) which was screened at least once every 8 h. The PRE-DELIRIC value was calculated within 24 h of CICU admission, and the E-PRE-DELIRIC value was calculated at CICU admission. The predictive performance of the models was evaluated by using the area under the receiver operating characteristic (AUROC) curve, and the calibration slope was assessed graphically by plotting.

Results

Delirium occurred in 677 patients (24.8%) when the patients were assessed thrice daily until 7 days of the CICU stay. The AUROC curve for the prediction of delirium was significantly greater for PRE-DELIRIC values [0.84, 95% confidence interval (CI): 0.82–0.86] than for E-PRE-DELIRIC values (0.79, 95% CI: 0.77–0.80) [z score of −6.24 (p < 0.001)]. Net reclassification improvement for the prediction of delirium increased by 0.27 (95% CI: 0.21–0.32, p < 0.001). Calibration was acceptable in the PRE-DELIRIC model (Hosmer-Lemeshow p = 0.170) but not in the E-PRE-DELIRIC model (Hosmer-Lemeshow p < 0.001).

Conclusion

Although both models have good predictive performance for the development of delirium, even in critically ill cardiac patients, the performance of the PRE-DELIRIC model might be superior to that of the E-PRE-DELIRIC model. Further studies are required to confirm our results and design a specific delirium prediction model for CICU patients.

Delirium in Older Adults: What a Surgeon Needs to Know

Delirium remains a challenging clinical problem in hospitalized older adults, especially for postoperative patients. This complication, with a high risk of postoperative mortality and an increased length of stay, frequently occurs in older adult patients. This brief narrative paper aims to review the recent literature regarding delirium and its most recent update. We also offer physicians a brief and essential clinical practice guide to managing this acute and common disease.

Longitudinal validation of an electronic health record delirium prediction model applied at admission in COVID-19 patients

OBJECTIVE

To validate a previously published machine learning model of delirium risk in hospitalized patients with coronavirus disease 2019 (COVID-19).

METHOD

Using data from six hospitals across two academic medical networks covering care occurring after initial model development, we calculated the predicted risk of delirium using a previously developed risk model applied to diagnostic, medication, laboratory, and other clinical features available in the electronic health record (EHR) at time of hospital admission. We evaluated the accuracy of these predictions against subsequent delirium diagnoses during that admission.

RESULTS

Of the 5102 patients in this cohort, 716 (14%) developed delirium. The model's risk predictions produced a c-index of 0.75 (95% CI, 0.73-0.77) with 27.7% of cases occurring in the top decile of predicted risk scores. Model calibration was diminished compared to the initial COVID-19 wave.

CONCLUSION

This EHR delirium risk prediction model, developed during the initial surge of COVID-19 patients, produced consistent discrimination over subsequent larger waves; however, with changing cohort composition and delirium occurrence rates, model calibration decreased. These results underscore the importance of calibration, and the challenge of developing risk models for clinical contexts where standard of care and clinical populations may shift.

Development and validation of prediction models for neurocognitive disorders in adult patients admitted to the ICU with sleep disturbance

BACKGROUND

Neurocognitive disorders (NCDs) and sleep disturbance are highly prevalent in the perioperative period and intensive care unit (ICU). There has been a lack of individualized evaluation tools designed for the high-risk NCDs in critically ill patients with sleep disturbance.

OBJECTIVES

The aim of this study was to develop and validate prediction models for NCDs among adult patients with sleep disturbance.

METHODS

The R software was used to analyze the dataset of adult patients admitted to the ICU with sleep disturbance, who were diagnosed following the codes of the International Classification of Diseases, 9th Revision (ICD-9) and 10th Revision (ICD-10) using the MIMIC-IV database. We used logistic regression and LASSO analyses to identify important risk factors associated with NCDs and develop nomograms for NCDs predictions. We measured the performances of the nomograms using the bootstrap resampling procedure, sensitivity, specificity of the receiver operating characteristic (ROC), area under the ROC curves (AUC), and decision curve analysis (DCA).

RESULTS

The prediction models shared the 10 risk factors (age, gender, midazolam, morphine, glucose, diabetes diseases, potassium, international normalized ratio, partial thromboplastin time, and respiratory rate). Cardiovascular diseases were included in the logistic regression, the sensitivity was 74.1%, and specificity was 64.6%. When platelet and Glasgow Coma Score (GCS) were included and cardiovascular diseases were removed in the LASSO prediction model, the sensitivity was 86.1% and specificity was 82.8%. Discriminative abilities of the logistic prediction and LASSO prediction models for NCDs in the validation set were evaluated as the AUC scores, which were 0.730 (95% CI 0.716-0.743) and 0.920 (95% CI 0.912-0.927). Net benefits of the prediction models were observed at threshold probabilities of 0.567 and 0.914.

CONCLUSIONS

The LASSO prediction model showed better performance than the logistic prediction model and should be preferred for nomogram-assisted decisions on clinical risk management of NCDs among adult patients with sleep disturbance in the ICU.

Tools Are Needed to Promote Sedation Practices for Mechanically Ventilated Patients

Suboptimal sedation practices continue to be frequent, although the updated guidelines for management of pain, agitation, and delirium in mechanically ventilated (MV) patients have been published for several years. Causes of low adherence to the recommended minimal sedation protocol are multifactorial. However, the barriers to translation of these protocols into standard care for MV patients have yet to be analyzed. In our view, it is necessary to develop fresh insights into the interaction between the patients' responses to nociceptive stimuli and individualized regulation of patients' tolerance when using analgesics and sedatives. By better understanding this interaction, development of novel tools to assess patient pain tolerance and to define and predict oversedation or delirium may promote better sedation practices in the future.

Delirium in Intensive Care

Purpose of Review

Delirium in the intensive care unit (ICU) has become increasingly acknowledged as a significant problem for critically ill patients affecting both the actual course of illness as well as outcomes. In this review, we focus on the current evidence and the gaps in knowledge.

Recent Findings

This review highlights several areas in which the evidence is weak and further research is needed in both pharmacological and non-pharmacological treatment. A better understanding of subtypes and their different response to therapy is needed and further studies in aetiology are warranted. Larger studies are needed to explore risk factors for developing delirium and for examining long-term consequences. Finally, a stronger focus on experienced delirium and considering the perspectives of both patients and their families is encouraged.

Summary

With the growing number of studies and a better framework for research leading to stronger evidence, the outcomes for patients suffering from delirium will most definitely improve in the years to come.

A Machine Learning-Based Algorithm for the Prediction of Intensive Care Unit Delirium (PRIDE): Retrospective Study

Background

Delirium frequently occurs among patients admitted to the intensive care unit (ICU). There is limited evidence to support interventions to treat or resolve delirium in patients who have already developed delirium. Therefore, the early recognition and prevention of delirium are important in the management of critically ill patients.

Objective

This study aims to develop and validate a delirium prediction model within 24 hours of admission to the ICU using electronic health record data. The algorithm was named the Prediction of ICU Delirium (PRIDE).

Methods

This is a retrospective cohort study performed at a tertiary referral hospital with 120 ICU beds. We only included patients who were 18 years or older at the time of admission and who stayed in the medical or surgical ICU. Patients were excluded if they lacked a Confusion Assessment Method for the ICU record from the day of ICU admission or if they had a positive Confusion Assessment Method for the ICU record at the time of ICU admission. The algorithm to predict delirium was developed using patient data from the first 2 years of the study period and validated using patient data from the last 6 months. Random forest (RF), Extreme Gradient Boosting (XGBoost), deep neural network (DNN), and logistic regression (LR) were used. The algorithms were externally validated using MIMIC-III data, and the algorithm with the largest area under the receiver operating characteristics (AUROC) curve in the external data set was named the PRIDE algorithm.

Results

A total of 37,543 cases were collected. After patient exclusion, 12,409 remained as our study population, of which 3816 (30.8%) patients experienced delirium incidents during the study period. Based on the exclusion criteria, out of the 96,016 ICU admission cases in the MIMIC-III data set, 2061 cases were included, and 272 (13.2%) delirium incidents occurred. The average AUROCs and 95% CIs for internal validation were 0.916 (95% CI 0.916-0.916) for RF, 0.919 (95% CI 0.919-0.919) for XGBoost, 0.881 (95% CI 0.878-0.884) for DNN, and 0.875 (95% CI 0.875-0.875) for LR. Regarding the external validation, the best AUROC were 0.721 (95% CI 0.72-0.721) for RF, 0.697 (95% CI 0.695-0.699) for XGBoost, 0.655 (95% CI 0.654-0.657) for DNN, and 0.631 (95% CI 0.631-0.631) for LR. The Brier score of the RF model is 0.168, indicating that it is well-calibrated.

Conclusions

A machine learning approach based on electronic health record data can be used to predict delirium within 24 hours of ICU admission. RF, XGBoost, DNN, and LR models were used, and they effectively predicted delirium. However, with the potential to advise ICU physicians and prevent ICU delirium, prospective studies are required to verify the algorithm’s performance.

Nursing intervention to prevent delirium in critically ill adults

OBJECTIVE

To determine the effectiveness of a nursing intervention for delirium prevention in critically ill patients.

METHOD

A quasi-experimental study was conducted with a non-equivalent control group and with evaluation before and after the intervention. 157 Patients were part of the intervention group and 134 of the control group. Patients were followed-up until they were discharged from the ICU or died. The incidence of delirium in both groups was compared. Additionally, the effect measures were adjusted for the propensity score.

RESULTS

The incidence and incidence rate of delirium in the control group were 20.1% and 33.1 per 1000 person-days (CI 95% 22.7 to 48.3) and in the intervention group was 0.6% and 0.64 per 1000 person-days (CI 95% 0.22 to 11.09), respectively. The crude Hazard Ratio was 0.06 (CI 95% 0,008 to 0,45) and adjusted 0.07 (CI 95% 0,009 to 0,60). The number needed to be treated was six.

CONCLUSION

Low incidence of delirium in critically ill patients intervened demonstrated the effectiveness of interventions. The average intervention time was 4 days with a 15-minutes dedication for each patient.

Prospective Randomized Controlled Pilot Study of High-Intensity Lightbox Phototherapy to Prevent ICU-Acquired Delirium Incidence

Background

This study aimed to evaluate the role of disturbed circadian rhythm in potentiating intensive care unit (ICU)-acquired delirium.Previous studies have demonstrated bright light therapy (BLT) as an effective modality to improve sleeping patterns and cognitive function in non-critically ill patients. However, its benefit in the ICU has not been clearly established. In this study, we aimed to evaluate the application of daily high-intensity phototherapy at the bedside to deter ICU delirium incidence and duration.

Methodology

This was a single center, prospective study conducted in ICUs at the Carilion Roanoke Memorial Hospital in Roanoke, VA. Adults patients admitted to the ICU from July 9, 2018 to March 20, 2020 were included in the study. The patients were subjected to 30-minute BLT session (10,000 lux) at the bedside starting at 0700 while in the ICU. Patients were randomized into either the control group (standard hospital lighting) or phototherapy group. Data were analyzed using Wilcoxon rank sum test for continuous variables, Pearson chi-square test for categorical variables, and logistic regression for multivariable analysis that examined significant risk factors for ICU delirium.

Results

Delirium incidence between BLT (18%) and control (17.5%) groups was non-significant. Total number of delirium-free, coma-free days, as determined by Confusion Assessment Method for the ICU, demonstrated no differences between groups with a median of 28 days (p = 0.516). In multivariable analysis, patients with a Sequential Organ Failure Assessment Score >3 also showed no significant change in ICU delirium incidence when provided bedside BLT compared to those with standard hospital lighting (odds ratio: 0.08; 95% confidence interval: 0.002-1.40; p = 0.867).

Conclusions

In this randomized control pilot study, daily morning 10,000 lux BLT of 30-minute duration alone was not associated with a significant decrease in ICU-acquired delirium incidence or duration compared to standard hospital lighting. Future studies should consider a nuanced approach to better elucidate the role of disturbed circadian rhythm in influencing ICU-acquired delirium by not only undertaking BLT during the day but also minimizing nighttime light exposure.

Development and External Validation of a Delirium Prediction Model for Hospitalized Patients With Coronavirus Disease 2019

Background

The coronavirus disease 2019 pandemic has placed unprecedented stress on health systems and has been associated with elevated risk for delirium. The convergence of pandemic resource limitation and clinical demand associated with delirium requires careful risk stratification for targeted prevention efforts.

Objectives

To develop an incident delirium predictive model among coronavirus disease 2019 patients.

Methods

We applied supervised machine learning to electronic health record data for inpatients with coronavirus disease 2019 at three hospitals to build an incident delirium diagnosis prediction model. We validated this model in three different hospitals. Both hospital cohorts included academic and community settings.

Results

Among 2907 patients across 6 hospitals, 488 (16.8%) developed delirium. Applying the predictive model in the external validation cohort of 755 patients, the c-index was 0.75 (0.71–0.79) and the lift in the top quintile was 2.1. At a sensitivity of 80%, the specificity was 56%, negative predictive value 92%, and positive predictive value 30%. Equivalent model performance was observed in subsamples stratified by age, sex, race, need for critical care and care at community vs. academic hospitals.

Conclusion

Machine learning applied to electronic health records available at the time of inpatient admission can be used to risk-stratify patients with coronavirus disease 2019 for incident delirium. Delirium is common among patients with coronavirus disease 2019, and resource constraints during a pandemic demand careful attention to the optimal application of predictive models.

Derivation and Validation of an Ensemble Model for the Prediction of Agitation in Mechanically Ventilated Patients Maintained Under Light Sedation

OBJECTIVES

Light sedation is recommended over deep sedation for invasive mechanical ventilation to improve clinical outcome but may increase the risk of agitation. This study aimed to develop and prospectively validate an ensemble machine learning model for the prediction of agitation on a daily basis.

DESIGN

Variables collected in the early morning were used to develop an ensemble model by aggregating four machine learning algorithms including support vector machines, C5.0, adaptive boosting with classification trees, and extreme gradient boosting with classification trees, to predict the occurrence of agitation in the subsequent 24 hours.

SETTING

The training dataset was prospectively collected in 95 ICUs from 80 Chinese hospitals on May 11, 2016, and the validation dataset was collected in 20 out of these 95 ICUs on December 16, 2019.

PATIENTS

Invasive mechanical ventilation patients who were maintained under light sedation for 24 hours prior to the study day and who were to be maintained at the same sedation level for the next 24 hours.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

A total of 578 invasive mechanical ventilation patients from 95 ICUs in 80 Chinese hospitals, including 459 in the training dataset and 119 in the validation dataset, were enrolled. Agitation was observed in 36% (270/578) of the invasive mechanical ventilation patients. The stepwise regression model showed that higher body temperature (odds ratio for 1°C increase: 5.29; 95% CI, 3.70-7.84; p < 0.001), greater minute ventilation (odds ratio for 1 L/min increase: 1.15; 95% CI, 1.02-1.30; p = 0.019), higher Richmond Agitation-Sedation Scale (odds ratio for 1-point increase: 2.43; 95% CI, 1.92-3.16; p < 0.001), and days on invasive mechanical ventilation (odds ratio for 1-d increase: 0.95; 95% CI, 0.93-0.98; p = 0.001) were independently associated with agitation in the subsequent 24 hours. In the validation dataset, the ensemble model showed good discrimination (area under the receiver operating characteristic curve, 0.918; 95% CI, 0.866-0.969) and calibration (Hosmer-Lemeshow test p = 0.459) in predicting the occurrence of agitation within 24 hours.

CONCLUSIONS

This study developed an ensemble model for the prediction of agitation in invasive mechanical ventilation patients under light sedation. The model showed good calibration and discrimination in an independent dataset.

Poor outcomes of delirium in the intensive care units are amplified by increasing age: A retrospective cohort study

BACKGROUND

Delirium in patients in intensive care units (ICUs) is an acute disturbance and fluctuation of cognition and consciousness. Though increasing age has been found to be related to ICU delirium, there is limited evidence of the effect of age on delirium outcomes. The aim of this study is to investigate the relationship between age categories and outcomes among ICU delirium patients.

METHODS

Data were extracted from the electronic ICU (eICU) Collaborative Research Database with records from 3,931 patients with delirium. Patients were classified into non-aged (<65 years), young-old (65-74 years), middle-old (75-84 years), and very-old (≥85 years) groups. A Cox regression model was built to examine the role of age in death in ICU and in hospital after controlling covariates.

RESULTS

The sample included 1,667 (42.4%) non-aged, 891 (22.7%) young-old, 848 (21.6%) middle-old, and 525 (13.3%) very-old patients. The ICU mortality rate was 8.3% and the hospital mortality rate was 15.4%. Compared with the non-aged group, the elderly patients (≥65 yeras) had higher mortality at ICU discharge (χ² =13.726, P=0.001) and hospital discharge (χ ²=56.347, P<0.001). The Cox regression analysis showed that age was an independent risk factor for death at ICU discharge (hazard ratio [HR]=1.502, 1.675, 1.840, 95% confidence interval [CI] 1.138-1.983, 1.250-2.244, 1.260-2.687; P=0.004, 0.001, 0.002 for the young-, middle- and very-old group, respectively) as well as death at hospital discharge (HR=1.801, 2.036, 2.642, 95% CI 1.454-2.230, 1.638-2.530, 2.047-3.409; all P<0.001).

CONCLUSIONS

The risks of death in the ICU and hospital increase with age among delirious patients.

ICU Delirium-Prediction Models: A Systematic Review

Supplemental Digital Content is available in the text.

Objective:

Summarize performance and development of ICU delirium-prediction models published within the past 5 years.

Data Sources:

Systematic electronic searches were conducted in April 2019 using PubMed, Embase, Cochrane Central, Web of Science, and Cumulative Index to Nursing and Allied Health Literature to identify peer-reviewed studies.

Study Selection:

Eligible studies were published in English during the past 5 years that specifically addressed the development, validation, or recalibration of delirium-prediction models in adult ICU populations.

Data Extraction:

Screened citations were extracted independently by three investigators with a 42% overlap to verify consistency using the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies.

Data Synthesis:

Eighteen studies featuring 23 distinct prediction models were included. Model performance varied greatly, as assessed by area under the receiver operating characteristic curve (0.62–0.94), specificity (0.50–0.97), and sensitivity (0.45–0.96). Most models used data collected from a single time point or window to predict the occurrence of delirium at any point during hospital or ICU admission, and lacked mechanisms for providing pragmatic, actionable predictions to clinicians.

Conclusions:

Although most ICU delirium-prediction models have relatively good performance, they have limited applicability to clinical practice. Most models were static, making predictions based on data collected at a single time-point, failing to account for fluctuating conditions during ICU admission. Further research is needed to create clinically relevant dynamic delirium-prediction models that can adapt to changes in individual patient physiology over time and deliver actionable predictions to clinicians.

Utility of a prediction model for delirium in intensive care unit patients (PRE-DELIRIC) in mechanically ventilated patients with sepsis

Aim

Delirium frequently develops in patients with sepsis during their intensive care unit (ICU) stay, which is associated with increased morbidity and mortality. A prediction model for delirium in patients in ICU, PRE‐DELIRIC, has been utilized in overall ICU patients, but its utility is uncertain among patients with sepsis. This study aims to examine the utility of PRE‐DELIRIC to predict delirium in mechanically ventilated patients with sepsis.

Methods

This is a post hoc analysis of a randomized clinical trial in eight Japanese ICUs, which aimed to evaluate the sedative strategy with/without dexmedetomidine in adult mechanically ventilated patients with sepsis. The Confusion Assessment Method for the ICU was used every day to assess for delirium throughout their ICU stay. We excluded patients who were delirious on the first day of ICU, those who were under sustained coma throughout their ICU stay, and those who stayed in the ICU less than 24 h. The discriminative ability of PRE‐DELIRIC was evaluated by measuring the area under the receiver operating characteristic curve (AUROC).

Results

Of the 201 patients enrolled in the trial, we analyzed 158 patients. The mean age was 69.4 ± 14.0 years, and 99 patients (63%) were men. Delirium occurred at least once during the ICU stay of 63 patients (40%). The AUROC of PRE‐DELIRIC was 0.60 (95% confidence interval, 0.50–0.69). Subgroup analyses indicated that PRE‐DELIRIC was useful in those with Sequential Organ Failure Assessment score >8 with AUROC of 0.65 (95% confidence interval, 0.51–0.77).

Conclusions

The PRE‐DELIRIC model could not predict delirium in mechanically ventilated patients with sepsis.

Validation of PREdiction of DELIRium in ICu patients (PRE-DELIRIC) among patients in intensive care units: A retrospective cohort study

BACKGROUND

An intensive care unit (ICU) delirium prediction tool, PREdiction of DELIRium in ICu patients (PRE-DELIRIC), has been developed and calibrated in a multinational project. However, there is a lack of evidence regarding the predictive ability of the PRE-DELIRIC among Chinese ICU patients.

AIM

To evaluate the predictive validity (discrimination and calibration) of PRE-DELIRIC.

DESIGN

This is a retrospective cohort study.

METHODS

A retrospective cohort study was conducted. Consecutive participants (a) admitted to the ICU for ≥24 hours, (b) aged ≥18 years, and (c) admitted to the ICU for the first time were included. Ten predictors (age, APACHE-II, urgent and admission category, urea level, metabolic acidosis, infection, coma, sedation, and morphine use) assessed within 24 hours upon ICU admission were assessed. Delirium was assessed using the Confusion Assessment Method for ICU. Outcomes included ICU length of stay and mortality. Discrimination and calibration were determined by the areas under the receiver operating characteristic curve (AUROC), box plot, and calibration plot.

RESULTS

A total of 375 ICU patients were included, with 44.0% of patients being delirious. Delirium was significantly associated with age, PRE-DELIRIC score, ICU length of stay, and mortality. The AUROC was 0.81 (95% confidence interval, 0.77-0.86). The optimal cut-off point identified by max Youden index was 49%. The calibration plot of pooled data demonstrated a calibration slope of 0.894 and an intercept of -0.178.

CONCLUSIONS

The PRE-DELIRIC has high predictive value and is suggested to be adopted in ICUs for early initiation of preventive interventions against delirium among high-risk patients.

RELEVANCE TO CLINICAL PRACTICE

Clinicians can adopt the PRE-DELIRIC among ICU patients to screen patients at high risk of developing delirium. Early initiative interventions could be implemented to reduce the negative impacts of ICU delirium.

Cognitive and psychosocial outcomes of mechanically ventilated intensive care patients with and without delirium

Objective

Delirium is common in intensive care patients and is associated with short- and long-term adverse outcomes. We investigated the long-term risk of cognitive impairment and post-traumatic stress disorder (PTSD) in intensive care patients with and without delirium.

Methods

This is a prospective cohort study in ICUs in two Australian university-affiliated hospitals. Patients were eligible if they were older than 18 years, mechanically ventilated for more than 24 h and did not meet exclusion criteria. Delirium was assessed using the Confusion Assessment Method for Intensive Care Unit. Variables assessing cognitive function and PTSD symptoms were collected at ICU discharge, after 6 and 12 months: Mini-Mental State Examination, Telephone Interview for Cognitive Status, Impact of Events Scale-Revised and Informant Questionnaire for Cognitive Decline (caregiver).

Results

103 participants were included of which 36% developed delirium in ICU. Patients with delirium were sicker and had longer duration of mechanical ventilation and ICU length of stay. After 12 months, 41/60 (68.3%) evaluable patients were cognitively impaired, with 11.6% representing the presence of symptoms consistent with dementia. When evaluated by the patient’s caregiver, the patient’s cognitive function was found to be severely impaired in a larger proportion of patients (14/60, 23.3%). Delirium was associated with worse cognitive function at ICU discharge, but not with long-term cognitive function. IES-R scores, measuring PTSD symptoms, were significantly higher in patients who had delirium compared to patients without delirium. In regression analysis, delirium was independently associated with cognitive function at ICU discharge and PTSD symptoms at 12 months.

Conclusions

Intensive care survivors have significant rates of long-term cognitive decline and PTSD symptoms. Delirium in ICU was independently associated with short-term but not long-term cognitive function, and with long-term PTSD symptoms.

Trial registration Australian New Zealand Clinical Trials Registry, ACTRN12616001116415, 15/8/2016 retrospectively registered, https://www.anzctr.org.au

Delirium risk prediction models for intensive care unit patients: A systematic review

OBJECTIVE

To systematically review the delirium risk prediction models for intensive care unit (ICU) patients.

METHODS

A systematic review was conducted. The Cochrane Library, PubMed, Ovid and Web of Science were searched to collect studies on delirium risk prediction models for ICU patients from database establishment to 31 March 2019. Two reviewers independently screened the literature according to the pre-determined inclusion and exclusion criteria, extracted the data and evaluated the risk of bias of the included studies using the CHARMS (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies) checklist. A descriptive analysis was used to describe and summarise the data.

RESULTS

A total of six models were included. All studies reported the area under the receiver operating characteristic curve (AUROC) of the prediction models in the derivation and (or) validation datasets as over 0.7 (from 0.75 to 0.9). Five models reported calibration metrics. Decreased cognitive reserve and the Acute Physiology and Chronic Health Evaluation II (APACHE-II) score were the most commonly reported predisposing and precipitating factors, respectively, of ICU delirium among all models. The small sample size, lack of external validation and the absence of or unreported blinding method increased the risk of bias.

CONCLUSION

According to the discrimination and calibration statistics reported in the original studies, six prediction models may have moderate power in predicting ICU delirium. However, this finding should be interpreted with caution due to the risk of bias in the included studies. More clinical studies should be carried out to validate whether these tools have satisfactory predictive performance in delirium risk prediction for ICU patients.

Delirium prediction in the intensive care unit: a temporal approach

The incidence of delirium in intensive care units is high and associated with poor outcomes; therefore, its prediction is desirable to establish preventive treatments. This retrospective study proposes a novel approach for delirium prediction. We analyzed static and temporal data from 10,475 patients admitted to one of 15 intensive care units (ICUs) in Alberta, Canada between January 1, 2014 and June 30, 2016. We tested 168 different combinations of study design parameters and five different predictive models (logistic regression, support vector machines, random forests, adaptive boosting and neural networks). The area under the receiver operating characteristic curve (AUROC) ranged from 0.754 (CI 95% ± 0.018) to 0.852 (± 0.033), with sensitivity and specificity respectively ranging from 0.739 (CI 95% ± 0.047) to 0.840 (CI 95% ± 0.064), and 0.770 (CI 95% ± 0.030) to 0.865 (CI 95% ± 0.038). These results are similar to previous studies; however, our approach allows for continuous updates and short-term prediction horizons which might provide major advantages.

External Validation of Two Models to Predict Delirium in Critically Ill Adults Using Either the Confusion Assessment Method-ICU or the Intensive Care Delirium Screening Checklist for Delirium Assessment

OBJECTIVES

To externally validate two delirium prediction models (early prediction model for ICU delirium and recalibrated prediction model for ICU delirium) using either the Confusion Assessment Method-ICU or the Intensive Care Delirium Screening Checklist for delirium assessment.

DESIGN

Prospective, multinational cohort study.

SETTING

Eleven ICUs from seven countries in three continents.

PATIENTS

Consecutive, delirium-free adults admitted to the ICU for greater than or equal to 6 hours in whom delirium could be reliably assessed.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The predictors included in each model were collected at the time of ICU admission (early prediction model for ICU delirium) or within 24 hours of ICU admission (recalibrated prediction model for ICU delirium). Delirium was assessed using the Confusion Assessment Method-ICU or the Intensive Care Delirium Screening Checklist. Discrimination was determined using the area under the receiver operating characteristic curve. The predictive performance was determined for the Confusion Assessment Method-ICU and Intensive Care Delirium Screening Checklist cohort, and compared with both prediction models' original reported performance. A total of 1,286 Confusion Assessment Method-ICU-assessed patients and 892 Intensive Care Delirium Screening Checklist-assessed patients were included. Compared with the area under the receiver operating characteristic curve of 0.75 (95% CI, 0.71-0.79) in the original study, the area under the receiver operating characteristic curve of the early prediction model for ICU delirium was 0.67 (95% CI, 0.64-0.71) for delirium as assessed using the Confusion Assessment Method-ICU and 0.70 (95% CI, 0.66-0.74) using the Intensive Care Delirium Screening Checklist. Compared with the original area under the receiver operating characteristic curve of 0.77 (95% CI, 0.74-0.79), the area under the receiver operating characteristic curve of the recalibrated prediction model for ICU delirium was 0.75 (95% CI, 0.72-0.78) for assessing delirium using the Confusion Assessment Method-ICU and 0.71 (95% CI, 0.67-0.75) using the Intensive Care Delirium Screening Checklist.

CONCLUSIONS

Both the early prediction model for ICU delirium and recalibrated prediction model for ICU delirium are externally validated using either the Confusion Assessment Method-ICU or the Intensive Care Delirium Screening Checklist for delirium assessment. Per delirium prediction model, both assessment tools showed a similar moderate-to-good statistical performance. These results support the use of either the early prediction model for ICU delirium or recalibrated prediction model for ICU delirium in ICUs around the world regardless of whether delirium is evaluated with the Confusion Assessment Method-ICU or Intensive Care Delirium Screening Checklist.

Influence of sedation on delirium recognition in critically ill patients: A multinational cohort study

BACKGROUND

Guidelines advocate intensive care unit (ICU) patients be regularly assessed for delirium using either the Confusion Assessment Method for the ICU (CAM-ICU) or the Intensive Care Delirium Screening Checklist (ICDSC). Single-centre studies, primarily with the CAM-ICU, suggest level of sedation may influence delirium screening results.

OBJECTIVE

The objective of this study was to determine the association between level of sedation and delirium occurrence in critically ill patients assessed with either the CAM-ICU or the ICDSC.

METHODS

This was a secondary analysis of a multinational, prospective cohort study performed in nine ICUs from seven countries. Consecutive ICU patients with a Richmond Agitation-Sedation Scale (RASS) of -3 to 0 at the time of delirium assessment where a RASS ≤ 0 was secondary to a sedating medication. Patients were assessed with either the CAM-ICU or the ICDSC. Logistic regression analysis was used to account for factors with the potential to influence level of sedation or delirium occurrence.

RESULTS

Among 1660 patients, 1203 patients underwent 5741 CAM-ICU assessments [9.6% were delirium positive; at RASS = 0 (3.3% were delirium positive), RASS = -1 (19.3%), RASS = -2 (35.1%); RASS = -3 (39.0%)]. The other 457 patients underwent 3210 ICDSC assessments [11.6% delirium positive; at RASS = 0 (4.9% were delirium positive), RASS = -1 (15.8%), RASS = -2 (26.6%); RASS = -3 (20.6%)]. A RASS of -3 was associated with more positive delirium evaluations (odds ratio: 2.31; 95% confidence interval: 1.34-3.98) in the CAM-ICU-assessed patients (vs. the ICDSC-assessed patients). At a RASS of 0, assessment with the CAM-ICU (vs. the ICDSC) was associated with fewer positive delirium evaluations (odds ratio: 0.58; 95% confidence interval: 0.43-0.78). At a RASS of -1 or -2, no association was found between the delirium assessment method used (i.e., CAM-ICU or ICDSC) and a positive delirium evaluation.

CONCLUSIONS

The influence of level of sedation on a delirium assessment result depends on whether the CAM-ICU or ICDSC is used. Bedside ICU nurses should consider these results when evaluating their sedated patients for delirium. Future research is necessary to compare the CAM-ICU and the ICDSC simultaneously in sedated and nonsedated ICU patients.

TRIAL REGISTRATION

ClinicalTrials.gov; NCT02518646.

Development and validation of a postoperative delirium prediction model for patients admitted to an intensive care unit in China: a prospective study

OBJECTIVES

We aimed to develop and validate a postoperative delirium (POD) prediction model for patients admitted to the intensive care unit (ICU).

DESIGN

A prospective study was conducted.

SETTING

The study was conducted in the surgical, cardiovascular surgical and trauma surgical ICUs of an affiliated hospital of a medical university in Heilongjiang Province, China.

PARTICIPANTS

This study included 400 patients (≥18 years old) admitted to the ICU after surgery.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome measure was POD assessment during ICU stay.

RESULTS

The model was developed using 300 consecutive ICU patients and was validated using 100 patients from the same ICUs. The model was based on five risk factors: Physiological and Operative Severity Score for the enumeration of Mortality and morbidity; acid-base disturbance and history of coma, diabetes or hypertension. The model had an area under the receiver operating characteristics curve of 0.852 (95% CI 0.802 to 0.902), Youden index of 0.5789, sensitivity of 70.73% and specificity of 87.16%. The Hosmer-Lemeshow goodness of fit was 5.203 (p=0.736). At a cutoff value of 24.5%, the sensitivity and specificity were 71% and 69%, respectively.

CONCLUSIONS

The model, which used readily available data, exhibited high predictive value regarding risk of ICU-POD at admission. Use of this model may facilitate better implementation of preventive treatments and nursing measures.

Prediction of ICU Delirium: Validation of Current Delirium Predictive Models in Routine Clinical Practice

OBJECTIVES

To investigate the ability of available delirium risk assessment tools to identify patients at risk of delirium in an Australian tertiary ICU.

DESIGN

Prospective observational study.

SETTING

An Australian tertiary ICU.

PATIENTS

All patients admitted to the study ICU between May 8, 2017, and December 31, 2017, were assessed bid for delirium throughout their ICU stay using the Confusion Assessment Method for ICU. Patients were included in this study if they remained in ICU for over 24 hours and were excluded if they were delirious on ICU admission, or if they were unable to be assessed using the Confusion Assessment Method for ICU during their ICU stay. Delirium risk was calculated for each patient using the prediction of delirium in ICU patients, early prediction of delirium in ICU patients, and Lanzhou models. Data required for delirium predictor models were obtained retrospectively from patients medical records.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

There were 803 ICU admissions during the study period, of which 455 met inclusion criteria. 35.2% (n = 160) were Confusion Assessment Method for ICU positive during their ICU admission. Delirious patients had significantly higher Acute Physiology and Chronic Health Evaluation III scores (median, 72 vs 54; p < 0.001), longer ICU (median, 4.8 vs 1.8 d; p < 0.001) and hospital stay (16.0 vs 8.16 d; p < 0.001), greater requirement of invasive mechanical ventilation (70% vs 21.4%; p < 0.001), and increased ICU mortality (6.3% vs 2.4%; p = 0.037). All models included in this study displayed moderate to good discriminative ability. Area under the receiver operating curve for the prediction of delirium in ICU patients was 0.79 (95% CI, 0.75-0.83); recalibrated prediction of delirium in ICU patients was 0.79 (95% CI, 0.75-0.83); early prediction of delirium in ICU patients was 0.72 (95% CI, 0.67-0.77); and the Lanzhou model was 0.77 (95% CI, 0.72-0.81).

CONCLUSIONS

The predictive models evaluated in this study demonstrated moderate to good discriminative ability to predict ICU patients' risk of developing delirium. Models calculated at 24-hours post-ICU admission appear to be more accurate but may have limited utility in practice.