Predicting Delirium Risk Using an Automated Mayo Delirium Prediction Tool: Development and Validation of a Risk-Stratification Model

Machine learning-based model to predict delirium in patients with advanced cancer treated with palliative care: a multicenter, patient-based registry cohort

This study aimed to present a new approach to predict to delirium admitted to the acute palliative care unit. To achieve this, this study employed machine learning model to predict delirium in patients in palliative care and identified the significant features that influenced the model. A multicenter, patient-based registry cohort study in South Korea between January 1, 2019, and December 31, 2020. Delirium was identified by reviewing the medical records based on the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition. The study dataset included 165 patients with delirium among 2314 patients with advanced cancer admitted to the acute palliative care unit. Seven machine learning models, including extreme gradient boosting, adaptive boosting, gradient boosting, light gradient boosting, logistic regression, support vector machine, and random forest, were evaluated to predict delirium in patients with advanced cancer admitted to the acute palliative care unit. An ensemble approach was adopted to determine the optimal model. For k-fold cross-validation, the combination of extreme gradient boosting and random forest provided the best performance, achieving the following accuracy metrics: 68.83% sensitivity, 70.85% specificity, 69.84% balanced accuracy, and 74.55% area under the receiver operating characteristic curve. The performance of the isolated testing dataset was also validated, and the machine learning model was successfully deployed on a public website ( http://ai-wm.khu.ac.kr/Delirium/ ) to provide public access to delirium prediction results in patients with advanced cancer. Furthermore, using feature importance analysis, sex was determined to be the top contributor in predicting delirium, followed by a history of delirium, chemotherapy, smoking status, alcohol consumption, and living with family. Based on a large-scale, multicenter, patient-based registry cohort, a machine learning prediction model for delirium in patients with advanced cancer was developed in South Korea. We believe that this model will assist healthcare providers in treating patients with delirium and advanced cancer.

Factors associated with delirium in a real-world acute-care setting: analysis considering the interdependence of clinical variables with the frailty syndrome

PURPOSE

Delirium risk assessment in the acute-care setting generally does not account for frailty. The objective of this retrospective study was to identify factors associated with delirium, considering the interdependency of clinical variables with frailty syndrome in complex older patients.

METHODS

The clinical records of 587 participants (248 M, median age 84) were reviewed, collecting clinical, anamnestic and pharmacological data. Frailty syndrome was assessed with the Clinical Frailty Scale (CFS). Delirium was the main study endpoint. The correlations of the considered anamnestic and clinical variables with delirium and its subtypes were investigated selecting only those variables not showing a high overlap with frailty. Correlations associated with a 25% excess of frequency of delirium in comparison with the average of the population were considered as statistically significant.

RESULTS

Delirium was detected in 117 (20%) participants. The presence of one among age > 85 years old, CFS > 4 and invasive devices explained 95% of delirium cases. The main factors maximizing delirium incidence at the individual level were dementia, other psychiatric illness, chronic antipsychotic treatment, and invasive devices. The coexistence of three of these parameters was associated with a peak frequency of delirium, ranging from 57 to 61%, mostly hypoactive forms.

CONCLUSIONS

In acute-care wards, frailty exhibited a strong association with delirium during hospitalization, while at the individual level, dementia and the use of antipsychotics remained important risk factors. Modern clinical prediction tools for delirium should account for frailty syndrome.

Online interpretable dynamic prediction models for postoperative delirium after cardiac surgery under cardiopulmonary bypass developed based on machine learning algorithms: A retrospective cohort study

OBJECTIVE

Postoperative delirium (POD) is strongly associated with poor early and long-term prognosis in cardiac surgery patients with cardiopulmonary bypass (CPB). This study aimed to develop dynamic prediction models for POD after cardiac surgery under CPB using machine learning (ML) algorithms.

METHODS

From July 2021 to June 2022, clinical data were collected from patients undergoing cardiac surgery under CPB at Nanjing First Hospital. A dataset from the same center (October 2022 to November 2022) was also used for temporal external validation. We used ML and deep learning to build models in the training set, optimized parameters in the test set, and finally validated the best model in the validation set. The SHapley Additive exPlanations (SHAP) method was introduced to explain the best models.

RESULTS

Of the 885 patients enrolled, 221 (25.0%) developed POD. 22 (22.0%) of 100 validation cohort patients developed POD. The preoperative and postoperative artificial neural network (ANN) models exhibited optimal performance. The validation results demonstrated satisfactory predictive performance of the ANN model, with area under the receiver operator characteristic curve (AUROC) values of 0.776 and 0.684 for the preoperative and postoperative models, respectively. Based on the ANN algorithm, we constructed dynamic, highly accurate, and interpretable web risk calculators for POD.

CONCLUSIONS

We successfully developed online interpretable dynamic ANN models as clinical decision aids to identify patients at high risk of POD before and after cardiac surgery to facilitate early intervention or care.

Assessing the Risk of Developing Delirium on Admission to Inpatient Rehabilitation: A Clinical Prediction Model

OBJECTIVES

To develop a clinical model to predict the risk of an individual patient developing delirium during inpatient rehabilitation, based on patient characteristics and clinical data available on admission.

DESIGN

Retrospective observational study based on electronic health record data.

SETTING AND PARTICIPANTS

We studied a previously validated data set of inpatients including incident delirium episodes during rehabilitation. These patients were admitted to ZURZACH Care, Rehaklinik Bad Zurzach, a Swiss inpatient rehabilitation clinic, between January 1, 2015, and December 31, 2018.

METHODS

We performed logistic regression analysis using backward and forward selection with alpha = 0.01 to remove any noninformative potential predictor. We subsequentially used the Akaike information criterion (AIC) to select the final model among the resulting "intermediate" models. Discrimination of the final prediction model was evaluated using the C-statistic.

RESULTS

Of the 20 candidate predictor variables, 6 were included in the final prediction model: a linear spline of age with 1 knot at 60 years and a linear spline of the functional independence measure (FIM), a measure of the functional degree of patients independency, with 1 knot at 64 points, diagnosis of disorders of fluid, electrolyte, and acid-base balance (E87), use of other analgesic and antipyretics (N02B), use of anti-parkinson drugs (N04B), and an anticholinergic burden score (ACB) of ≥3 points.

CONCLUSIONS AND IMPLICATIONS

Our clinical prediction model could, upon validation, identify patients at risk of incident delirium at admission to inpatient rehabilitation, and thus enable targeted prevention strategies.

Predicting delirium in older non-intensive care unit inpatients: development and validation of the DELIrium risK Tool (DELIKT)

BACKGROUND

Effective delirium prevention could benefit from automatic risk stratification of older inpatients using routinely collected clinical data.

AIM

Primary aim was to develop and validate a delirium prediction model (DELIKT) suitable for implementation in hospitals. Secondary aim was to select an anticholinergic burden scale as a predictor.

METHOD

We used one cohort for model development and another for validation with electronically available data collected within the first 24 h of admission. Included were patients aged ≥ 65, hospitalised ≥ 48 h with no stay > 24 h in an intensive care unit. Predictors, such as administrative and laboratory variables or an anticholinergic burden scale, were selected using a combination of feature selection filter method and forward/backward selection. The final model was based on logistic regression and the DELIKT was derived from the β-coefficients. We report the following performance measures: area under the curve, sensitivity, specificity and odds ratio.

RESULTS

Both cohorts were similar and included over 10,000 patients each (mean age 77.6 ± 7.6 years) with 11% experiencing delirium. The model included nine variables: age, medical department, dementia, hemi-/paraplegia, catheterisation, potassium, creatinine, polypharmacy and the anticholinergic burden measured with the Clinician-rated Anticholinergic Scale (CrAS). The external validation yielded an AUC of 0.795. With a cut-off at 20 points in the DELIKT, we received a sensitivity of 79.7%, specificity of 62.3% and an odds ratio of 5.9 (95% CI 5.2, 6.7).

CONCLUSION

The DELIKT is a potentially automatic tool with predictors from standard care including the CrAS to identify patients at high risk for delirium.

Implementing a Delirium Risk Stratification Tool and Rounds to Identify and Prevent Delirium in Hospitalized Older Adults

BACKGROUND

Up to 40% of delirium cases are preventable, and early identification is key to improve patient outcomes.

PURPOSE

To implement and evaluate a multidisciplinary delirium intervention program.

INTERVENTION

The delirium intervention program targeted patients at high risk for delirium and included patient and nurse education, risk stratification, multidisciplinary rounds, a nonpharmacological intervention bundle, and a treatment order set.

RESULTS

After implementation, there was a reduction in length of stay of 6.3 days ( P = .01), a 24% decrease in disposition to a skilled nursing facility ( P = .05), and increased detection of delirium by nurses.

CONCLUSION

Positive patient outcomes were achieved by employing a multifactorial approach for delirium identification, prevention, and management. The components of this quality improvement project provide guidance to hospitals seeking to develop a delirium intervention program.

Natural Language Processing CAM Algorithm Improves Delirium Detection Compared With Conventional Methods

Delirium is known to be underdiagnosed and underdocumented. Delirium detection in retrospective studies occurs mostly by clinician diagnosis or nursing documentation. This study aims to assess the effectiveness of natural language processing-confusion assessment method (NLP-CAM) algorithm when compared to conventional modalities of delirium detection. A multicenter retrospective study analyzed 4351 COVID-19 hospitalized patient records to identify delirium occurrence utilizing three different delirium detection modalities namely clinician diagnosis, nursing documentation, and the NLP-CAM algorithm. Delirium detection by any of the 3 methods is considered positive for delirium occurrence as a comparison. NLP-CAM captured 80% of overall delirium, followed by clinician diagnosis at 55%, and nursing flowsheet documentation at 43%. Increase in age, Charlson comorbidity score, and length of hospitalization had increased delirium detection odds regardless of the detection method. Artificial intelligence-based NLP-CAM algorithm, compared to conventional methods, improved delirium detection from electronic health records and holds promise in delirium diagnostics.

Diagnosis of delirium: a practical approach

Delirium is an acute disorder of fluctuating attention and awareness with cardinal features that allow it to be positively distinguished from other causes of an acute confusional state. These features include fluctuations, prominent inattentiveness with other cognitive deficits, a change in awareness and visual hallucinations. We describe a framework for diagnosing delirium, noting the need to consider certain caveats and differential diagnoses. Delirium is a clinical diagnosis where a thorough history and clinical examination are much more helpful diagnostically than any single test or combination of tests.

Forecasting Postoperative Delirium in Older Adult Patients with Fast-and-Frugal Decision Trees

Postoperative delirium (POD) is associated with increased complication and mortality rates, particularly among older adult patients. However, guideline recommendations for POD detection and management are poorly implemented. Fast-and-frugal trees (FFTrees), which are simple prediction algorithms, may be useful in this context. We compared the capacity of simple FFTrees with two more complex models—namely, unconstrained classification trees (UDTs) and logistic regression (LogReg)—for the prediction of POD among older surgical patients in the perioperative setting. Models were trained and tested on the European BioCog project clinical dataset. Based on the entire dataset, two different FFTrees were developed for the pre-operative and postoperative settings. Within the pre-operative setting, FFTrees outperformed the more complex UDT algorithm with respect to predictive balanced accuracy, nearing the prediction level of the logistic regression. Within the postoperative setting, FFTrees outperformed both complex models. Applying the best-performing algorithms to the full datasets, we proposed an FFTree using four cues (Charlson Comorbidity Index (CCI), site of surgery, physical status and frailty status) for the pre-operative setting and an FFTree containing only three cues (duration of anesthesia, age and CCI) for the postoperative setting. Given that both FFTrees contained considerably fewer criteria, which can be easily memorized and applied by health professionals in daily routine, FFTrees could help identify patients requiring intensified POD screening.

Validation and Recalibration of Modified Mayo Delirium Prediction (MDP) Tool in a Hospitalized Cohort

BACKGROUND

Delirium prediction can augment and optimize care of older adults. Mayo delirium prediction (MDP) tool is a robust tool, developed from a large retrospective data set. MDP tool predicts delirium risk for hospitalized older adults, within 24 hours of hospital admission, based on risk factor information available from electronic health record. We intend to validate the prediction performance of this tool and optimize the tool for clinical use.

DESIGN

Observational cohort study SETTINGS: Mayo Clinic Hospitals, Rochester, MN PARTICIPANTS: All Hospitalized older adults (age >50 years) from December 2019 to June 2020. Patients with an admitting diagnosis of substance use disorder were excluded.

INTERVENTION

Original MDP tool was modified to adjust for the fall risk variable as a binary variable that will facilitate broader applicability across different fall risk tools. The modified MDP tool was validated in the retrospective derivation and validation data set which yielded similar prediction capability (AUROC = 0.85, 0.83 respectively).

MEASUREMENTS

Diagnosis of delirium was captured by flowsheet diagnosis of delirium documented by nursing staff in medical record. Predictive variable data were collected daily.

RESULTS

8055 patients were included in the study (median age 71 years). The modified MDP tool delirium prediction compared to delirium occurrence was 4% in the low-risk group, 17.8% in the medium-risk group, and 45.3% in the high-risk group (AUROC of 0.80). Recalibration of the tool was attempted to further optimize the tool that resulted in both simplification and increased performance (AUROC 0.82). The simplified tool was able to predict delirium in hospitalized patients admitted to both medical and surgical services.

CONCLUSIONS

Validation of modified MDP tool revealed good prediction capabilities. Recalibration resulted in simplification with increased performance of the tool in both medical and surgical hospitalized patients.

REcognizing DElirium in geriatric Emergency Medicine: The REDEEM risk stratification score

OBJECTIVE

The objective was to derive a risk score that uses variables available early during the emergency department (ED) encounter to identify high-risk geriatric patients who may benefit from delirium screening.

METHODS

This was an observational study of older adults age ≥ 75 years who presented to an academic ED and who were screened for delirium during their ED visit. Variable selection from candidate predictors was performed through a LASSO-penalized logistic regression. A risk score was derived from the final prediction model, and predictive accuracy characteristics were calculated with 95% confidence intervals (CIs).

RESULTS

From the 967 eligible ED visits, delirium was detected in 107 (11.1%). The area under the curve for the REcognizing DElirium in Emergency Medicine (REDEEM) score was 0.901 (95% CI = 0.864-0.938). The REEDEM risk score included 10 different variables (seven based on triage information and three obtained during early history taking) with a score ranging from -3 to 66. Using an optimal cutoff of ≥11, we found a sensitivity of 84.1% (90 of 107 ED delirium patients, 95% CI = 75.5%-90.2%) and a specificity of 86.6% (745 of 860 non-ED delirium patients, 95% CI = 84.1%-88.8%). A lower cutoff of ≥5 was found to minimize false negatives with an improved sensitivity at 91.6% (98 of 107 ED delirium patients, 95% CI = 84.2%-95.8%).

CONCLUSION

A risk stratification score was derived with the potential to augment delirium recognition in geriatric ED patients. This has the potential to assist on delirium-targeted screening of high-risk patients in the ED. Validation of REDEEM, however, is needed prior to implementation.

Delirium occurrence and association with outcomes in hospitalized COVID-19 patients

Delirium is reported to be one of the manifestations of coronavirus infectious disease 2019 (COVID-19) infection. COVID-19 hospitalized patients are at a higher risk of delirium. Pathophysiology behind the association of delirium and COVID-19 is uncertain. We analyzed the association of delirium occurrence with outcomes in hospitalized COVID-19 patients, across all age groups, at Mayo Clinic hospitals.A retrospective study of all hospitalized COVID-19 patients at Mayo Clinic between March 1, 2020 and December 31, 2020 was performed. Occurrence of delirium and outcomes of mortality, length of stay, readmission, and 30-day mortality after hospital discharge were measured. Chi-square test, student t-test, survival analysis, and logistic regression analysis were performed to measure and compare outcomes of delirium group adjusted for age, sex, Charlson comorbidity score, and COVID-19 severity with no-delirium group.A total of 4351 COVID-19 patients were included in the study. Delirium occurrence in the overall study population was noted to be 22.4%. The highest occurrence of delirium was also noted in patients with critical COVID-19 illness severity. A statistically significant OR 4.35 (3.27-5.83) for in-hospital mortality and an OR 4.54 (3.25-6.38) for 30-day mortality after discharge in the delirium group were noted. Increased hospital length of stay, 30-day readmission, and need for skilled nursing facility on discharge were noted in the delirium group. Delirium in hospitalized COVID-19 patients is a marker for increased mortality and morbidity. In this group, outcomes appear to be much worse when patients are older and have a critical severity of COVID-19 illness.