Prediction of long-term mortality in ICU patients: model validation and assessing the effect of using in-hospital versus long-term mortality on benchmarking

Establishing an artificial intelligence-based predictive model for long-term health-related quality of life for infected patients in the ICU

Objective

To develop a model using a Chinese ICU infection patient database to predict long-term health-related quality of life (HRQOL) in survivors.

Methods

A patient database from the ICU of the Fourth People's Hospital in Zigong was analyzed, including data from 2019 to 2020. The subjects of the study were ICU infection survivors, and their post-discharge HRQOL was assessed through the SF-36 survey. The primary outcomes were the physical component summary (PCS) and mental component summary (MCS). We used artificial intelligence techniques for both feature selection and model building. Least absolute shrinkage and selection operator regression was used for feature selection, extreme gradient boosting (XGBoost) was used for model building, and the area under the receiver operating characteristic curve (AUROC) was used to assess model performance.

Results

The study included 917 ICU infection survivors. The median follow-up was 507.8 days. Their SF-36 scores, including PCS and MCS, were below the national average. The final prognostic model showed an AUROC of 0.72 for PCS and 0.63 for MCS. Within the sepsis subgroup, the predictive model AUROC values for PCS and MCS were 0.76 and 0.68, respectively.

Conclusions

This study established a valuable prognostic model using artificial intelligence to predict long-term HRQOL in ICU infection patients, which supports clinical decision making, but requires further optimization and validation.

External Validation of Mortality Prediction Models for Critical Illness Reveals Preserved Discrimination but Poor Calibration

OBJECTIVES

In a recent scoping review, we identified 43 mortality prediction models for critically ill patients. We aimed to assess the performances of these models through external validation.

DESIGN

Multicenter study.

SETTING

External validation of models was performed in the Simple Intensive Care Studies-I (SICS-I) and the Finnish Acute Kidney Injury (FINNAKI) study.

PATIENTS

The SICS-I study consisted of 1,075 patients, and the FINNAKI study consisted of 2,901 critically ill patients.

MEASUREMENTS AND MAIN RESULTS

For each model, we assessed: 1) the original publications for the data needed for model reconstruction, 2) availability of the variables, 3) model performance in two independent cohorts, and 4) the effects of recalibration on model performance. The models were recalibrated using data of the SICS-I and subsequently validated using data of the FINNAKI study. We evaluated overall model performance using various indexes, including the (scaled) Brier score, discrimination (area under the curve of the receiver operating characteristics), calibration (intercepts and slopes), and decision curves. Eleven models (26%) could be externally validated. The Acute Physiology And Chronic Health Evaluation (APACHE) II, APACHE IV, Simplified Acute Physiology Score (SAPS)-Reduced (SAPS-R)' and Simplified Mortality Score for the ICU models showed the best scaled Brier scores of 0.11' 0.10' 0.10' and 0.06' respectively. SAPS II, APACHE II, and APACHE IV discriminated best; overall discrimination of models ranged from area under the curve of the receiver operating characteristics of 0.63 (0.61-0.66) to 0.83 (0.81-0.85). We observed poor calibration in most models, which improved to at least moderate after recalibration of intercepts and slopes. The decision curve showed a positive net benefit in the 0-60% threshold probability range for APACHE IV and SAPS-R.

CONCLUSIONS

In only 11 out of 43 available mortality prediction models, the performance could be studied using two cohorts of critically ill patients. External validation showed that the discriminative ability of APACHE II, APACHE IV, and SAPS II was acceptable to excellent, whereas calibration was poor.

Long-term Survival and Quality of Life among Survivors Discharged from a Respiratory ICU in North India: A Prospective Study

Background

Advancements in the intensive care unit (ICU) have improved critically ill subjects' short-term outcomes. However, there is a need to understand the long-term outcomes of these subjects. Herein, we study the long-term outcomes and factors associated with poor outcomes in critically ill subjects with medical illnesses.

Materials and methods

All subjects (≥12 years) discharged after an ICU stay of at least 48 hours were included. We evaluated the subjects at 3 and 6 months after ICU discharge. At each visit, subjects were administered the World Health Organization Quality of Life Instrument (WHO-QOL-BREF) questionnaire. The primary outcome was mortality at 6 months after ICU discharge. The key secondary outcome was quality of life (QOL) at 6 months.

Results

In total, 265 subjects were admitted to the ICU, of whom 53 subjects (20%) died in the ICU, and 54 were excluded. Finally, 158 subjects were included: 10 (6.3%) subjects were lost to follow-up. The mortality at 6 months was 17.7% (28/158). Most subjects [16.5% (26/158)] died within the initial 3 months after ICU discharge. Quality of life scores were low in all the domains of WHO-QOL-BREF. About 12% (n = 14) of subjects could not perform the activity of daily living at 6 months. After adjusting for covariates, ICU-acquired weakness at the time of discharge (OR 15.12; 95% CI, 2.08-109.81, p <0.01) and requirement for home ventilation (OR 22; 95% CI, 3.1-155, p <0.01) were associated with mortality at 6 months.

Conclusion

Intensive care unit survivors have a high risk of death and a poor QOL during the initial 6 months following discharge.

How to cite this article

Kodati R, Muthu V, Agarwal R, Dhooria S, Aggarwal AN, Prasad KT, et al. Long-term Survival and Quality of Life among Survivors Discharged from a Respiratory ICU in North India: A Prospective Study. Indian J Crit Care Med 2022;26(10):1078-1085.

Superiority of Simplified Acute Physiologic Score II Compared with Acute Physiologic and Chronic Health Evaluation II and Sequential Organ Failure Assessment Scores for Predicting 48-Hour Mortality in Patients Receiving Continuous Kidney Replacement Therapy

BACKGROUND

Predicting early mortality is important in patients undergoing continuous kidney replacement therapy (CKRT), especially in the first 48 h. This study aimed to determine the predictive performance of the Simplified Acute Physiologic Score (SAPS) II, the Acute Physiologic and Chronic Health Evaluation (APACHE) II, and the Sequential Organ Failure Assessment (SOFA) scores for early mortality in patients receiving CKRT.

METHODS

Data from patients with acute kidney injury receiving CKRT were consecutively and retrospectively obtained at a tertiary medical center between August 2017 and March 2021. The outcomes included 48-h and 7-day mortality. The scoring systems were evaluated via discrimination at the time of CKRT initiation (using area under the receiver operating characteristics curve [AUROC]) and calibration (via Hosmer-Lemeshow goodness-of-fit C statistics).

RESULTS

Among eligible 652 patients, 95 (14.6%) and 212 (32.5%) died within 48 h and within 7 days, respectively. The AUROC for SAPS II (0.71, 95% confidence interval [CI]: 0.65-0.77, p = 0.016 vs. APACHE II score, p = 0.044 vs. SOFA score) was significantly higher than that of the APACHE II (0.66, 95% CI: 0.60-0.72) and SOFA scores (0.66, 95% CI: 0.60-0.72) for 48-h mortality. However, no significant differences in the AUROCs for SAPS II, APACHE II, and SOFA scores for 7-day mortality were observed. The calibration of the SAPS II for 48-h and 7-day mortality was adequate (p = 0.507 and p = 0.141, respectively).

CONCLUSIONS

The predictive performance of SAPS II for mortality within the first 48 h was superior to that of the APACHE II and SOFA scores in patients with acute kidney injury receiving CKRT.

Place of death and phenomenon of going home to die in Chinese adults: A prospective cohort study

BACKGROUND

China is embracing an ageing population without sustainable end-of-life care services. However, changes in place of death and trends of going home to die (GHTD) from the hospital remains unknown.

METHODS

A total of 42,956 participants from the China Kadoorie Biobank, a large Chinese cohort, who died between 2009 and 2017 was included into analysis. GHTD was defined as death at home within 7 days after discharge from the hospital. A modified Poisson regression was used to investigate temporal trends of the place of death and GHTD, and estimate prevalence ratios (PRs) and 95% confidence intervals (CIs) for the association of GHTD with health insurance (HI) schemes.

FINDINGS

From 2009 to 2017, home remained the most common place of death (71·5%), followed by the hospital (21·6%). The proportion of GHTD for Urban and Rural Residents' Basic Medical Insurance (URRBMI) beneficiaries was around six times higher than that for Urban Employee Basic Medical Insurance (UEBMI) beneficiaries (66·0% vs 11·6%). Besides, a substantial increase in the proportion of GHTD throughout the study period was observed regardless of HI schemes (4·4% annually for URRBMI, and 5·4% for UEBMI). Compared with UEBMI beneficiaries, URRBMI beneficiaries were more likely to experience GHTD, with an adjusted PR (95% CI) of 1·19 (95% CI: 1·12, 1·27) (P<0·001).

INTERPRETATION

In China, most of deaths occurred at home, with a large proportion of decedents GHTD from the hospital, especially for URRBMI beneficiaries. Substantial variation in the phenomenon of GHTD across HI schemes indicates inequalities in end-of-life care utilization.

FUNDING

The National Natural Science Foundation of China, the Kadoorie Charitable Foundation, the National Key R&D Program of China, the Chinese Ministry of Science and Technology.

Variation in severity-adjusted resource use and outcome in intensive care units

PURPOSE

Intensive care patients have increased risk of death and their care is expensive. We investigated whether risk-adjusted mortality and resources used to achieve survivors change over time and if their variation is associated with variables related to intensive care unit (ICU) organization and structure.

METHODS

Data of 207,131 patients treated in 2008-2017 in 21 ICUs in Finland, Estonia and Switzerland were extracted from a benchmarking database. Resource use was measured using ICU length of stay, daily Therapeutic Intervention Scoring System Scores (TISS) and purchasing power parity-adjusted direct costs (2015-2017; 17 ICUs). The ratio of observed to severity-adjusted expected resource use (standardized resource use ratio; SRUR) was calculated. The number of expected survivors and the ratio of observed to expected mortality (standardized mortality ratio; SMR) was based on a mortality prediction model covering 2015-2017. Fourteen a priori variables reflecting structure and organization were used as explanatory variables for SRURs in multivariable models.

RESULTS

SMR decreased over time, whereas SRUR remained unchanged, except for decreased TISS-based SRUR. Direct costs of one ICU day, TISS score and ICU admission varied between ICUs 2.5-5-fold. Differences between individual ICUs in both SRUR and SMR were up to > 3-fold, and their evolution was highly variable, without clear association between SRUR and SMR. High patient turnover was consistently associated with low SRUR but not with SMR.

CONCLUSION

The wide and independent variation in both SMR and SRUR suggests that they should be used together to compare the performance of different ICUs or an individual ICU over time.

Is intensive care unit mortality a valid survival outcome measure related to critical illness?

RATIONALE

Use of death as an outcome of intensive care unit (ICU) admission may be biased by differential discharge decisions.

OBJECTIVE

To determine the validity of ICU survival status as an outcome measure of all cause case-fatality.

METHODS

A retrospective cohort of first admissions among adults to four ICUs in North Brisbane, Australia was assembled. Death in ICU (censored at discharge or 30 days) was compared with 30-day overall case-fatality.

RESULTS

The 30-day overall case-fatality was 8.1% (2436/29,939). One thousand six hundred and thirty-one deaths occurred within the ICU stay and 576 subsequent during hospital post-ICU discharge within 30-days; ICU and hospital case-fatality rates were 5.4% and 7.4%, respectively. An additional 229 patients died after hospital separation within 30 days of ICU admission of which 110 (48.0%) were transferred to another acute care hospital, 80 (34.9%) discharged home, and 39 (17.0%) transferred to an aged care/chronic care/rehabilitation facility. Patients who died after ICU discharge were older, had higher APACHE III scores, were more likely to be elective surgical patients, and were less likely to be out of state residents or managed in a tertiary referral hospital. Limiting determination of case-fatality to ICU information alone would correctly detect 95% (780/821) of all-cause mortality at day 3, 90% (1093/1213) at day 5, 75% (1524/2019) at day 15, 72% (1592/2244) at day 21, and 67% (1631/2436) at day 30 of follow-up.

CONCLUSIONS

Use of ICU case-fatality significantly underestimates the true burden and biases assessment of determinants of critical illness-related mortality in our region.

Development of a practically usable prediction model for quality of life of ICU survivors: A sub-analysis of the MONITOR-IC prospective cohort study

PURPOSE

As the goal of ICU treatment is survival in good health, we aimed to develop a prediction model for ICU survivors' change in quality of life (QoL) one year after ICU admission.

MATERIALS & METHODS

This is a sub-study of the prospective cohort MONITOR-IC study. Adults admitted ≥12 h to the ICU of a university hospital between July 2016-January 2019 were included. Moribund patients were excluded. Change in QoL one year after ICU admission was quantified using the EuroQol five-dimensional (EQ-5D-5L) questionnaire, and Short-Form 36 (SF-36). Multivariable linear regression analysis and best subsets regression analysis (SRA) were used. Models were internally validated by bootstrapping.

RESULTS

The PREdicting PAtients' long-term outcome for Recovery (PREPARE) model was developed (n = 1308 ICU survivors). The EQ-5D-models had better predictive performance than the SF-36-models. Explained variance (adjusted R²) of the best model (33 predictors) was 58.0%. SRA reduced the number of predictors to 5 (adjusted R² = 55.3%, SE = 0.3), including QoL, diagnosis of a Cardiovascular Incident and frailty before admission, sex, and ICU-admission following planned surgery.

CONCLUSIONS

Though more long-term data are needed to ascertain model accuracy, in future, the PREPARE model may be used to better inform and prepare patients and their families for ICU recovery.

Impact on Patient Outcomes of Pharmacist Participation in Multidisciplinary Critical Care Teams: A Systematic Review and Meta-Analysis

OBJECTIVES

The objective of this systematic review and meta-analysis was to assess the effects of including critical care pharmacists in multidisciplinary ICU teams on clinical outcomes including mortality, ICU length of stay, and adverse drug events.

DATA SOURCES

PubMed, EMBASE, and references from previous relevant systematic studies.

STUDY SELECTION

We included randomized controlled trials and nonrandomized studies that reported clinical outcomes such as mortality, ICU length of stay, and adverse drug events in groups with and without critical care pharmacist interventions.

DATA EXTRACTION

We extracted study details, patient characteristics, and clinical outcomes.

DATA SYNTHESIS

From the 4,725 articles identified as potentially eligible, 14 were included in the analysis. Intervention of critical care pharmacists as part of the multidisciplinary ICU team care was significantly associated with the reduced likelihood of mortality (odds ratio, 0.78; 95% CI, 0.73-0.83; p < 0.00001) compared with no intervention. The mean difference in ICU length of stay was -1.33 days (95% CI, -1.75 to -0.90 d; p < 0.00001) for mixed ICUs. The reduction of adverse drug event prevalence was also significantly associated with multidisciplinary team care involving pharmacist intervention (odds ratio for preventable and nonpreventable adverse drug events, 0.26; 95% CI, 0.15-0.44; p < 0.00001 and odds ratio, 0.47; 95% CI, 0.28-0.77; p = 0.003, respectively).

CONCLUSIONS

Including critical care pharmacists in the multidisciplinary ICU team improved patient outcomes including mortality, ICU length of stay in mixed ICUs, and preventable/nonpreventable adverse drug events.

Mortality Prediction After Cardiac Surgery: Higgins' Intensive Care Unit Admission Score Revisited

BACKGROUND

This study was performed to develop and validate a cardiac surgical intensive care risk adjustment model for mixed cardiac surgery based on a few preoperative laboratory tests, extracorporeal circulation time, and measurements at arrival to the intensive care unit.

METHODS

This was a retrospective study of admissions to 5 cardiac surgical intensive care units in Sweden that submitted data to the Swedish Intensive Care Registry. Admissions from 2008 to 2014 (n = 21,450) were used for model development, whereas admissions from 2015 to 2016 (n = 6463) were used for validation. Models were built using logistic regression with transformation of raw values or categorization into groups.

RESULTS

The final model showed good performance, with an area under the receiver operating characteristics curve of 0.86 (95% confidence interval, 0.83-0.89), a Cox calibration intercept of -0.16 (95% confidence interval, -0.47 to 0.19), and a slope of 1.01 (95% confidence interval, 0.89-1.13) in the validation cohort.

CONCLUSIONS

Eleven variables available on admission to the intensive care unit can be used to predict 30-day mortality after cardiac surgery. The model performance was better than those of general intensive care risk adjustment models used in cardiac surgical intensive care and also avoided the subjective assessment of the cause of admission. The standardized mortality ratio improves over time in Swedish cardiac surgical intensive care.

Mortality prediction models in the adult critically ill: A scoping review

BACKGROUND

Mortality prediction models are applied in the intensive care unit (ICU) to stratify patients into different risk categories and to facilitate benchmarking. To ensure that the correct prediction models are applied for these purposes, the best performing models must be identified. As a first step, we aimed to establish a systematic review of mortality prediction models in critically ill patients.

METHODS

Mortality prediction models were searched in four databases using the following criteria: developed for use in adult ICU patients in high-income countries, with mortality as primary or secondary outcome. Characteristics and performance measures of the models were summarized. Performance was presented in terms of discrimination, calibration and overall performance measures presented in the original publication.

RESULTS

In total, 43 mortality prediction models were included in the final analysis. In all, 15 models were only internally validated (35%), 13 externally (30%) and 10 (23%) were both internally and externally validated by the original researchers. Discrimination was assessed in 42 models (98%). Commonly used calibration measures were the Hosmer-Lemeshow test (60%) and the calibration plot (28%). Calibration was not assessed in 11 models (26%). Overall performance was assessed in the Brier score (19%) and the Nagelkerke's R² (4.7%).

CONCLUSIONS

Mortality prediction models have varying methodology, and validation and performance of individual models differ. External validation by the original researchers is often lacking and head-to-head comparisons are urgently needed to identify the best performing mortality prediction models for guiding clinical care and research in different settings and populations.

External validation of the Simplified Mortality Score for the Intensive Care Unit (SMS-ICU)

BACKGROUND

The Simplified Mortality Score for the Intensive Care Unit (SMS-ICU) is a clinical prediction model, which estimates the risk of 90-day mortality in acutely ill adult ICU patients using 7 readily available variables. We aimed to externally validate the SMS-ICU and compare its discrimination with existing prediction models used with 90-day mortality as the outcome.

METHODS

We externally validated the SMS-ICU using data from 3282 patients included in the Stress Ulcer Prophylaxis in the Intensive Care Unit trial, which randomised acutely ill adult ICU patients with risk factors for gastrointestinal bleeding to prophylactic pantoprazole or placebo in 33 ICUs in Europe. We assessed discrimination, calibration and overall performance of the SMS-ICU and compared discrimination with the commonly used and more complex SAPS II and SOFA scores.

RESULTS

Mortality at day 90 was 30.7%. The discrimination (area under the receiver operating characteristic curve) for the SMS-ICU was 0.67 (95% CI: 0.65-0.69), as compared with 0.68 (95% CI: 0.66-0.70, P = 0.35) for SAPS II and 0.63 (95% CI: 0.61-0.65, P < 0.001) for the SOFA score. Calibration (intercept and slope) was 0.001 and 0.786, respectively, and Nagelkerke's R² (overall performance) was 0.06. The proportions of missing data for the SMS-ICU, SAPS II and SOFA scores were 0.2%, 8.5% and 6.8%, respectively.

CONCLUSIONS

Discrimination for 90-day mortality of the SMS-ICU in this cohort was poor, but similar to SAPS II and better than that of the SOFA score with markedly less missing data.

Performance of SAPS II according to ICU length of stay: A Danish nationwide cohort study

BACKGROUND

Intensive care unit (ICU) severity scores use data available at admission or shortly thereafter. There are limited contemporary data on how the prognostic performance of these scores is affected by ICU length of stay (LOS).

METHODS

We conducted a nationwide cohort study using routinely collected health data from the Danish Intensive Care Database. We included adults with ICU admissions ≥24 hours between 1 January 2012 and 30 June 2016, who survived to ICU discharge and had valid ICU LOS and vital status data registered. We assessed discrimination of the Simplified Acute Physiology Score (SAPS) II for predicting mortality 90 days after ICU discharge, followed by recalibration of the model and assessment of standardized mortality ratios (SMRs) and calibration. Performance was assessed in the entire cohort and stratified by ICU LOS quartiles.

RESULTS

We included 44 523 patients. Increasing SAPS II was associated with increasing ICU LOS. Overall discrimination (area under the receiver-operating characteristics curve) of SAPS II was 0.70 (95% CI: 0.70-0.71), with decreasing discrimination from the first (0.75, 95% CI: 0.73-0.76) to the last (0.64, 95% CI: 0.63-0.65) ICU LOS quartile. SMRs were lower (less deaths) than expected in the first ICU LOS quartile and higher (more deaths) than expected in the last two ICU LOS quartiles. Calibration decreased with increasing ICU LOS.

CONCLUSIONS

We observed that discrimination and calibration of SAPS II decreased with increasing ICU LOS, and that this affected SMRs. These findings should be acknowledged when using SAPS II for clinical, research and administrative purposes.

Performance of SAPS II according to ICU length of stay: Protocol for an observational study

BACKGROUND

Severity scores, including the Simplified Acute Physiology Score (SAPS) II, are widely used in the intensive care unit (ICU) to predict mortality outcomes using data from ICU admission or shortly hereafter. For patients with longer ICU length of stay (LOS), the predictive performance of admission-based severity scores may deteriorate compared to patients with shorter ICU LOS. This protocol and statistical analysis plan outlines a study that will assess the influence of ICU LOS on the performance of SAPS II for predicting 90-day post-ICU mortality.

METHODS

A Danish nationwide cohort study including adult (≥18 years) ICU patients admitted to a Danish ICU between 1 January 2012 and 30 June 2016. The study will be conducted using the Danish Intensive Care Database (DID), which contains data routinely, prospectively, and consecutively reported for all Danish ICU admissions. Discrimination of SAPS II for predicting 90-day post-ICU mortality will be assessed for the entire cohort and stratified according to ICU LOS. A first-level recalibration of SAPS II will be performed, and if adequate, standardised mortality ratios and calibration stratified according to ICU LOS will be reported.

CONCLUSIONS

The outlined large, nationwide cohort study will provide important, contemporary information about the influence of ICU LOS on severity score performance relevant for ICU clinicians, researchers, and administrators. Publication of the protocol and statistical analysis plan prior to study conduct ensures transparency, and limits the risk of publication bias, post hoc changes in analyses, and challenges with multiple comparisons.

Out-of-hours intensive care unit admission and 90-day mortality: a Danish nationwide cohort study

BACKGROUND

Mortality rates in critically ill adult patients admitted to the intensive care unit (ICU) remains high, and numerous patient- and disease-related adverse prognostic factors have been identified. In recent years, studies in a variety of emergency conditions suggested that outcome is dependent on the time of hospital admission. The importance of out-of-hours admission to the ICU has been sparsely evaluated and with ambiguous findings. We assessed the association between out-of-hours (16:00 to 07:00) and weekend admission to the ICU, respectively, and 90-day mortality in a nationwide cohort.

METHODS

We included all Danish adult patients admitted to the ICU between 1 January 2011 and 30 June 2014, with an ICU stay > 24 h. The crude and adjusted association between out-of-hours and weekend admission and 90-day mortality was assessed (odds ratio (ORs) with 95% confidence intervals (CI)).

RESULTS

A total of 44,797 patients were included, 53.3% were admitted out-of-hours, and 22.6% during weekends. Median age was 67 years (interquartile range (IQR) 55-76), and median SAPS II was 42 (IQR 30-54). Patients admitted in-hours vs. out-of-hours displayed a 90-day mortality rate of 41.0% vs. 44.2%. The adjusted association (OR with 95% CI) between out-of-hours admission and 90-day mortality was 1.07 (1.02-1.11), and the adjusted association (OR with 95% CI) between weekend admission and 90-day mortality was 1.10 (1.05-1.15).

CONCLUSION

This nationwide study suggests that critically ill adult patients admitted to the ICU during weekends and out-of-hours, and with an ICU stay > 24 h are at slightly increased risk of mortality.

Health-related quality of life in intensive care survivors: Associations with social support, comorbidity, and pain interference

BACKGROUND

Experiences during a stay in the intensive care unit (ICU), including pain, delirium, physical deterioration, and the critical illness itself, may all influence survivors' health-related quality of life (HRQOL). However, few studies have examined the influence of social support, comorbidity, and pain interference on ICU survivors' HRQOL.

OBJECTIVES

To investigate possible associations between social support, number of comorbidities, and pain interference on HRQOL in ICU survivors.

METHODS

ICU survivors responded to a survey 3 months (n = 118) and 1 year (n = 89) after ICU discharge. HRQOL was measured using the Short Form Health Survey-12 (v1), social support using the revised Social Provision Scale, pain interference using the Brief Pain Inventory-Short Form, and comorbidities using the Self-Administered Comorbidity Questionnaire.

RESULTS

Physical and mental HRQOL were reduced at both 3 months and 1 year in ICU survivors compared with the general population. This reduction was more pronounced at 3 months for physical HRQOL, while a small reduction in mental HRQOL was not clinically relevant. Social support was statistical significantly positively associated with mental HRQOL at 3 months, while number of comorbidities was statistical significantly associated with a reduction in physical HRQOL at 3 months and 1 year and mental HRQOL at 1 year. Lastly pain interference was significantly associated with a reduction in physical HRQOL at 3 months and 1 year.

CONCLUSIONS

ICU survivors primarily report reduced physical HRQOL. Social support was positively associated with mental HRQOL, while number of comorbidities, and pain interference were all significantly associated with a reduction in HRQOL. Pain interference was associated with the largest reduction in HRQOL.

The association between outcome-based quality indicators for intensive care units

Purpose

To assess and improve the effectiveness of ICU care, in-hospital mortality rates are often used as principal quality indicator for benchmarking purposes. Two other often used, easily quantifiable, quality indicators to assess the efficiency of ICU care are based on readmission to the ICU and ICU length of stay. Our aim was to examine whether there is an association between case-mix adjusted outcome-based quality indicators in the general ICU population as well as within specific subgroups.

Materials and methods

We included patients admitted in 2015 of all Dutch ICUs. We derived the standardized in-hospital mortality ratio (SMR); the standardized readmission ratio (SRR); and the standardized length of stay ratio (SLOSR). We expressed association through Pearson’s correlation coefficients.

Results

The SMR ranged from 0.6 to 1.5; the SRR ranged from 0.7 to 2.1; and the SLOSR ranged from 0.7 to 1.3. For the total ICU population we found no significant associations. We found a positive, non-significant, association between SMR and SLOSR for admissions with low-mortality risk, (r = 0.25; p = 0.024), and a negative association between these indicators for admissions with high-mortality risk (r = -0.49; p<0.001).

Conclusion

Overall, we found no association at ICU population level. Differential associations were found between performance on mortality and length of stay within different risk strata. We recommend users of quality information to take these three outcome indicators into account when benchmarking ICUs as they capture different aspects of ICU performance. Furthermore, we suggest to report quality indicators for patient subgroups.

Development and internal validation of the Simplified Mortality Score for the Intensive Care Unit (SMS-ICU)

BACKGROUND

Intensive care unit (ICU) mortality prediction scores deteriorate over time, and their complexity decreases clinical applicability and commonly causes problems with missing data. We aimed to develop and internally validate a new and simple score that predicts 90-day mortality in adults upon acute admission to the ICU: the Simplified Mortality Score for the Intensive Care Unit (SMS-ICU).

METHODS

We used data from an international cohort of 2139 patients acutely admitted to the ICU and 1947 ICU patients with severe sepsis/septic shock from 2009 to 2016. We performed multiple imputations for missing data and used binary logistic regression analysis with variable selection by backward elimination, followed by conversion to a simple point-based score. We assessed the apparent performance and validated the score internally using bootstrapping to present optimism-corrected performance estimates.

RESULTS

The SMS-ICU comprises seven variables available in 99.5% of the patients: two numeric variables: age and lowest systolic blood pressure, and five dichotomous variables: haematologic malignancy/metastatic cancer, acute surgical admission and use of vasopressors/inotropes, respiratory support and renal replacement therapy. Discrimination (area under the receiver operating characteristic curve) was 0.72 (95% CI: 0.71-0.74), overall performance (Nagelkerke's R² ) was 0.19 and calibration (intercept and slope) was 0.00 and 0.99, respectively. Optimism-corrected performance was similar to apparent performance.

CONCLUSIONS

The SMS-ICU predicted 90-day mortality with reasonable and stable performance. If performance remains adequate after external validation, the SMS-ICU could prove a valuable tool for ICU clinicians and researchers because of its simplicity and expected very low number of missing values.

Prevalence, Location, and Characteristics of Chronic Pain in Intensive Care Survivors

BACKGROUND

A growing number of studies have addressed the long-term consequences of intensive care unit (ICU) treatment, but few have studied the prevalence of chronic pain and pain characteristics longitudinally.

AIMS

The goal of the work described here was to investigate the prevalence and characteristics of chronic pain in ICU survivors 3 months and 1 year after ICU discharge and to identify risk factors for chronic pain 1 year after ICU discharge.

DESIGN

The design used was an explorative and longitudinal study.

SETTING/PATIENTS

The patients in this work had stayed >48 hours in two mixed ICUs in Oslo University Hospital, a tertiary referral hospital.

METHODS

Patients completed a survey questionnaire 3 months and 1 year after ICU discharge. Pain was assessed using the Brief Pain Inventory-Short Form.

RESULTS

At 3 months after discharge, 58 of 118 ICU survivors (49.2%) reported pain, and at 1 year after discharge, 34 of 89 survivors (38.2%) reported pain. The most common sites of pain at 3 months were the shoulder and abdomen; the shoulder remained the second most common site at 1 year. There was an increase in the interference of pain with daily life at 1 year. Possible risk factors for chronic pain at 1 year were increased severity of illness, organ failure, ventilator time >12 days, and ICU length of stay >15 days. The most common sites of pain were not linked to the admission diagnosis.

CONCLUSIONS

These findings may enable health care providers to improve care and rehabilitation for this patient group.

Development of a prediction model for long-term quality of life in critically ill patients

PURPOSE

We developed a prediction model for quality of life (QOL) 1 year after intensive care unit (ICU) discharge based upon data available at the first ICU day to improve decision-making.

METHODS

The database of a 1-year prospective study concerning long-term outcome and QOL (assessed by EuroQol-5D) in critically ill adult patients consecutively admitted to the ICU of a university hospital was used. Cases with missing data were excluded. Utility indices at baseline (UIb) and at 1 year (UI1y) were surrogates for QOL. For 1-year non-survivors UI1y was set at zero. The grouped lasso technique selected the most important variables in the prediction model. R² and adjusted R² were calculated.

RESULTS

1831 of 1953 cases (93.8%) were complete. UI1y depended significantly on: UIb (P<0.001); solid tumor (P<0.001); age (P<0.001); activity of daily living (P<0.001); imaging (P<0.001); APACHE II-score (P=0.001); ≥80 years (P=0.001); mechanical ventilation (P=0.006); hematological patient (P=0.007); SOFA-score (P=0.008); tracheotomy (P=0.018); admission diagnosis surgical P<0.001 (versus medical); and comorbidity (P=0.049). Only baseline health status and surgical patients were positively associated with UI1y. R² was 0.3875 and adjusted R² 0.3807.

CONCLUSION

Although only 40% of variability in long-term QOL could be explained, this prediction model can be helpful in decision-making.

Comparing Time-Fixed Mortality Prediction Models and Their Effect on ICU Performance Metrics Using the Simplified Acute Physiology Score 3

OBJECTIVES

To examine ICU performance based on the Simplified Acute Physiology Score 3 using 30-day, 90-day, or 180-day mortality as outcome measures and compare results with 30-day mortality as reference.

DESIGN

Retrospective cohort study of ICU admissions from 2010 to 2014.

SETTING

Sixty-three Swedish ICUs that submitted data to the Swedish Intensive Care Registry.

PATIENTS

The development cohort was first admissions to ICU during 2011-2012 (n = 53,546), and the validation cohort was first admissions to ICU during 2013-2014 (n = 57,729).

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Logistic regression was used to develop predictive models based on a first level recalibration of the original Simplified Acute Physiology Score 3 model but with 30-day, 90-day, or 180-day mortality as measures of outcome. Discrimination and calibration were excellent for the development dataset. Validation in the more recent 2013-2014 database showed good discrimination (C-statistic: 0.85, 0.84, and 0.83 for the 30-, 90-, and 180-d models, respectively), and good calibration (standardized mortality ratio: 0.99, 0.99, and 1.00; Hosmer-Lemeshow goodness of fit H-statistic: 66.4, 63.7, and 81.4 for the 30-, 90-, and 180-d models, respectively). There were modest changes in an ICU's standardized mortality ratio grouping (< 1.00, not significant, > 1.00) when follow-up was extended from 30 to 90 days and 180 days, respectively; about 11-13% of all ICUs.

CONCLUSIONS

The recalibrated Simplified Acute Physiology Score 3 hospital outcome prediction model performed well on long-term outcomes. Evaluation of ICU performance using standardized mortality ratio was only modestly sensitive to the follow-up time. Our results suggest that 30-day mortality may be a good benchmark of ICU performance. However, the duration of follow-up must balance between what is most relevant for patients, most affected by ICU care, least affected by administrative policies and practically feasible for caregivers.

Simplified Mortality Score for the Intensive Care Unit (SMS-ICU): protocol for the development and validation of a bedside clinical prediction rule

INTRODUCTION

Mortality prediction scores are widely used in intensive care units (ICUs) and in research, but their predictive value deteriorates as scores age. Existing mortality prediction scores are imprecise and complex, which increases the risk of missing data and decreases the applicability bedside in daily clinical practice. We propose the development and validation of a new, simple and updated clinical prediction rule: the Simplified Mortality Score for use in the Intensive Care Unit (SMS-ICU).

METHODS AND ANALYSIS

During the first phase of the study, we will develop and internally validate a clinical prediction rule that predicts 90-day mortality on ICU admission. The development sample will comprise 4247 adult critically ill patients acutely admitted to the ICU, enrolled in 5 contemporary high-quality ICU studies/trials. The score will be developed using binary logistic regression analysis with backward stepwise elimination of candidate variables, and subsequently be converted into a point-based clinical prediction rule. The general performance, discrimination and calibration of the score will be evaluated, and the score will be internally validated using bootstrapping. During the second phase of the study, the score will be externally validated in a fully independent sample consisting of 3350 patients included in the ongoing Stress Ulcer Prophylaxis in the Intensive Care Unit trial. We will compare the performance of the SMS-ICU to that of existing scores.

ETHICS AND DISSEMINATION

We will use data from patients enrolled in studies/trials already approved by the relevant ethical committees and this study requires no further permissions. The results will be reported in accordance with the Transparent Reporting of multivariate prediction models for Individual Prognosis Or Diagnosis (TRIPOD) statement, and submitted to a peer-reviewed journal.

Predictive Performance of the Simplified Acute Physiology Score (SAPS) II and the Initial Sequential Organ Failure Assessment (SOFA) Score in Acutely Ill Intensive Care Patients: Post-Hoc Analyses of the SUP-ICU Inception Cohort Study

Purpose

Severity scores including the Simplified Acute Physiology Score (SAPS) II and the Sequential Organ Failure Assessment (SOFA) score are used in intensive care units (ICUs) to assess disease severity, predict mortality and in research. We aimed to assess the predictive performance of SAPS II and the initial SOFA score for in-hospital and 90-day mortality in a contemporary international cohort.

Methods

This was a post-hoc study of the Stress Ulcer Prophylaxis in the Intensive Care Unit (SUP-ICU) inception cohort study, which included acutely ill adults from ICUs across 11 countries (n = 1034). We compared the discrimination of SAPS II and initial SOFA scores, compared the discrimination of SAPS II in our cohort with the original cohort, assessed the calibration of SAPS II customised to our cohort, and compared the discrimination for 90-day mortality vs. in-hospital mortality for both scores. Discrimination was evaluated using areas under the receiver operating characteristics curves (AUROC). Calibration was evaluated using Hosmer-Lemeshow’s goodness-of-fit Ĉ-statistic.

Results

AUROC for in-hospital mortality was 0.80 (95% confidence interval (CI) 0.77–0.83) for SAPS II and 0.73 (95% CI 0.69–0.76) for initial SOFA score (P<0.001 for the comparison). Calibration of the customised SAPS II for predicting in-hospital mortality was adequate (P = 0.60). Discrimination of SAPS II was reduced compared with the original SAPS II validation sample (AUROC 0.80 vs. 0.86; P = 0.001). AUROC for 90-day mortality was 0.79 (95% CI 0.76–0.82; P = 0.74 for comparison with in-hospital mortality) for SAPS II and 0.71 (95% CI 0.68–0.75; P = 0.66 for comparison with in-hospital mortality) for the initial SOFA score.

Conclusions

The predictive performance of SAPS II was similar for in-hospital and 90-day mortality and superior to that of the initial SOFA score, but SAPS II’s performance has decreased over time. Use of a contemporary severity score with improved predictive performance may be of value.

Individual and Clustered Rankability of ICUs According to Case-Mix-Adjusted Mortality

OBJECTIVES

The performance of ICUs can be compared by ranking them into a league table according to their risk-adjusted mortality rate. The statistical quality of a league table can be expressed as its rankability, the percentage of variation between ICUs attributable to unexplained differences. We examine whether we can improve the rankability of our league table by using data from a longer period or by grouping ICUs with similar performance constructing a league table of clusters rather than individual ICUs.

DESIGN

We developed a league table for risk-adjusted mortality rate with its rankability. The effect of assessment period was determined using a resampling procedure. Hierarchical clustering was used to obtain clusters of similar ICUs.

PATIENTS

We used data from ICUs participating in the Dutch National Intensive Care Evaluation registry between 2011 and 2013.

MEASUREMENTS AND MAIN RESULTS

We constructed league tables using 157,394 admissions from 78 ICUs with risk-adjusted mortality rate between 5.9% and 13.9% per ICU over the inclusion period. The rankability was 73% for 2013 and 89% for the whole period 2011-2013. Rankability over the year 2013 increased till 98% when clustering ICUs, reaching an optimum at a league table of seven clusters.

CONCLUSIONS

We conclude that, when using data from a single year, the rankability of a league table of Dutch ICUs based on risk-adjusted mortality rate was unacceptably low. We could improve the rankability of this league table by increasing the period of data collection or by grouping similar ICUs into clusters and constructing a league table of clusters of ICUs rather than individual ICUs. Ranking clusters of ICUs could be useful for identifying possible differences in performance between clusters of ICUs.

Short Term Survival after Admission for Heart Failure in Sweden: Applying Multilevel Analyses of Discriminatory Accuracy to Evaluate Institutional Performance

BACKGROUND

Hospital performance is frequently evaluated by analyzing differences between hospital averages in some quality indicators. The results are often expressed as quality charts of hospital variance (e.g., league tables, funnel plots). However, those analyses seldom consider patients heterogeneity around averages, which is of fundamental relevance for a correct evaluation. Therefore, we apply an innovative methodology based on measures of components of variance and discriminatory accuracy to analyze 30-day mortality after hospital discharge with a diagnosis of Heart Failure (HF) in Sweden.

METHODS

We analyzed 36,943 patients aged 45-80 treated in 565 wards at 71 hospitals during 2007-2009. We applied single and multilevel logistic regression analyses to calculate the odds ratios and the area under the receiver-operating characteristic (AUC). We evaluated general hospital and ward effects by quantifying the intra-class correlation coefficient (ICC) and the increment in the AUC obtained by adding random effects in a multilevel regression analysis (MLRA). Finally, the Odds Ratios (ORs) for specific ward and hospital characteristics were interpreted jointly with the proportional change in variance (PCV) and the proportion of ORs in the opposite direction (POOR).

FINDINGS

Overall, the average 30-day mortality was 9%. Using only patient information on age and previous hospitalizations for different diseases we obtained an AUC = 0.727. This value was almost unchanged when adding sex, country of birth as well as hospitals and wards levels. Average mortality was higher in small wards and municipal hospitals but the POOR values were 15% and 16% respectively.

CONCLUSIONS

Swedish wards and hospitals in general performed homogeneously well, resulting in a low 30-day mortality rate after HF. In our study, knowledge on a patient's previous hospitalizations was the best predictor of 30-day mortality, and this information did not improve by knowing the sex and country of birth of the patient or where the patient was treated.

[Quality assurance with administrative data (QSR): follow-up in quality measurement - an analysis of patient records]

The present study analyses the information gain obtained by evaluating adverse events during follow-up compared to the sole analysis of events during the initial hospital stay for quality measurement purposes. The analysis is based on AOK administrative data from the years 2010 to 2012. The analyses were carried out for 10 quality indicators from the 4 QSR sectors knee replacement for osteoarthritis, appendectomy, prostate surgery for benign prostatic syndrome (BPS) and therapeutic cardiac catheterization (PCI) in patients with myocardial infarction. A total of 409,774 AOK cases were included. For almost all indicators considered, a relevant share of complications can be found to have occurred only after discharge from the initial hospitalization (7.7 %-92.6 %). Furthermore, there is only a weak connection between the findings from the first hospitalization and those from the follow-up period (0.0449 < r < 0.1935). 26-66 % of the hospitals will be classified differently based on Standardized Mortality/Morbidity Ratio (SMR) quartiles if follow-up events are included in the quality assessment (with the exception of "Other Complications after PCI" of 14 %). In summary, quality assessment is improved considerably by evaluating the follow-up period for almost all indicators considered. A quality measurement based solely on events in the initial hospital stay obscures relevant adverse events that have an impact on a comparative hospital quality assessment for these indicators.

Ensuring comparisons of health-care providers are fair: development and validation of risk prediction models for critically ill patients

Background

National clinical audit has a key role in ensuring quality in health care. When comparing outcomes between providers, it is essential to take the differing case mix of patients into account to make fair comparisons. Accurate risk prediction models are therefore required.

Objectives

To improve risk prediction models to underpin quality improvement programmes for the critically ill (i.e. patients receiving general or specialist adult critical care or experiencing an in-hospital cardiac arrest).

Design

Risk modelling study nested within prospective data collection.

Setting

Adult (general/specialist) critical care units and acute hospitals in the UK.

Participants

Patients admitted to an adult critical care unit and patients experiencing an in-hospital cardiac arrest attended by the hospital-based resuscitation team.

Interventions

None.

Main outcome measures

Acute hospital mortality (adult critical care); return of spontaneous circulation (ROSC) greater than 20 minutes and survival to hospital discharge (in-hospital cardiac arrest).

Data sources

The Case Mix Programme (adult critical care) and National Cardiac Arrest Audit (in-hospital cardiac arrest).

Results

The current Intensive Care National Audit & Research Centre (ICNARC) model was externally validated using data for 29,626 admissions to critical care units in Scotland (2007–9) and outperformed the Acute Physiology And Chronic Health Evaluation (APACHE) II model in terms of discrimination (c-index 0.848 vs. 0.806) and accuracy (Brier score 0.140 vs. 0.157). A risk prediction model for cardiothoracic critical care was developed using data from 17,002 admissions to five units (2010–12) and validated using data from 10,238 admissions to six units (2013–14). The model included prior location/urgency, blood lactate concentration, Glasgow Coma Scale (GCS) score, age, pH, platelet count, dependency, mean arterial pressure, white blood cell (WBC) count, creatinine level, admission following cardiac surgery and interaction terms, and it had excellent discrimination (c-index 0.904) and accuracy (Brier score 0.055). A risk prediction model for admissions to all (general/specialist) adult critical care units was developed using data from 155,239 admissions to 232 units (2012) and validated using data from 90,017 admissions to 216 units (2013). The model included systolic blood pressure, temperature, heart rate, respiratory rate, partial pressure of oxygen in arterial blood/fraction of inspired oxygen, pH, partial pressure of carbon dioxide in arterial blood, blood lactate concentration, urine output, creatinine level, urea level, sodium level, WBC count, platelet count, GCS score, age, dependency, past medical history, cardiopulmonary resuscitation, prior location/urgency, reason for admission and interaction terms, and it outperformed the current ICNARC model for discrimination and accuracy overall (c-index 0.885 vs. 0.869; Brier score 0.108 vs. 0.115) and across unit types. Risk prediction models for in-hospital cardiac arrest were developed using data from 14,688 arrests in 122 hospitals (2011–12) and validated using data from 7791 arrests in 143 hospitals (2012–13). The models included age, sex (for ROSC > 20 minutes), prior length of stay in hospital, reason for attendance, location of arrest, presenting rhythm, and interactions between rhythm and location. Discrimination for hospital survival exceeded that for ROSC > 20 minutes (c-index 0.811 vs. 0.720).

Limitations

The risk prediction models developed were limited by the data available within the current national clinical audit data sets.

Conclusions

We have developed and validated risk prediction models for cardiothoracic and adult (general and specialist) critical care units and for in-hospital cardiac arrest.

Future work

Future development should include linkage with other routinely collected data to enhance available predictors and outcomes.

Funding details

The National Institute for Health Research Health Services and Delivery Research programme.

In-hospital vs. 30-day mortality in the critically ill - a 2-year Swedish intensive care cohort analysis

BACKGROUND

Standardised mortality ratio (SMR) is a common quality indicator in critical care and is the ratio between observed mortality and expected mortality. Typically, in-hospital mortality is used to derive SMR, but the use of a time-fixed, more objective, end-point has been advocated. This study aimed to determine the relationship between in-hospital mortality and 30-day mortality on a comprehensive Swedish intensive care cohort.

METHODS

A retrospective study on patients >15 years old, from the Swedish Intensive Care Register (SIR), where intensive care unit (ICU) admissions in 2009-2010 were matched with the corresponding hospital admissions in the Swedish Hospital Discharge Register. Recalibrated SAPS (Simplified Acute Physiology Score) 3 models were developed to predict and compare in-hospital and 30-day mortality. SMR based on in-hospital mortality and on 30-day mortality were compared between ICUs and between groups with different case-mixes, discharge destinations and length of hospital stays.

RESULTS

Sixty-five ICUs with 48861 patients, of which 35610 were SAPS 3 scored, were included. Thirty-day mortality (17%) was higher than in-hospital mortality (14%). The SMR based on 30-day mortality and that based on in-hospital mortality differed significantly in 7/53 ICUs, for patients with sepsis, for elective surgery-admissions and in groups categorised according to discharge destination and hospital length of stay.

CONCLUSION

Choice of mortality end-point influences SMR. The extent of the influence depends on hospital-, ICU- and patient cohort characteristics as well as inter-hospital transfer rates, as all these factors influence the difference between SMR based on 30-day mortality and SMR based on in-hospital mortality.

Unconscious trauma patients: outcome differences between southern Finland and Germany-lesson learned from trauma-registry comparisons

PURPOSE

International trauma registry comparisons are scarce and lack standardised methodology. Recently, we performed a 6-year comparison between southern Finland and Germany. Because an outcome difference emerged in the subgroup of unconscious trauma patients, we aimed to identify factors associated with such difference and to further explore the role of trauma registries for evaluating trauma-care quality.

METHODS

Unconscious patients [Glasgow Coma Scale (GCS) 3-8] with severe blunt trauma [Injury Severity Score (ISS) ≥16] from Helsinki University Hospital's trauma registry (TR-THEL) and the German Trauma Registry (TR-DGU) were compared from 2006 to 2011. The primary outcome measure was 30-day in-hospital mortality. Expected mortality was calculated by Revised Injury Severity Classification (RISC) score. Patients were separated into clinically relevant subgroups, for which the standardised mortality ratios (SMR) were calculated and compared between the two trauma registries in order to identify patient groups explaining outcome differences.

RESULTS

Of the 5243 patients from the TR-DGU and 398 from the TR-THEL included, nine subgroups were identified and analyzed separately. Poorer outcome appeared in the Finnish patients with penetrating head injury, and in Finnish patients under 60 years with isolated head injury [TR-DGU SMR = 1.06 (95 % CI = 0.94-1.18) vs. TR-THEL SMR = 2.35 (95 % CI = 1.20-3.50), p = 0.001 and TR-DGU SMR = 1.01 (95 % CI = 0.87-1.16) vs. TR-THEL SMR = 1.40 (95 % CI = 0.99-1.81), p = 0.030]. A closer analysis of these subgroups in the TR-THEL revealed early treatment limitations due to their very poor prognosis, which was not accounted for by the RISC.

CONCLUSION

Trauma registry comparison has several pitfalls needing acknowledgement: the explanation for outcome differences between trauma systems can be a coincidence, a weakness in the scoring system, true variation in the standard of care, or hospitals' reluctance to include patients with hopeless prognosis in registry. We believe, however, that such comparisons are a feasible method for quality control.

Designing optimal mortality risk prediction scores that preserve clinical knowledge

Many in-hospital mortality risk prediction scores dichotomize predictive variables to simplify the score calculation. However, hard thresholding in these additive stepwise scores of the form "add x points if variable v is above/below threshold t" may lead to critical failures. In this paper, we seek to develop risk prediction scores that preserve clinical knowledge embedded in features and structure of the existing additive stepwise scores while addressing limitations caused by variable dichotomization. To this end, we propose a novel score structure that relies on a transformation of predictive variables by means of nonlinear logistic functions facilitating smooth differentiation between critical and normal values of the variables. We develop an optimization framework for inferring parameters of the logistic functions for a given patient population via cyclic block coordinate descent. The parameters may readily be updated as the patient population and standards of care evolve. We tested the proposed methodology on two populations: (1) brain trauma patients admitted to the intensive care unit of the Dell Children's Medical Center of Central Texas between 2007 and 2012, and (2) adult ICU patient data from the MIMIC II database. The results are compared with those obtained by the widely used PRISM III and SOFA scores. The prediction power of a score is evaluated using area under ROC curve, Youden's index, and precision-recall balance in a cross-validation study. The results demonstrate that the new framework enables significant performance improvements over PRISM III and SOFA in terms of all three criteria.

A history of outcome prediction in the ICU

PURPOSE OF REVIEW

There are few first-hand accounts that describe the history of outcome prediction in critical care. This review summarizes the authors' personal perspectives about the development and evolution of Acute Physiology and Chronic Health Evaluation over the past 35 years.

RECENT FINDINGS

We emphasize what we have learned in the past and more recently our perspectives about the current status of outcome prediction, and speculate about the future of outcome prediction.

SUMMARY

There is increasing evidence that superior accuracy in outcome prediction requires complex modeling with detailed adjustment for diagnosis and physiologic abnormalities. Thus, an automated electronic system is recommended for gathering data and generating predictions. Support, either public or private, is required to assist users and to update and improve models. Current outcome prediction models have increasingly focused on benchmarks for resource use, a trend that seems likely to increase in the future.

Why try to predict ICU outcomes?

PURPOSE OF REVIEW

To describe why the prediction of ICU outcomes is essential to underpin critical care quality improvement programmes.

RECENT FINDINGS

Recent literature demonstrates that risk-adjusted mortality is a widely used and well-accepted quality indicator for benchmarking ICU performance. Ongoing research continues to address the best ways to present the results of benchmarking through either direct comparison among institutions (e.g., by funnel plots) or indirect comparison against the risk predictions from a risk model (e.g., by process control charts). There is also ongoing research and debate regarding event-based outcomes (e.g., hospital mortality) versus time-based outcomes (e.g., 30-day mortality). Beyond benchmarking, ICU outcome prediction models have a role in risk adjustment and risk stratification in randomized controlled trials, and adjusting for confounding in nonrandomized, observational research. Recent examples include comparing risk-adjusted outcomes according to 'capacity strain' on the ICU and extending propensity matching methods to evaluate outcomes of patients managed with a pulmonary artery catheter, among others. Risk models may have a role in communicating risk, but their utility for individual patient decision-making is limited.

SUMMARY

Risk-adjusted mortality has strong support from the critical care community as a quality indicator for benchmarking ICU performance but is dependent on up-to-date, accurate risk models. ICU outcome prediction can also contribute to both randomized and nonrandomized research and potentially contribute to individual patient management, although generic risk models should not be used to guide individual treatment decisions.

Predicting one-year mortality of critically ill patients with early acute kidney injury: data from the prospective multicenter FINNAKI study

INTRODUCTION

No predictive models for long-term mortality in critically ill patients with acute kidney injury (AKI) exist. We aimed to develop and validate two predictive models for one-year mortality in patients with AKI based on data (1) on intensive care unit (ICU) admission and (2) on the third day (D3) in the ICU.

METHODS

This substudy of the FINNAKI study comprised 774 patients with early AKI (diagnosed within 24 hours of ICU admission). We selected predictors a priori based on previous studies, clinical judgment, and differences between one-year survivors and non-survivors in patients with AKI. We validated the models internally with bootstrapping.

RESULTS

Of 774 patients, 308 (39.8%, 95% confidence interval (CI) 36.3 to 43.3) died during one year. Predictors of one-year mortality on admission were: advanced age, diminished premorbid functional performance, co-morbidities, emergency admission, and resuscitation or hypotension preceding ICU admission. The area under the receiver operating characteristic curve (AUC) (95% CI) for the admission model was 0.76 (0.72 to 0.79) and the mean bootstrap-adjusted AUC 0.75 (0.74 to 0.75). Advanced age, need for mechanical ventilation on D3, number of co-morbidities, higher modified SAPS II score, the highest bilirubin value by D3, and the lowest base excess value on D3 remained predictors of one-year mortality on D3. The AUC (95% CI) for the D3 model was 0.80 (0.75 to 0.85) and by bootstrapping 0.79 (0.77 to 0.80).

CONCLUSIONS

The prognostic performance of the admission data-based model was acceptable, but not good. The D3 model for one-year mortality performed fairly well in patients with early AKI.

Integration of palliative care in the context of rapid response: a report from the Improving Palliative Care in the ICU advisory board

Rapid response teams (RRTs) can effectively foster discussions about appropriate goals of care and address other emergent palliative care needs of patients and families facing life-threatening illness on hospital wards. In this article, The Improving Palliative Care in the ICU (IPAL-ICU) Project brings together interdisciplinary expertise and existing data to address the following: special challenges for providing palliative care in the rapid response setting, knowledge and skills needed by RRTs for delivery of high-quality palliative care, and strategies for improving the integration of palliative care with rapid response critical care. We discuss key components of communication with patients, families, and primary clinicians to develop a goal-directed treatment approach during a rapid response event. We also highlight the need for RRT expertise to initiate symptom relief. Strategies including specific clinician training and system initiatives are then recommended for RRT care improvement. We conclude by suggesting that as evaluation of their impact on other outcomes continues, performance by RRTs in meeting palliative care needs of patients and families should also be measured and improved.

Are all deaths recorded equally? The impact of hospice care on risk-adjusted mortality

BACKGROUND

Hospice care provides dignity and comfort at the end of life. While patients transferred to hospice die, they are often not recorded as in-hospital deaths in trauma registries or in some administrative discharge data. Mortality rates for the purpose of database research, performance improvement, or public reporting may therefore be artificially low. The current study sought to determine the impact of discharges to hospice on risk-adjusted mortality for trauma deaths reported to the Trauma Quality Improvement Program.

METHODS

Performance from Trauma Quality Improvement Program centers in 2011 was evaluated using risk-adjusted mortality with observed-to-expected mortality ratios derived from a logistic regression model. The impact of discharge to hospice on performance was measured by determining changes in performance if hospice cases were treated as survivors rather than deaths. Differences between groups were compared by nonparametric Wilcoxon rank-sum test.

RESULTS

From the 167 centers with 126,259 injured patients, there were 8,862 deaths: 746 (8.4%) were discharged to a hospice, and the remainder was counted as in-hospital deaths. Overall, 106 centers (63.5%) reported at least one discharge to hospice, with the proportion of deaths ranging from 1.6% to 57%. Logistic regression demonstrated that age greater than 70 years (odds ratio [OR], 4.3; 95% confidence interval [CI], 3.5-5.1), male sex (OR, 0.7; 95% CI, 0.6-0.8), nonblack race (OR, 1.9; 95% CI, 1.3-2.7), noncommercial insurance (OR, 1.4; 95% CI, 1.1-1.7), and comorbidity counts greater than 2 (OR, 1.3; 95% CI, 1.1-1.6) were associated with hospice care. If patients transferred to a hospice were treated as survivors in the estimation of risk-adjusted mortality, 34 centers (20%) would have a change in status. Changes would be in both directions for average-performing centers, while high-performing centers would seem worse and poor-performing centers would seem better. For centers that reported hospice deaths, the relative risk-adjusted mortality decreased by 8.8% for every 10% increase in the proportion of deaths recorded as discharged to a hospice.

CONCLUSION

Given the large variation in the proportion of deaths recorded as discharged to a hospice rather than as in-hospital deaths, there is the potential for significant distortion of actual performance. Failure to consider this potential may misguide efforts directing performance improvement, research, and national reporting. Discharges to a hospice should be included with in-hospital deaths when reporting risk-adjusted mortality.

Predicting six-month mortality of patients with traumatic brain injury: usefulness of common intensive care severity scores

Introduction

The aim of this study was to evaluate the usefulness of the APACHE II (Acute Physiology and Chronic Health Evaluation II), SAPS II (Simplified Acute Physiology Score II) and SOFA (Sequential Organ Failure Assessment) scores compared to simpler models based on age and Glasgow Coma Scale (GCS) in predicting long-term outcome of patients with moderate-to-severe traumatic brain injury (TBI) treated in the intensive care unit (ICU).

Methods

A national ICU database was screened for eligible TBI patients (age over 15 years, GCS 3–13) admitted in 2003–2012. Logistic regression was used for customization of APACHE II, SAPS II and SOFA score-based models for six-month mortality prediction. These models were compared to an adjusted SOFA-based model (including age) and a reference model (age and GCS). Internal validation was performed by a randomized split-sample technique. Prognostic performance was determined by assessing discrimination, calibration and precision.

Results

In total, 1,625 patients were included. The overall six-month mortality was 33%. The APACHE II and SAPS II-based models showed good discrimination (area under the curve (AUC) 0.79, 95% confidence interval (CI) 0.75 to 0.82; and 0.80, 95% CI 0.77 to 0.83, respectively), calibration (P > 0.05) and precision (Brier score 0.166 to 0.167). The SOFA-based model showed poor discrimination (AUC 0.68, 95% CI 0.64 to 0.72) and precision (Brier score 0.201) but good calibration (P > 0.05). The AUC of the SOFA-based model was significantly improved after the insertion of age and GCS (∆AUC +0.11, P < 0.001). The performance of the reference model was comparable to the APACHE II and SAPS II in terms of discrimination (AUC 0.77; compared to APACHE II, ΔAUC −0.02, P = 0.425; compared to SAPS II, ΔAUC −0.03, P = 0.218), calibration (P > 0.05) and precision (Brier score 0.181).

Conclusions

A simple prognostic model, based only on age and GCS, displayed a fairly good prognostic performance in predicting six-month mortality of ICU-treated patients with TBI. The use of the more complex scoring systems APACHE II, SAPS II and SOFA added little to the prognostic performance.