An analysis of excess mortality not predicted to occur by APACHE III in an Australian level III intensive care unit

Predictive performance of Acute Physiological and Chronic Health Evaluation releases II to IV: a single New Zealand centre experience

There is debate in Australia and New Zealand around the appropriate use of illness severity scoring systems in Australasian intensive care units. The international benchmark is the Acute Physiological and Chronic Health Evaluation (APACHE) system. In order to compare the performance of recent APACHE releases, we audited 2080 sequential patients admitted between 1 January 2006 and 31 March 2008 to the Middlemore Hospital intensive care unit, Auckland, New Zealand. We compared the predictive performance of the proprietary APACHE II, IIIh, IIIj and IV releases, and the performance of a 'localised' version of APACHE II containing re-estimated coefficients derived from a legacy dataset (7703 sequential patients admitted between 1 January 1997 and 31 December 2005). Discrimination assessed by receiver operating characteristic curves was highest with the APACHE III and IV releases, and significantly better than the APACHE II releases. Calibration assessed by the Hosmer-Lemeshow statistic was poor with all releases, although it was best with APACHE IV and 'localised' version of the APACHE II release. Overall accuracy assessed by the Brier Mean Probability score and Shapiro's R statistic was best with APACHE IV. Our study suggests the possibility of improved prediction in moving to APACHE IV from older releases, although broader multicentre study within the Australian and New Zealand critical care community is warranted. Our study also suggests localisation of the APACHE system offers further opportunity to improve prediction, although these improvements may not be major without ground-up development of a new risk prediction model within our local critical care setting.

A comparison of admission and worst 24-hour Acute Physiology and Chronic Health Evaluation II scores in predicting hospital mortality: a retrospective cohort study

Introduction

The Acute Physiology and Chronic Health Evaluation (APACHE) II score is widely used in the intensive care unit (ICU) as a scoring system for research and clinical audit purposes. Physiological data for calculation of the APACHE II score are derived from the worst values in the first 24 hours after admission to the ICU. The collection of physiological data on admission only is probably logistically easier, and this approach is used by some ICUs. This study compares the performance of APACHE II scores calculated using admission data with those obtained from the worst values in the first 24 hours.

Materials and Methods

This was a retrospective cohort study using prospectively collected data from a tertiary ICU. There were no missing physiological data and follow-up for mortality was available for all patients in the database. The admission and the worst 24-hour physiological variables were used to generate the admission APACHE II score and the worst 24-hour APACHE II score, and the corresponding predicted mortality, respectively.

Results

There were 11,107 noncardiac surgery ICU admissions during 11 years from 1 January 1993 to 31 December 2003. The mean admission and the worst 24-hour APACHE II score were 12.7 and 15.4, and the derived predicted mortality estimates were 15.5% and 19.3%, respectively. The actual hospital mortality was 16.3%. The overall discrimination ability, as measured by the area under the receiver operating characteristic curve, of the admission APACHE II model (83.8%, 95% confidence interval = 82.9–84.7) and the worst 24-hour APACHE II model (84.6%, 95% confidence interval = 83.7–85.5) was not significantly different (P = 1.00).

Conclusion

Substitution of the worst 24-hour physiological variables with the admission physiological variables to calculate the admission APACHE II score maintains the overall discrimination ability of the traditional APACHE II model. The admission APACHE II model represents a potential alternative model to the worst 24-hour APACHE II model in critically ill nontrauma patients.

Use of the Sequential Organ Failure Assessment score as a severity score

OBJECTIVE

To evaluate whether the SOFA score can be used to develop a model to predict intensive care unit (ICU) mortality in different countries.

DESIGN AND SETTING

Analysis of a prospectively collected database. Patients with ICU stay longer than 2 days were studied to develop a mortality prediction model based on measurements of organ dysfunction.

PATIENTS

748 patients from six countries.

MEASUREMENTS AND RESULTS

Two logistic regression models were constructed, one based on the SOFA maximum (SOFA Max model) and the other on variables identified by multivariate regression (SOFA Max-infection model). The H and C statistics had a p value above 0.05 for both models, but the D statistics showed a poor performance on the SOFA Max model when stratified for the presence of infection. Subsequent analysis was performed with SOFA Max-infection model. The area under the curve was 0.853. There were no statistically significant differences in observed and predicted mortalities except for one country which had a higher than predicted ICU mortality both in the overall population (28.3 vs. 19.1%) and in the noninfected patients (21.4 vs. 12.6%).

CONCLUSIONS

The SOFA Max adjusted for age and the presence of infection can predict mortality in this population, but in one country the ICU mortality was higher than expected. Our data do not allow us to determine the reasons behind these differences, and further studies to detect differences in mortality between countries and to elucidate the basis for these differences should be encouraged.

Prospective independent validation of APACHE III models in an Australian tertiary adult intensive care unit

Evaluation of the performance of the APACHE III (Acute Physiology and Chronic Health Evaluation) ICU (intensive care unit) and hospital mortality models at the Princess Alexandra Hospital, Brisbane is reported. Prospective collection of demographic, diagnostic, physiological, laboratory, admission and discharge data of 5681 consecutive eligible admissions (1 January 1995 to 1 January 2000) was conducted at the Princess Alexandra Hospital, a metropolitan Australian tertiary referral medical/surgical adult ICU ROC (receiver operating characteristic) curve areas for the APACHE III ICU mortality and hospital mortality models demonstrated excellent discrimination. Observed ICU mortality (9.1%) was significantly overestimated by the APACHE III model adjusted for hospital characteristics (10.1%), but did not significantly differ from the prediction of the generic APACHE III model (8.6%). In contrast, observed hospital mortality (14.8%) agreed well with the prediction of the APACHE III model adjusted for hospital characteristics (14.6%), but was significantly underestimated by the unadjusted APACHE III model (13.2%). Calibration curves and goodness-of-fit analysis using Hosmer-Lemeshow statistics, demonstrated that calibration was good with the unadjusted APACHE III ICU mortality model, and the APACHE III hospital mortality model adjusted for hospital characteristics. Post hoc analysis revealed a declining annual SMR (standardized mortality rate) during the study period. This trend was present in each of the non-surgical, emergency and elective surgical diagnostic groups, and the change was temporally related to increased specialist staffing levels. This study demonstrates that the APACHE III model performs well on independent assessment in an Australian hospital. Changes observed in annual SMR using such a validated model support an hypothesis of improved survival outcomes 1995-1999.

Day of the week of intensive care admission and patient outcomes: a multisite regional evaluation

BACKGROUND

Relationships between day of the week of admission to hospitals and hospital outcomes have been poorly studied. Intensive care units (ICUs) appear to be uniquely suited to examine such a question given the unpredictability of ICU admissions and the clinical instability of their patient populations.

METHODS

This retrospective cohort study included 156,136 patients admitted to 38 ICUs in 28 hospitals in a large Midwestern metropolitan area during 1991 to 1997. Demographic and clinical data were collected from patients' medical records and used in multivariable risk-adjustment models that examined the risk for in-hospital death and ICU length of stay.

RESULTS

The adjusted odds of in-hospital death were 9% higher (OR 1.09; 95% CI, 1.04-1.15; P <0.001) for weekend admissions (Saturday or Sunday) than in patients admitted midweek (Tuesday through Thursday). However, the adjusted odds of death were also higher (P <0.001) for patients admitted on Monday (OR 1.09) or Friday (OR 1.08). Findings were generally similar in analyses stratified by admission type (medical vs. surgical), hospital teaching status, and illness severity. Adjusted ICU length of stay was 4% longer (P <0.001) for weekend or Friday admissions, compared with midweek admissions.

CONCLUSIONS

Patients admitted to an ICU on the weekend have a modestly higher risk for death and ICU length of stay. However, the similar risk for death in patients admitted on Friday and Monday suggests that "weekend effects" may be more related to unmeasured severity of illness and/or selection bias than to differences in quality of care.