[ICU performance: results of a French study involving 80,000 ICU stays]

Determinants of the calibration of SAPS II and SAPS 3 mortality scores in intensive care: a European multicenter study

Background

The aim of the Simplified Acute Physiology Score (SAPS) II and SAPS 3 is to predict the mortality of patients admitted to intensive care units (ICUs). Previous studies have suggested that the calibration of these scores may vary across countries, centers, and/or characteristics of patients. In the present study, we aimed to assess determinants of the calibration of these scores.

Methods

We assessed the calibration of the SAPS II and SAPS 3 scores among 5266 patients admitted to ICUs during a 4-week period at 120 centers in 17 European countries. We obtained calibration curves, Brier scores, and standardized mortality ratios. Points attributed to SAPS items were reevaluated and compared with those of the original scores. Finally, we tested associations between the calibration and center characteristics.

Results

The mortality was overestimated by both scores: The standardized mortality ratios were 0.75 (95% CI 0.71–0.79) for the SAPS II score and 0.91 (95% CI 0.86–0.96) for the SAPS 3 score. This overestimation was partially explained by changes in associations between some items of the scores and mortality, especially the heart rate, Glasgow Coma Scale score, and diagnosis of AIDS for SAPS II. The calibration of both scores was better in countries with low health expenditures. The between-center variability in calibration curves was much greater than expected by chance.

Conclusions

Both scores overestimate current mortality among European ICU patients. The magnitude of the miscalibration of SAPS II and SAPS 3 scores depends not only on patient characteristics but also on center characteristics. Furthermore, much between-center variability in calibration remains unexplained by these factors.

Trial registration

ClinicalTrials.gov identifier: NCT01422070. Registered 19 August 2011.

Electronic supplementary material

The online version of this article (doi:10.1186/s13054-017-1673-6) contains supplementary material, which is available to authorized users.

Prognostic performance of the Simplified Acute Physiology Score II in major Croatian hospitals: a prospective multicenter study

Aim

To perform an external validation of the original Simplified Acute Physiology Score II (SAPS II) system and to assess its performance in a selected group of patients in major Croatian hospitals.

Methods

A prospective, multicenter study was conducted in five university hospitals and one general hospital during a six-month period between November 1, 2007 and May 1, 2008. Standardized hospital mortality ratio (SMR) was calculated from the mean predicted mortality of all the 2756 patients and the actual mortality for the same group of patients. The validation of SAPS II was made using the area under receiver operating characteristic curve (AUC), 2 × 2 classification tables, and Hosmer-Lemeshow tests.

Results

The predicted mortality was as low as 14.6% due to a small proportion of medical patients and the SMR being 0.89 (95% confidence interval [CI], 0.78-0.98). The SAPS II system demonstrated a good discriminatory power as measured by the AUC (0.85; standard error [SE] = 0.012; 95% CI = 0.840-0.866; P < 0.001). This system significantly overestimated the actual mortality (Hosmer-Lemeshow goodness-of-fit H statistic: χ² = 584.4; P < 0.001 and C statistics: χ²₈ = 313.0; P < 0.001) in the group of patients included in the study.

Conclusion

The SAPS II had a good discrimination, but it significantly overestimated the observed mortality in comparison with the predicted mortality in this group of patients in Croatia. Therefore, caution is required when an evaluation is performed at the individual level.

Can outcome prediction data change patient outcomes and organizational outcomes?

PURPOSE OF REVIEW

Intensive care medicine consumes a high share of healthcare costs, and there is growing pressure to use the scarce resources efficiently. Accordingly, organizational issues and quality management have become an important focus of interest in recent years. Here, we will review current concepts of how outcome data can be used to identify areas requiring action.

RECENT FINDINGS

Using recently established models of outcome assessment, wide variability between individual ICUs is found, both with respect to outcome and resource use. Such variability implies that there are large differences in patient care processes not only within the ICU but also in pre-ICU and post-ICU care. Indeed, measures to improve the patient process in the ICU (including care of the critically ill, patient safety, and management of the ICU) have been presented in a number of recently published papers.

SUMMARY

Outcome assessment models provide an important framework for benchmarking. They may help the individual ICU to spot appropriate fields of action, plan and initiate quality improvement projects, and monitor the consequences of such activity.

Outcome prediction in critical care: the Simplified Acute Physiology Score models

PURPOSE OF REVIEW

Outcome prediction models measuring severity of illness of patients admitted to the intensive care unit should predict hospital mortality. This review describes the state-of-the-art of Simplified Acute Physiology Score models from the clinical and managerial perspectives. Methodological issues concerning the effects of differences between new samples and original databases in which the models were developed are considered.

RECENT FINDINGS

The progressive lack of fit of the Simplified Acute Physiology Score II in independent intensive care unit populations induced investigators to propose customizations and expansions as potential evolutions for Simplified Acute Physiology Score II. We do not know whether those solutions did solve the issue because there are no demonstrations of consistent good fit in new databases. The recently developed Simplified Acute Physiology Score 3 Admission Score with customization for geographical areas is discussed. The points shared by the Simplified Acute Physiology Score models and the pros and cons for each of them are introduced.

SUMMARY

Comparisons of intensive care unit performance should take into account not only the patient severity of illness, but also the effect of the 'intensive care unit variable', that is, differences in human resources, structure, equipment, management and organization of the intensive care unit. In the future, moving from patient and geographical area adjustment to resource use could allow the user to adjust for differences in healthcare provision.