Identifying quality outliers in a large, multiple-institution database by using customized versions of the Simplified Acute Physiology Score II and the Mortality Probability Model II0

Development and validation of a nomogram for predicting in-hospital mortality in ICU patients with infective endocarditis

BACKGROUND

Infective endocarditis (IE) is a disease with high in-hospital mortality. The objective of the present investigation was to develop and validate a nomogram that precisely anticipates in-hospital mortality in ICU individuals diagnosed with infective endocarditis.

METHODS

Retrospectively collected clinical data of patients with IE admitted to the ICU in the MIMIC IV database were analyzed using the Least Absolute Shrinkage and Selection Operator (LASSO) regression to identify potential hazards. A logistic regression model incorporating multiple factors was established, and a dynamic nomogram was generated to facilitate predictions. To assess the classification performance of the model, an ROC curve was generated, and the AUC value was computed as an indicator of its diagnostic accuracy. The model was subjected to calibration curve analysis and the Hosmer-Lemeshow (HL) test to assess its goodness of fit. To evaluate the clinical relevance of the model, decision-curve analysis (DCA) was conducted.

RESULTS

The research involved a total of 676 patients, who were divided into two cohorts: a training cohort comprising 473 patients and a validation cohort comprising 203 patients. The allocation ratio between the two cohorts was 7:3. Based on the independent predictors identified through LASSO regression, the final selection for constructing the prediction model included five variables: lactate, bicarbonate, white blood cell count (WBC), platelet count, and prothrombin time (PT). The nomogram model demonstrated a robust diagnostic ability in both the cohorts used for training and validation. This is supported by the respective area under the curve (AUC) values of 0.843 and 0.891. The results of the calibration curves and HL tests exhibited acceptable conformity between observed and predicted outcomes. According to the DCA analysis, the nomogram model demonstrated a notable overall clinical advantage compared to the APSIII and SAPSII scoring systems.

CONCLUSIONS

The nomogram developed during the study proved to be highly accurate in forecasting the mortality of patients with IE during hospitalization in the ICU. As a result, it may be useful for clinicians in decision-making and treatment.

Discrete-time survival analysis in the critically ill: a deep learning approach using heterogeneous data

Prediction of survival for patients in intensive care units (ICUs) has been subject to intense research. However, no models exist that embrace the multiverse of data in ICUs. It is an open question whether deep learning methods using automated data integration with minimal pre-processing of mixed data domains such as free text, medical history and high-frequency data can provide discrete-time survival estimates for individual ICU patients. We trained a deep learning model on data from patients admitted to ten ICUs in the Capital Region of Denmark and the Region of Southern Denmark between 2011 and 2018. Inspired by natural language processing we mapped the electronic patient record data to an embedded representation and fed the data to a recurrent neural network with a multi-label output layer representing the chance of survival at different follow-up times. We evaluated the performance using the time-dependent concordance index. In addition, we quantified and visualized the drivers of survival predictions using the SHAP methodology. We included 37,355 admissions of 29,417 patients in our study. Our deep learning models outperformed traditional Cox proportional-hazard models with concordance index in the ranges 0.72–0.73, 0.71–0.72, 0.71, and 0.69–0.70, for models applied at baseline 0, 24, 48, and 72 h, respectively. Deep learning models based on a combination of entity embeddings and survival modelling is a feasible approach to obtain individualized survival estimates in data-rich settings such as the ICU. The interpretable nature of the models enables us to understand the impact of the different data domains.

Dynamic and explainable machine learning prediction of mortality in patients in the intensive care unit: a retrospective study of high-frequency data in electronic patient records

BACKGROUND

Many mortality prediction models have been developed for patients in intensive care units (ICUs); most are based on data available at ICU admission. We investigated whether machine learning methods using analyses of time-series data improved mortality prognostication for patients in the ICU by providing real-time predictions of 90-day mortality. In addition, we examined to what extent such a dynamic model could be made interpretable by quantifying and visualising the features that drive the predictions at different timepoints.

METHODS

Based on the Simplified Acute Physiology Score (SAPS) III variables, we trained a machine learning model on longitudinal data from patients admitted to four ICUs in the Capital Region, Denmark, between 2011 and 2016. We included all patients older than 16 years of age, with an ICU stay lasting more than 1 h, and who had a Danish civil registration number to enable 90-day follow-up. We leveraged static data and physiological time-series data from electronic health records and the Danish National Patient Registry. A recurrent neural network was trained with a temporal resolution of 1 h. The model was internally validated using the holdout method with 20% of the training dataset and externally validated using previously unseen data from a fifth hospital in Denmark. Its performance was assessed with the Matthews correlation coefficient (MCC) and area under the receiver operating characteristic curve (AUROC) as metrics, using bootstrapping with 1000 samples with replacement to construct 95% CIs. A Shapley additive explanations algorithm was applied to the prediction model to obtain explanations of the features that drive patient-specific predictions, and the contributions of each of the 44 features in the model were analysed and compared with the variables in the original SAPS III model.

FINDINGS

From a dataset containing 15 615 ICU admissions of 12 616 patients, we included 14 190 admissions of 11 492 patients in our analysis. Overall, 90-day mortality was 33·1% (3802 patients). The deep learning model showed a predictive performance on the holdout testing dataset that improved over the timecourse of an ICU stay: MCC 0·29 (95% CI 0·25-0·33) and AUROC 0·73 (0·71-0·74) at admission, 0·43 (0·40-0·47) and 0·82 (0·80-0·84) after 24 h, 0·50 (0·46-0·53) and 0·85 (0·84-0·87) after 72 h, and 0·57 (0·54-0·60) and 0·88 (0·87-0·89) at the time of discharge. The model exhibited good calibration properties. These results were validated in an external validation cohort of 5827 patients with 6748 admissions: MCC 0·29 (95% CI 0·27-0·32) and AUROC 0·75 (0·73-0·76) at admission, 0·41 (0·39-0·44) and 0·80 (0·79-0·81) after 24 h, 0·46 (0·43-0·48) and 0·82 (0·81-0·83) after 72 h, and 0·47 (0·44-0·49) and 0·83 (0·82-0·84) at the time of discharge.

INTERPRETATION

The prediction of 90-day mortality improved with 1-h sampling intervals during the ICU stay. The dynamic risk prediction can also be explained for an individual patient, visualising the features contributing to the prediction at any point in time. This explanation allows the clinician to determine whether there are elements in the current patient state and care that are potentially actionable, thus making the model suitable for further validation as a clinical tool.

FUNDING

Novo Nordisk Foundation and the Innovation Fund Denmark.

Risk-Adjusted Overall Mortality as a Quality Measure in the Cardiovascular Intensive Care Unit

Risk-adjusted mortality has been proposed as a quality of care indicator to gauge cardiovascular intensive care Unit (CICU) performance. Mortality is easily measured, readily understandable, and a meaningful outcome for the patient, provider, administrative agencies, and other key stakeholders. Disease-specific risk-adjusted mortality is commonly used in cardiovascular medicine as an indicator of care quality, for external accreditation, and to determine payer reimbursement. However, the evidence base for overall risk-adjusted mortality in the CICU is limited, with most available data coming from the general critical care literature. In addition, existing risk-adjusted mortality models vary considerably in terms of approach and composition, and there is no nationally recognized standard. Thus, the objective of this study was to review the use of risk-adjusted mortality as a measure of overall unit performance and quality of care in the CICU. We found a considerable variability in the risk-adjustment methodology for cardiovascular disease. Although predictive models for disease-specific risk-adjusted mortality in cardiovascular disease have been developed, there are limited published data on overall risk-adjusted mortality for the CICU. Without standardization of risk-adjustment methodology, researchers are often required to use existing risk-adjustment models developed in noncardiac patient populations. Further studies are needed to establish whether risk-adjusted overall CICU mortality is a valid performance measure and whether it reflects care quality.

Mortality Measures to Profile Hospital Performance for Patients With Septic Shock

OBJECTIVES

Sepsis care is becoming a more common target for hospital performance measurement, but few studies have evaluated the acceptability of sepsis or septic shock mortality as a potential performance measure. In the absence of a gold standard to identify septic shock in claims data, we assessed agreement and stability of hospital mortality performance under different case definitions.

DESIGN

Retrospective cohort study.

SETTING

U.S. acute care hospitals.

PATIENTS

Hospitalized with septic shock at admission, identified by either implicit diagnosis criteria (charges for antibiotics, cultures, and vasopressors) or by explicit International Classification of Diseases, 9th revision, codes.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We used hierarchical logistic regression models to determine hospital risk-standardized mortality rates and hospital performance outliers. We assessed agreement in hospital mortality rankings when septic shock cases were identified by either explicit International Classification of Diseases, 9th revision, codes or implicit diagnosis criteria. Kappa statistics and intraclass correlation coefficients were used to assess agreement in hospital risk-standardized mortality and hospital outlier status, respectively. Fifty-six thousand six-hundred seventy-three patients in 308 hospitals fulfilled at least one case definition for septic shock, whereas 19,136 (33.8%) met both the explicit International Classification of Diseases, 9th revision, and implicit septic shock definition. Hospitals varied widely in risk-standardized septic shock mortality (interquartile range of implicit diagnosis mortality: 25.4-33.5%; International Classification of Diseases, 9th revision, diagnosis: 30.2-38.0%). The median absolute difference in hospital ranking between septic shock cohorts defined by International Classification of Diseases, 9th revision, versus implicit criteria was 37 places (interquartile range, 16-70), with an intraclass correlation coefficient of 0.72, p value of less than 0.001; agreement between case definitions for identification of outlier hospitals was moderate (kappa, 0.44 [95% CI, 0.30-0.58]).

CONCLUSIONS

Risk-standardized septic shock mortality rates varied considerably between hospitals, suggesting that septic shock is an important performance target. However, efforts to profile hospital performance were sensitive to septic shock case definitions, suggesting that septic shock mortality is not currently ready for widespread use as a hospital quality measure.

Impact of outlier status on critical care patient outcomes: Does boarding medical intensive care unit patients make a difference?

PURPOSE

To evaluate the impact of outlier status, or the practice of boarding ICU patients in distant critical care units, on clinical and utilization outcomes.

MATERIALS AND METHODS

Retrospective observational study of all consecutive admissions to the MICU service between April 1, 2014-January 3, 2016, at an urban university hospital.

RESULTS

Of 1931 patients, 117 were outliers (6.1%) for the entire duration of their ICU stay. In adjusted analyses, there was no association between outlier status and hospital (OR 1.21, 95% CI 0.72-2.05, p=0.47) or ICU mortality (OR 1.20, 95% CI 0.64-2.25, p=0.57). Outliers had shorter hospital and ICU lengths of stay (LOS) in addition to fewer ventilator days. Crossover patients who had variable outlier exposure also had no increase in hospital (OR 1.61; 95% CI 0.80-3.23; p=0.18) or ICU mortality (OR 1.05; 95% CI 0.43-2.54; p=0.92) after risk-adjustment.

CONCLUSIONS

Boarding of MICU patients in distant units during times of bed nonavailability does not negatively influence patient mortality or LOS. Increased hospital and ventilator utilization observed among non-outliers in the home unit may be attributable, at least in part, to differences in patient characteristics. Prospective investigation into the practice of ICU boarding will provide further confirmation of its safety.

Quality of Quality Measurement

Background

The validity of basing healthcare reimbursement policy on pay-for-performance is grounded in the accuracy of performance measurement.

Methods

Monte Carlo simulation was used to examine the accuracy of performance profiling as a function of statistical methodology, case volume, and the extent to which hospital or physician performance deviates from the average.

Results

There is extensive variation in the true-positive rate and false discovery rate as a function of model specification, hospital quality, and hospital case volume. Hierarchical and nonhierarchical modeling are both highly accurate at very high case volumes for very low-quality hospitals. At equivalent case volumes and hospital effect sizes, the true-positive rate is higher for nonhierarchical modeling than for hierarchical modeling, but the false discovery rate is generally much lower for hierarchical modeling than for nonhierarchical modeling. At low hospital case volumes (200) that are typical for many procedures, and for hospitals with twice the rate of death or major complications for patients undergoing isolated coronary artery bypass graft surgery at the average hospital, hierarchical modeling missed 90.6% of low-quality hospitals, whereas nonhierarchical modeling missed 65.3%. However, at low case volumes, 38.9% of hospitals classified as low-quality outliers using nonhierarchical modeling were actually average quality, compared to 5.3% using hierarchical modeling.

Conclusions

Nonhierarchical modeling frequently misclassified average-quality hospitals as low quality. Hierarchical modeling commonly misclassified low-quality hospitals as average. Assuming that the consequences of misclassifying an average-quality hospital as low quality outweigh the consequences of misclassifying a low-quality hospital as average, hierarchical modeling may be the better choice for quality measurement.

Sharing intimacy in "open" intensive care units

PURPOSE

Opening intensive care units (ICUs) is particularly relevant because of a new Swiss law granting the relatives of patients without decision-making capability a central role in medical decisions. The main objectives of the study were to assess how the presence of relatives is viewed by patients, health care providers, and relatives themselves and to evaluate the perception of the level of intrusiveness into the personal sphere during admission.

MATERIAL AND METHODS

In a longitudinal and prospective design, qualitative questionnaires were submitted concomitantly to patients, relatives, and health care providers consecutively over a 6-month period. The study was conducted in the 4 ICUs of the public hospitals of Canton Ticino (Switzerland).

RESULTS

The questionnaires collected from patients, relatives, and health care providers were 176, 173, and 134, respectively. The analysis of the answers of 120 patient-relative pairs showed consistent results (P < .0001), whereas those of health care providers were significantly different (P < .0001), regarding both the usefulness of opening ICUs to patient relatives and what was stressful during admission.

CONCLUSIONS

Relatives in these "open" ICUs share a great deal of intimacy with the patients. Their presence and the deriving benefits were seen as very positive by patients and relatives themselves. Skepticism, instead, prevailed among health care providers.

Evaluating the performance of Australian and New Zealand intensive care units in 2009 and 2010

The Australian and New Zealand Intensive Care Society Adult Patient Database (ANZICS APD) is one of the largest databases of its kind in the world and collects individual admissions' data from intensive care units (ICUs) around Australia and New Zealand. Use of this database for monitoring and comparing the performance of ICUs, quantified by the standardised mortality ratio, poses several theoretical and computational challenges, which are addressed in this paper. In particular, the expected number of deaths must be appropriately estimated, the ICU casemix adjustment must be adequate, statistical variation must be fully accounted for, and appropriate adjustment for multiple comparisons must be made. Typically, one or more of these issues have been neglected in ICU comparison studies. Our approach to the analysis proceeds by fitting a random coefficient hierarchical logistic regression model for the inhospital death of each patient, with patients clustered within ICUs. We anticipate the majority of ICUs will be estimated as performing 'usually' after adjusting for important clinical covariates. We take as a starting point the ideas in Ohlssen et al and estimate an appropriate null model that we expect these ICUs to follow, taking a frequentist rather than a Bayesian approach. This methodology allows us to rigorously account for the aforementioned statistical issues and to determine if there are any ICUs contributing to the Australian and New Zealand Intensive Care Society database that have comparatively unusual performance. In addition to investigating the yearly performance of the ICUs, we also estimate changes in individual ICU performance between 2009 and 2010 by adjusting for regression-to-the-mean.

Is the SAPS II score valid in surgical intensive care unit patients?

AIMS AND OBJECTIVES

We investigated the performance of the simplified acute physiology score II (SAPS II) in a large cohort of surgical intensive care unit (ICU) patients and tested the hypothesis that customization of the score would improve the uniformity of fit in subgroups of surgical ICU patients.

METHODS

Retrospective analysis of prospectively collected data from all 12,938 patients admitted to a postoperative ICU between January 2004 and January 2009. Probabilities of hospital death were calculated for original and customized (C1-SAPS II and C2-SAPS II) scores. A priori subgroups were defined according to age, probability of death according to the SAPS II score, ICU length of stay (LOS), surgical procedures and type of admission.

RESULTS

The median ICU LOS was 1 (1-3) day. ICU and hospital mortality rates were 5.8% and 10.3%, respectively. Discrimination of the SAPS II was moderate [area under receiver operating characteristic curve (aROC) = 0.76 (0.75-0.78)], but calibration was poor. This model markedly overestimated hospital mortality rates [standardized mortality rate: 0.35 (0.33-0.37)]. First-level customization (C1-SAPS II) did not improve discrimination in the whole cohort or the subgroups, but calibration improved in some subgroups. Second-level customization (C2-SAPS II) improved discrimination in the whole cohort [aROC = 0.82 (0.79-0.85)] and most of the subgroups (aROC range 0.65-86). Calibration in this model (C2-SAPS II) improved in the whole cohort and in subgroups except in patients with ICU LOS 4-14 days and those undergoing neuro- or gastrointestinal surgery.

CONCLUSIONS

In this large cohort of surgical ICU patients, performance of the original SAPS II model was generally poor. Although second-level customization improved discrimination and calibration in the whole cohort and most of the subgroups, it failed to simultaneously improve calibration in the subgroups stratified according to the type of surgery, age or ICU LOS.

Development and validation of a model that uses enhanced administrative data to predict mortality in patients with sepsis

OBJECTIVE

We aimed to determine whether a sepsis risk-adjustment model that uses only administrative data could be used when other intensive care unit risk-adjustment methods are unavailable.

DESIGN

Cohort study with development and validation cohorts.

PATIENTS

The development cohort included 166,931 patients at 309 hospitals that cared for at least 100 patients with sepsis between 2004 and 2006. The validation cohort included 357 adult sepsis patients who were enrolled in Project IMPACT, 2002-2009.

MEASUREMENTS AND MAIN RESULTS

We developed a multilevel mixed-effects logistic regression model to predict mortality at the patient level. Predictors included patient demographics (age, sex, race, insurance type), site and source of sepsis, presence of 25 individual comorbidities, treatment (within the first 2 days of hospitalization) with mechanical ventilation and/or vasopressors, and/or admission to the intensive care unit (within 2 days of hospitalization). We validated this model in 357 sepsis patients who were admitted to the intensive care unit at a single academic medical center and who had a valid Acute Physiology and Chronic Health Evaluation II score, a valid Simplified Acute Physiology Score II, and a valid Mortality Probability Model III score. Overall, 33,192 patients (19.9%) died in the hospital. In the development cohort, the predicted mortality ranged from 0.002 to 0.938 with a mean of 0.199. The model's area under the receiver operating characteristic curve was 0.78. In the validation cohort, all models had modest discriminatory ability and the areas under the receiver operating characteristic curves of all models were statistically similar (Acute Physiology and Chronic Health Evaluation II, 0.71; Simplified Acute Physiology Score II, 0.74; Mortality Probability Model III, 0.69; administrative model, 0.69; p value that the areas under the receiver operating characteristic curves are different, .35). The Hosmer-Lemeshow statistic was significant (p < .01) for Acute Physiology and Chronic Health Evaluation II, Simplified Acute Physiology Score II, and Mortality Probability Model III but was nonsignificant (p = .11) for the administrative model.

CONCLUSIONS

A sepsis mortality model using detailed administrative data has discrimination similar to and calibration superior to those of existing severity scores that require chart review. This model may be a useful alternative method of severity adjustment for benchmarking purposes or for conducting large, retrospective epidemiologic studies of sepsis patients.

[Verification of validity of MPM II for neurological patients in intensive care units]

PURPOSE

Mortality Probability Model (MPM) II is a model for predicting mortality probability of patients admitted to ICU. This study was done to test the validity of MPM II for critically ill neurological patients and to determine applicability of MPM II in predicting mortality of neurological ICU patients.

METHODS

Data were collected from medical records of 187 neurological patients over 18 yr of age who were admitted to the ICU of C University Hospital during the period from January 2008 to May 2009. Collected data were analyzed through χ(2) test, t-test, Mann-Whiteny test, goodness of fit test, and ROC curve.

RESULTS

As to mortality according to patients' general and clinically related characteristics, mortality was statistically significantly different for ICU stay, hospital stay, APACHE III score, APACHE predicted death rate, GCS, endotracheal intubation, and central venous catheter. Results of Hosmer-Lemeshow goodness-of-fit test were MPM II(0) (χ(2)=0.02, p=.989), MPM II(24) (χ(2)=0.99 p=.805), MPM II(48) (χ(2)=0.91, p=.822), and MPM II(72) (χ(2)=1.57, p=.457), and results of the discrimination test using the ROC curve were MPM II(0), .726 (p<.001), MPM II(24), .764 (p<.001), MPM II(48), .762 (p<.001), and MPM II(72), .809 (p<.001).

CONCLUSION

MPM II was found to be a valid mortality prediction model for neurological ICU patients.

Severity of illness scoring systems in the intensive care unit

OBJECTIVE

Adult intensive care unit prognostic models have been used for predicting patient outcome for three decades. The goal of this review is to describe the different versions of the main adult intensive care unit prognostic models and discuss their potential roles.

DATA SOURCE

PubMed search and review of the relevant medical literature.

SUMMARY

The main prognostic models for assessing the overall severity of illness in critically ill adults are Acute Physiology and Chronic Health Evaluation, Simplified Acute Physiology Score, and Mortality Probability Model. Simplified Acute Physiology Score and Mortality Probability Model have been updated to their third versions and Acute Physiology and Chronic Health Evaluation to its fourth version. The development of prognostic models is usually followed by internal and external validation and performance assessment. Performance is assessed by area under the receiver operating characteristic curve for discrimination and Hosmer-Lemeshow statistic for calibration. The areas under the receiver operating characteristic curve of Simplified Acute Physiology Score 3, Acute Physiology and Chronic Health Evaluation IV, and Mortality Probability Model0 III were 0.85, 0.88, and 0.82, respectively, and all these three fourth-generation models had good calibration. The models have been extensively used for case-mix adjustment in clinical research and epidemiology, but their role in benchmarking, performance improvement, resource use, and clinical decision support has been less well studied.

CONCLUSIONS

The fourth-generation Acute Physiology and Chronic Health Evaluation, Simplified Acute Physiology Score 3, Acute Physiology and Chronic Health Evaluation IV, and Mortality Probability Model0 III adult prognostic models, perform well in predicting mortality. Future studies are needed to determine their roles for benchmarking, performance improvement, resource use, and clinical decision support.

The performance of customised APACHE II and SAPS II in predicting mortality of mixed critically ill patients in a Thai medical intensive care unit

The aim of this study was to evaluate and compare the performance of customised Acute Physiology and Chronic Health Evaluation HII (APACHE II) and Simplified Acute Physiology Score HII (SAPS II) in predicting hospital mortality of mixed critically ill Thai patients in a medical intensive care unit. A prospective cohort study was conducted over a four-year period. The subjects were randomly divided into calibration and validation groups. Logistic regression analysis was used for customisation. The performance of the scores was evaluated by the discrimination, calibration and overall fit in the overall group and across subgroups in the validation group. Two thousand and forty consecutive intensive care unit admissions during the study period were split into two groups. Both customised models showed excellent discrimination. The area under the receiver operating characteristic curve of the customised APACHE II was greater than the customised SAPS II (0.925 and 0.892, P < 0.001). Hosmer-Lemeshow goodness-of-fit showed good calibration for the customised APACHE II in overall populations and various subgroups but insufficient calibration for the customised SAPS II. The customised SAPS II showed good calibration in only the younger, postoperative and sepsis patients subgroups. The overall performance of the customised APACHE II was better than the customised SAPS II (Brier score 0.089 and 0.109, respectively). Our results indicate that the customised APACHE II shows better performance than the customised SAPS II in predicting hospital mortality and could be used to predict mortality and quality assessment in our unit or other intensive care units with a similar case mix.

Relationship between discharge practices and intensive care unit in-hospital mortality performance: evidence of a discharge bias

CONTEXT

Current intensive care unit performance measures include in-hospital mortality after intensive care unit admission. This measure does not account for deaths occurring after transfer to another hospital or soon after discharge and therefore, may be biased.

OBJECTIVE

Determine how transfer rates to other acute care hospitals and early post-discharge mortality rates impact hospital performance assessments using an in-hospital mortality model.

DESIGN, SETTING, AND PARTICIPANTS

Data were retrospectively collected on 10,502 eligible intensive care unit patients across 35 California hospitals between 2001 and 2004.

MEASURES

We calculated the rates of acute care hospital transfers and early post-discharge mortality (30-day overall mortality-30-day in-hospital mortality) for each hospital. We assessed hospital performance with standardized mortality ratios (SMRs) using the Mortality Probability Model III. Using regression models, we explored the relationship between in-hospital SMRs and the rates of hospital transfers or early post-discharge mortality. We explored the same relationship using a 30-day SMR.

RESULTS

In multivariable models, for each 1% increase in patients transferred to another acute care hospital, there was an in-hospital SMR reduction of -0.021 (-0.040-0.001). Additionally, a 1% increase in early post-discharge mortality was associated with an in-hospital SMR reduction of -0.049 (-0.142-0.045). Assessing hospital performance based upon 30-day mortality end point resulted in SMRs closer to 1.0 for hospitals at high and low ends of in-hospital mortality performance.

CONCLUSIONS

Variations in transfer rates and potentially discharge timing appear to bias in-hospital SMR calculations. A 30-day mortality model is a potential alternative that may limit this bias.

Pharmacoepidemiology of QT-interval prolonging drug administration in critically ill patients

PURPOSE

Commonly prescribed medications produce QT-prolongation and are associated with torsades de pointes in non-acutely ill patients. We examined patterns of QT-prolonging drug use in critically ill individuals.

METHODS

An administrative critical care database was utilized to identify patients receiving drugs associated with QT-interval prolongation or torsades de pointes for > or = 24 hours.

RESULTS

Data from 212 016 individuals collected over a 63-month period was examined to identify 6125 patients (2.9%) receiving QT-interval prolonging drugs. These individuals had a mean (+/-SE) age of 63.0 (+/-0.2) years, were predominately male (55.4%) and Caucasian (84.4%), and were exposed to QT-interval prolonging agents for a mean (+/-SE) 53.1 (+/-0.4)% of their ICU length of stay. Respiratory and cardiovascular illnesses were the most common reasons for ICU admission (17.2, 12.0%, respectively). The most frequently administered agents were amiodarone (23.5%), haloperidol (19.8%), and levofloxacin (19.7%); no other single agent accounted for more than 10% of QT-interval prolonging drugs prescribed. Coadministration of QT-prolonging drugs occurred in 1139 patients (18.6%). These patients had higher ICU mortality rate and longer ICU lengths of stay, compared to patients not receiving coadministered drugs (p < 0.001 for both). For patients receiving coadministered drugs, overlap occurred for 71.4 (+/-0.8)% of the time that the drugs were given. Amiodarone coadministration with antibiotics, haloperidol coadministration with antibiotics, and haloperidol coadministration with amiodarone, comprised 15.2, 13.7, and 9.4%, of all coadministered agents, respectively.

CONCLUSIONS

QT-prolonging drugs were used in a minority of critically ill patients. Prospective evaluation in the ICU environment is necessary to determine whether administration of these agents is associated with adverse cardiac events comparable to those reported in ambulatory patients.

Outcome prediction in critical care: the Mortality Probability Models

PURPOSE OF REVIEW

The comparison of morbidity, mortality, and length-of-stay outcomes in patients receiving critical care requires adjustment based on their presenting illness. These adjustments are made with severity-of-illness models. These models must be periodically updated to reflect current medical practices. This article will review the history of the Mortality Probability Model (MPM), discuss why and how it was recently updated, and outline examples of MPM use.

RECENT FINDINGS

All severity-of-illness models have limitations, especially if a unit's patient population becomes highly specialized. In these situations, customized models may provide better accuracy. The MPMs include those calculated at admission (MPM0) and additional models at 24, 48, and 72 h (MPM 24, MPM 48, and MPM 72). The model is now in its third iteration (MPM 0-III). Length of stay (LOS) and subgroup models have also been developed.

SUMMARY

Understanding appropriate application of models such as MPM is important as transparency in healthcare drives demand for severity-adjusted outcomes data.

Outcome prediction in critical care: the Simplified Acute Physiology Score models

PURPOSE OF REVIEW

Outcome prediction models measuring severity of illness of patients admitted to the intensive care unit should predict hospital mortality. This review describes the state-of-the-art of Simplified Acute Physiology Score models from the clinical and managerial perspectives. Methodological issues concerning the effects of differences between new samples and original databases in which the models were developed are considered.

RECENT FINDINGS

The progressive lack of fit of the Simplified Acute Physiology Score II in independent intensive care unit populations induced investigators to propose customizations and expansions as potential evolutions for Simplified Acute Physiology Score II. We do not know whether those solutions did solve the issue because there are no demonstrations of consistent good fit in new databases. The recently developed Simplified Acute Physiology Score 3 Admission Score with customization for geographical areas is discussed. The points shared by the Simplified Acute Physiology Score models and the pros and cons for each of them are introduced.

SUMMARY

Comparisons of intensive care unit performance should take into account not only the patient severity of illness, but also the effect of the 'intensive care unit variable', that is, differences in human resources, structure, equipment, management and organization of the intensive care unit. In the future, moving from patient and geographical area adjustment to resource use could allow the user to adjust for differences in healthcare provision.

Are high-quality cardiac surgeons less likely to operate on high-risk patients compared to low-quality surgeons? Evidence from New York State

CONTEXT

It is unknown whether high-risk cardiac surgical patients have less access to high-quality surgeons compared with lower-risk patients.

OBJECTIVE

To determine whether high-quality surgeons are less likely to perform coronary artery bypass graft (CABG) surgery on high-risk patients compared with low-quality surgeons.

DESIGN, SETTING, AND PATIENTS

Retrospective cohort study using the New York State (NYS) CABG Surgery Reporting System (CSRS) of all patients undergoing CABG surgery in NYS who were discharged between 1997 and 1999 (51,750 patients; 2.20 percent mortality). Regression modeling was used to estimate the association between surgeon quality and patient risk of death. Surgeon quality was quantified using the observed-to-expected mortality ratio (O-to-E ratio).

RESULTS

Higher-risk patients are more likely to receive CABG surgery from higher-quality surgeons. For every 10 percentage point increase in patient risk of death (e.g., from 5 to 15 percent), there is an absolute reduction of 0.034 in the surgeon O-to-E ratio (p < .001).

CONCLUSION

This study suggests that high-risk CABG patients are significantly more likely to receive care from high-quality surgeons compared with lower risk patients.

The impact of different prognostic models and their customization on institutional comparison of intensive care units

OBJECTIVES

To evaluate the influence of choice of a prognostic model and the effect of customization of these models on league tables (i.e., rank-order listing) in which intensive care units (ICUs) are ranked by standardized mortality ratios using Acute Physiology and Chronic Health Evaluation (APACHE) II, Simplified Acute Physiology Score (SAPS) II, and Mortality Probability Model II (MPM24II).

DESIGN

Retrospective analysis of prospectively collected data on ICU admissions.

SETTING

Forty Dutch ICUs.

PATIENTS

A data set from a national registry of 86,427 patients from January 2002 to October 2006.

INTERVENTIONS

The league tables associated with the different models were compared to evaluate their agreement. Bootstrapping was used to quantify the uncertainty in the ranks for ICUs. First, for each ICU the median rank and its 95% confidence interval were identified for each model. Then, for a given pair of models, for each ICU the median difference in rank and its associated 95% confidence interval were computed. A difference in rank for an ICU for a given pair of models was considered relevant if it was statistically significant and if one of the models would categorize this ICU as a performance outlier (excellent performer or very poor performer) while the other did not.

MEASUREMENTS AND MAIN RESULTS

For 20 ICUs, there was a significant difference in rank (2-19 positions) between one or more pairs of models. Three ICUs were rated as performance outliers by one of the models, while the other excluded this possibility with 95% certainty. Furthermore, for ten ICUs, one or more pairs of models classified these ICUs as performance outliers while the other model did not do so with certainty. Regarding the agreement between the original models and their customized versions, in all cases the median change in rank was three positions or less and the models fully agreed with respect to which ICUs should be classified as performance outliers.

CONCLUSIONS

Institutional comparison based on case-mix adjusted league tables is sensitive to the choice of prognostic model but not to customization of these models. League tables should always display the uncertainty associated with institutional ranks.

Transferring critically ill patients out of hospital improves the standardized mortality ratio: a simulation study

BACKGROUND

Transferring critically ill patients to other acute care hospitals may artificially impact benchmarking measures. We sought to quantify the effect of out-of-hospital transfers on the standardized mortality ratio (SMR), an outcome-based measure of ICU performance.

METHODS

We performed a cohort study and Monte Carlo simulation using data from 85 ICUs participating in the acute physiology and chronic health evaluation (APACHE) clinical information system from 2002 to 2003. The SMR (observed divided by expected hospital mortality) was calculated for each ICU using APACHE IV risk adjustment. A set number of patients was randomly assigned to be transferred out alive rather than experience their original outcome. The SMR was recalculated, and the mean simulated SMR was compared to the original.

RESULTS

The mean (+/- SD) baseline SMR was 1.06 +/- 0.19. In the simulation, increasing the number of transfers by 2% and 6% over baseline decreased the SMR by 0.10 +/- 0.03 and 0.14 +/- 0.03, respectively. At a 2% increase, 27 ICUs had a decrease in SMR of > 0.10, and two ICUs had a decrease in SMR of > 0.20. Transferring only one additional patient per month was enough to create a bias of > 0.1 in 27 ICUs.

CONCLUSIONS

Increasing the number of acute care transfers by a small amount can significantly bias the SMR, leading to incorrect inference about ICU quality. Sensitivity to the variation in hospital discharge practices greatly limits the use of the SMR as a quality measure.

Quantifying risk and benchmarking performance in the adult intensive care unit

Morbidity, mortality, and length-of-stay outcomes in patients receiving critical care are difficult to interpret unless they are risk-stratified for diagnosis, presenting severity of illness, and other patient characteristics. Acuity adjustment systems for adults include the Acute Physiology And Chronic Health Evaluation (APACHE), the Mortality Probability Model (MPM), and the Simplified Acute Physiology Score (SAPS). All have recently been updated and recalibrated to reflect contemporary results. Specialized scores are also available for patient subpopulations where general acuity scores have drawbacks. Demand for outcomes data is likely to grow with pay-for-performance initiatives as well as for routine clinical, prognostic, administrative, and research applications. It is important for clinicians to understand how these scores are derived and how they are properly applied to quantify patient severity of illness and benchmark intensive care unit performance.

Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit*

OBJECTIVE

Numerous factors can cause delays in transfer to an intensive care unit for critically ill emergency department patients. The impact of delays is unknown. We aimed to determine the association between emergency department "boarding" (holding admitted patients in the emergency department pending intensive care unit transfer) and outcomes for critically ill patients.

DESIGN

This was a cross-sectional analytical study using the Project IMPACT database (a multicenter U.S. database of intensive care unit patients). Patients admitted from the emergency department to the intensive care unit (2000-2003) were included and divided into two groups: emergency department boarding >or=6 hrs (delayed) vs. emergency department boarding <6 hrs (nondelayed). Demographics, intensive care unit procedures, length of stay, and mortality were analyzed. Groups were compared using chi-square, Mann-Whitney, and unpaired Student's t-tests.

SETTING

Emergency department and intensive care unit.

PATIENTS

Patients admitted from the emergency department to the intensive care unit (2000-2003).

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Main outcomes were intensive care unit and hospital survival and intensive care unit and hospital length of stay. During the study period, 50,322 patients were admitted. Both groups (delayed, n = 1,036; nondelayed, n = 49,286) were similar in age, gender, and do-not-resuscitate status, along with Acute Physiology and Chronic Health Evaluation II score in the subgroup for which it was recorded. Among hospital survivors, the median hospital length of stay was 7.0 (delayed) vs. 6.0 days (nondelayed) (p < .001). Intensive care unit mortality was 10.7% (delayed) vs. 8.4% (nondelayed) (p < .01). In-hospital mortality was 17.4% (delayed) vs. 12.9% (nondelayed) (p < .001). In the stepwise logistic model, delayed admission, advancing age, higher Acute Physiology and Chronic Health Evaluation II score, male gender, and diagnostic categories of trauma, intracerebral hemorrhage, and neurologic disease were associated with lower hospital survival (odds ratio for delayed admission, 0.709; 95% confidence interval, 0.561-0.895).

CONCLUSIONS

Critically ill emergency department patients with a >or=6-hr delay in intensive care unit transfer had increased hospital length of stay and higher intensive care unit and hospital mortality. This suggests the need to identify factors associated with delayed transfer as well as specific determinants of adverse outcomes.

Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III)

OBJECTIVE

To update the Mortality Probability Model at intensive care unit (ICU) admission (MPM0-II) using contemporary data.

DESIGN

Retrospective analysis of data from 124,855 patients admitted to 135 ICUs at 98 hospitals participating in Project IMPACT between 2001 and 2004. Independent variables considered were 15 MPM0-II variables, time before ICU admission, and code status. Univariate analysis and multivariate logistic regression were used to identify risk factors associated with hospital mortality.

SETTING

One hundred thirty-five ICUs at 98 hospitals.

PATIENTS

Patients in the Project IMPACT database eligible for MPM0-II scoring.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Hospital mortality rate in the current data set was 13.8% vs. 20.8% in the MPM0-II cohort. All MPM0-II variables remained associated with mortality. Clinical conditions with high relative risks in MPM0-II also had high relative risks in MPM0-III. Gastrointestinal bleeding is now associated with lower mortality risk. Two factors have been added to MPM0-III: "full code" resuscitation status at ICU admission, and "zero factor" (absence of all MPM0-II risk factors except age). Seven two-way interactions between MPM0-II variables and age were included and reflect the declining marginal contribution of acute and chronic medical conditions to mortality risk with increasing age. Lead time before ICU admission and pre-ICU location influenced individual outcomes but did not improve model discrimination or calibration. MPM0-III calibrates well by graphic comparison of actual vs. expected mortality, overall standardized mortality ratio (1.018; 95% confidence interval, 0.996-1.040) and a low Hosmer-Lemeshow goodness-of-fit statistic (11.62; p = .31). The area under the receiver operating characteristic curve was 0.823.

CONCLUSIONS

MPM0-II risk factors remain relevant in predicting ICU outcome, but the 1993 model significantly overpredicts mortality in contemporary practice. With the advantage of a much larger sample size and the addition of new variables and interaction effects, MPM0-III provides more accurate comparisons of actual vs. expected ICU outcomes.

Use of a Matching Algorithm to Evaluate Hospital Coronary Artery Bypass Grafting Performance as an Alternative to Conventional Risk Adjustment

BACKGROUND

Although public reporting of hospital and physician performance is a cornerstone of the effort to improve health care quality, the optimal approach to risk adjustment is unknown.

OBJECTIVE

We sought to assess hospital quality using a matching algorithm based on a generalized distance metric and to compare this approach to the more traditional regression-based approach. DESIGN/ DATA SOURCE: This was a retrospective study using the New York State (NYS) Coronary Artery Bypass Surgery Reporting System (CSRS), focusing on all patients undergoing isolated CABG surgery in NYS who were discharged in 1999 (18,116 patients). Patients from specific hospital were matched to a control group using the Mahalanobis distance. The hospitals' expected mortality rate was calculated in 2 ways: (1) as the mortality rate of the control group or (2) as the mortality rate predicted by the NYS CABG model. Hospitals whose observed mortality rate was significantly different from their expected mortality rate (OE difference) were defined as quality outliers.

RESULTS

The 2 risk-adjustment methodologies disagreed on the outlier status of 4 of the 33 hospitals. Kappa analysis demonstrated substantial agreement between these 2 methods for identifying quality outliers: kappa = 0.61. There was excellent agreement between the point estimates of the OE difference obtained using these 2 risk adjustment methodologies.

CONCLUSION

Basing outcome assessment on either matching or regression modeling yielded similar findings on hospital ranking but only moderate level of agreement on hospital quality. The use of matching may enhance the transparency and acceptance of outcome report cards by hospitals and physicians.

Methods to assess performance of models estimating risk of death in intensive care patients: a review

Models that estimate the probability of death of intensive care unit patients can be used to stratify patients according to the severity of their condition and to control for casemix and severity of illness. These models have been used for risk adjustment in quality monitoring, administration, management and research and as an aid to clinical decision making. Models such as the Mortality Prediction Model family, SAPS II, APACHE II, APACHE III and the organ system failure models provide estimates of the probability of in-hospital death of ICU patients. This review examines methods to assess the performance of these models. The key attributes of a model are discrimination (the accuracy of the ranking in order of probability of death) and calibration (the extent to which the model's prediction of probability of death reflects the true risk of death). These attributes should be assessed in existing models that predict the probability of patient mortality, and in any subsequent model that is developed for the purposes of estimating these probabilities. The literature contains a range of approaches for assessment which are reviewed and a survey of the methodologies used in studies of intensive care mortality models is presented. The systematic approach used by Standards for Reporting Diagnostic Accuracy provides a framework to incorporate these theoretical considerations of model assessment and recommendations are made for evaluation and presentation of the performance of models that estimate the probability of death of intensive care patients.

Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients

OBJECTIVE

To improve the accuracy of the Acute Physiology and Chronic Health Evaluation (APACHE) method for predicting hospital mortality among critically ill adults and to evaluate changes in the accuracy of earlier APACHE models.

DESIGN

: Observational cohort study.

SETTING

A total of 104 intensive care units (ICUs) in 45 U.S. hospitals.

PATIENTS

A total of 131,618 consecutive ICU admissions during 2002 and 2003, of which 110,558 met inclusion criteria and had complete data.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We developed APACHE IV using ICU day 1 information and a multivariate logistic regression procedure to estimate the probability of hospital death for randomly selected patients who comprised 60% of the database. Predictor variables were similar to those in APACHE III, but new variables were added and different statistical modeling used. We assessed the accuracy of APACHE IV predictions by comparing observed and predicted hospital mortality for the excluded patients (validation set). We tested discrimination and used multiple tests of calibration in aggregate and for patient subgroups. APACHE IV had good discrimination (area under the receiver operating characteristic curve = 0.88) and calibration (Hosmer-Lemeshow C statistic = 16.9, p = .08). For 90% of 116 ICU admission diagnoses, the ratio of observed to predicted mortality was not significantly different from 1.0. We also used the validation data set to compare the accuracy of APACHE IV predictions to those using APACHE III versions developed 7 and 14 yrs previously. There was little change in discrimination, but aggregate mortality was systematically overestimated as model age increased. When examined across disease, predictive accuracy was maintained for some diagnoses but for others seemed to reflect changes in practice or therapy.

CONCLUSIONS

APACHE IV predictions of hospital mortality have good discrimination and calibration and should be useful for benchmarking performance in U.S. ICUs. The accuracy of predictive models is dynamic and should be periodically retested. When accuracy deteriorates they should be revised and updated.

Variation in pediatric intensive care therapies and outcomes by race, gender, and insurance status

CONTEXT

The differential allocation of medical resources to adult patients according to characteristics such as race, gender, and insurance status raises the serious concern that such issues apply to critically ill children as well.

OBJECTIVE

This study examined whether medical resources and outcomes for children admitted to pediatric intensive care units differed according to race, gender, or insurance status.

DESIGN

An observational analysis was conducted with use of prospectively collected data from a multicenter cohort. Data were collected on 5,749 consecutive admissions for children from three pediatric intensive care units located in large urban children's hospitals.

PARTICIPANTS

Children aged </=18 years admitted over an 18-month period beginning in June 1996 formed the study sample.

MAIN OUTCOME MEASURES

Hospital mortality, length of hospital stay, and overall resource use were examined in relation to severity of illness. Standardized ratios were formed with generalized regression analyses that included the Pediatric Index of Mortality for risk adjustment.

RESULTS

After adjustment for differences in illness severity, standardized mortality ratios and overall resource use were similar with regard to race, gender, and insurance status, but uninsured children had significantly shorter lengths of stay in the pediatric intensive care unit. Uninsured children also had significantly greater physiologic derangement on admission (mortality probability, 8.1%; 95% confidence interval [CI], 6.2-10.0) than did publicly insured (3.6%; 95% CI, 3.2-4.0) and commercially insured patients (3.7%; 95% CI, 3.3-4.1). Consistent with greater physiologic derangement, hospital mortality was higher among uninsured children than insured children.

CONCLUSIONS

Risk-adjusted mortality and resource use for critically ill children did not differ according to race, gender, or insurance status. Policies to expand health insurance to children appear more likely to affect physiologic derangement on admission rather than technical quality of care in the pediatric intensive care unit setting.

Relationship between tracheostomy timing and duration of mechanical ventilation in critically ill patients

OBJECTIVE

Tracheostomy practice in the setting of critical illness is controversial because evidence demonstrating unequivocal benefit is lacking. We undertook this study to determine the relationship between tracheostomy timing and duration of mechanical ventilation, intensive care unit length of stay, and hospital length of stay and to evaluate the relative influence of clinical and nonclinical factors on tracheostomy practice.

DESIGN

Analysis of Project Impact, a multi-institutional critical care administrative database.

SETTING

Medical school.

PATIENTS

Data from 43,916 patients were reviewed.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Tracheostomy was performed in 2,473 (5.6%) of 43,916 patients analyzed. Tracheostomy patients had a higher survival rate than nontracheostomy patients (78.1 vs. 71.7%, p < .001) and underwent this procedure following a median (25th-75th percentile) of 9.0 (5.0-14.0) days of ventilatory support. Tracheostomy frequency and timing varied significantly comparing patient, intensive care unit, and hospital characteristics (p < .05 for all). Tracheostomy timing correlated significantly with duration of mechanical ventilation (r = .690), intensive care unit (r = .610), and hospital length of stay (r = .341, p < .001 for all). At most, 22% of patients were supported via tracheostomy at any given time. Although a minority, tracheostomy patients accounted for 26.2%, 21.0%, and 13.5% of all ventilator, intensive care unit, and hospital days, respectively.

CONCLUSIONS

Although practice varies substantially, tracheostomy timing appears significantly associated with duration of mechanical ventilation, intensive care unit length of stay, and hospital length of stay. These findings emphasize the need for an adequately supported multiple-center trial to better define patient selection for tracheostomy and to test the hypothesis that timing of this procedure influences clinically important outcomes.

Evaluating the performance of an institution using an intensive care unit benchmark

OBJECTIVES

To describe the performances of selected intensive care units (ICUs) in a single institution using the Acute Physiology and Chronic Health Evaluation (APACHE) III benchmark and to propose interventions that may improve performance.

PATIENTS AND METHODS

In this retrospective study, we analyzed APACHE III data from critically ill patients admitted to ICUs at the Mayo Clinic in Rochester, Minn, between October 1994 and December 2003. We retrieved ICU performance measures based on first ICU day APACHE III values. Standardized ratios were defined as ratios of measured to predicted values. The primary performance measure was the standardized mortality ratio, and secondary performance measures were length of stay (LOS) ratios, low-risk monitor ICU admission rates, and ICU readmission rates. We calculated 95% confidence intervals (CIs) for each performance, graded as good, average, or poor.

RESULTS

Among 46,381 patients admitted during the study period, 57.5% were in surgical ICUs, 24.8% in a medical ICU, and 17.7% in a surgical-medical ICU. Low-risk monitoring accounted for 37.2% of admissions. Hospital standardized mortality ratios (95% CI) were 0.95 (0.90-0.99), 0.86 (0.81-0.91), and 0.70 (0.66-0.74) for medical, multispecialty, and surgical ICUs, respectively. Hospital LOS ratios (95% CI) were 0.83 (0.81-0.85), 0.91 (0.88-0.93), and 0.99 (0.97-1.00) for medical, multispecialty, and surgical ICUs, respectively. The ICU readmission rate for each ICU was higher than the 6.7% reported in the medical literature. Performances were good in mortality, average to good in LOS, average in low-risk admission, and poor in ICU readmission.

CONCLUSIONS

A national benchmarking database can highlight the strengths and weaknesses of ICUs. The performances of ICUs in a single institution may differ; therefore, the performance of each unit should be evaluated individually.

SAPS�II revisited

OBJECTIVE

To construct and validate an update of the Simplified Acute Physiology Score II (SAPS II) for the evaluation of clinical performance of Intensive Care Units (ICU).

DESIGN AND SETTING

Retrospective analysis of prospectively collected multicenter data in 32 ICUs located in the Paris area belonging to the Cub-Rea database and participating in a performance evaluation project.

PATIENTS

33,471 patients treated between 1999 and 2000.

MEASUREMENTS AND RESULTS

Two logistic regression models based on SAPS II were developed to estimate in-hospital mortality among ICU patients. The second model comprised reevaluation of original items of SAPS II and integration of the preadmission location and chronic comorbidity. Internal and external validation were performed. In the two validation samples the most complex model had better calibration than the original SAPS II for in-hospital mortality but its discrimination was not significantly higher (area under ROC curve 0.89 vs. 0.87 for SAPS II). Second-level customization and integration of new items improved uniformity of fit for various categories of patients except for diagnosis-related groups. The rank order of ICUs was modified according to the model used.

CONCLUSIONS

The overall performance of SAPS II derived models was good, even in the context of a community cohort and routinely gathered data. However, one-half the variation of outcome remains unexplained after controlling for admission characteristics, and uniformity of prediction across diagnostic subgroups was not achieved. Differences in case-mix still limit comparisons of quality of care.

Mortality rate and length of stay of patients admitted to the intensive care unit in July*

OBJECTIVE

At the beginning of each academic year in July, inexperienced residents and fellows begin to care for patients. This inexperience can lead to poor patient outcome, especially in patients admitted to the intensive care unit (ICU). The objective of this study was to determine the impact of July ICU admission on patient outcome.

DESIGN

Retrospective, cohort study.

SETTING

Academic, tertiary medical center.

PATIENTS

Patients admitted to the ICU from October 1994 through September 2002.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Demographics, Acute Physiology and Chronic Health Evaluation (APACHE) III score and predicted mortality, admission source, admission date, intensity of treatment, ICU length of stay (LOS), and hospital mortality of 29,084 patients were obtained. The actual and predicted weighted ICU LOS and their ratio were calculated. Logistic regression analysis was used to compare the hospital mortality rate of patients admitted to the ICU in July with those admitted during the rest of the year, with adjustment for potentially confounding variables. The patients' mean age was 62.3 +/- 17.6 yrs; 57.3% were male and 95.5% white. Both the customized predicted and observed hospital mortality rates of the entire cohort were 8.2%. The majority (76.7%) of the patients were discharged home, and 15.1% were discharged to other facilities. When adjusted for potentially confounding variables, ICU admission in July was not associated with higher hospital mortality rate compared with any other month. There were no significant differences in the discharge location of patients between July and any one of the other months. There were no statistically significant differences in the weighted ICU LOS ratio between July and any of the other months.

CONCLUSIONS

ICU admission in July is not associated with increased hospital mortality rate or ICU length of stay.

Using hierarchical modeling to measure ICU quality

OBJECTIVE

To determine whether hierarchical modeling agrees with conventional logistic regression modeling on the identity of ICU quality outliers within a large multi-institutional database.

DESIGN

Retrospective database analysis.

SETTING AND PATIENTS

Subset of the Project IMPACT database consisting of 40435 adult patients admitted to surgical, medical, and mixed surgical-medical ICUs ( n=55) between 1997 and 1999 who met inclusion criteria for SAPS II.

MEASUREMENTS AND RESULTS

The SAPS II score was customized to this database using conventional logistic regression and using a hierarchical (random coefficients) model. Both models exhibited excellent discrimination ( Cstatistic) and calibration (Hosmer-Lemeshow statistic). The hierarchical and nonhierarchical models had C statistics of.870 and.865, and HL statistics of 3.71 ( p>.88, df=8) and 8.94 ( p>.35, df=8), respectively. Since the random effects component of the hierarchical model accounts for between-hospital variability, only the fixed-effects coefficients were used to calculate the expected mortality rate based on the hierarchical model. The ratio and 95% confidence intervals of the observed to expected mortality rate were calculated using both models for each ICU. ICUs whose observed/expected ratio was either less than 1 or greater than 1, and whose 95% confidence interval did not include 1 were labeled as either high-performance or low-performance outliers, respectively. Analysis using kappa statistic revealed almost perfect agreement between the two models (nonhierarchical vs. hierarchical) on the identity of ICU quality outliers.

CONCLUSIONS

Models obtained by customizing SAPS II using a nonhierarchical and a hierarchical approach exhibit excellent agreement on the identity of ICU quality outliers.

Gender-related differences in intensive care: a multiple-center cohort study of therapeutic interventions and outcome in critically ill patients

OBJECTIVE

To determine whether gender-related differences exist in the provided level of care and outcome in a large cohort of critically ill patients.

DESIGN

Prospective, observational cohort study with data collection from January 1, 1998, to December 31, 2000.

SETTING

Thirty-one intensive care units in Austria.

PATIENTS

A total of 25,998 adult patients, consecutively admitted to 31 intensive care units in Austria.

INTERVENTIONS

We assessed severity of illness, level of provided care, and vital status at hospital discharge.

MEASUREMENTS AND MAIN RESULTS

Of 25,998 patients, 58.3% were male and 41.7% were female. Hospital mortality rate was slightly higher in women (18.1%) than in men (17.2%), but severity of illness-adjusted mortality rate was not different. Men received an overall increased level of care and had a significantly higher probability of receiving invasive procedures, such as mechanical ventilation (odds ratio [OR], 1.22; 95% confidence interval [CI], 1.16-1.28), single vasoactive medication (OR, 1.18; 95% CI, 1.12-1.24), multiple vasoactive medication (OR, 1.21; 95% CI, 1.15-1.28), intravenous replacement of large fluid losses (OR, 1.14; 95% CI, 1.08-1.20), central venous catheter (OR, 1.06; 95% CI, 1.01-1.12), peripheral arterial catheter (OR, 1.15; 95% CI, 1.10-1.22), pulmonary artery catheter (OR, 1.48; 95% CI, 1.34-1.62), renal replacement therapy (OR, 1.28; 95% CI, 1.16-1.42), and intracranial pressure measurement (OR, 1.34; 95% CI, 1.18-1.53).

CONCLUSIONS

In a large cohort of critically ill patients, no differences in severity of illness-adjusted mortality rate between men and women were found. Despite a higher severity of illness in women, men received an increased level of care and underwent more invasive procedures. This different therapeutic approach in men did not translate into a better outcome.