ICU scoring systems do not allow prediction of patient outcomes or comparison of ICU performance

Factors associated with limitation of life support: Post-ICU mortality case study of a tertiary hospital

BACKGROUND

Life-sustaining treatment limitation (LSV) is the medical act of withdrawing or not initiating measures that are considered futile in a patient's specific situation. LSV in critically ill patients remains a difficult topic to study, due to the multitude of factors that condition it.

OBJECTIVE

To determine factors related to LSV in ICU in cases of post-ICU in-hospital mortality, as well as factors associated with survival after discharge from ICU.

DESIGN

Retrospective longitudinal study.

AMBIT

Intensive care unit of a tertiary hospital.

PATIENTS

People who died in the hospitalization ward after ICU treatment between January 2014 and December 2019.

INTERVENTIONS

None. This is an observational study.

VARIABLES OF INTEREST

Age, sex, probability of death, type of admission, LSV in ICU, oncological disease, dependence, invasive mechanical ventilation, emergency hemodialysis, transfusion of blood products, nosocomial infection (NI), pre-ICU, intra-ICU and post-ICU stays.

RESULTS

Of 114 patients who died outside the ICU, 49 had LSV registered in the ICU (42.98%). Age and stay prior to ICU admission were positively associated with LSV (OR 1,03 and 1,08, respectively). Patients without LSV had a higher post-ICU stay, while it was lower for male patients.

CONCLUSIONS

Our results support that LSV established within the ICU can avoid complications commonly associated with unnecessary prolongation of stay, such as NI.

Prospective comparison of three risk score models at three different surgical intensive care units

PURPOSE

Although risk score models are of great value, their use is restricted because of the additional effort involved. The aim of this study was to compare three different score systems. Each of these requires a different degree of effort by the medical staff. One of the score systems is solely based on routine laboratory parameters. Data were collected on three different ICUs units, with each showing a large variety in patients' health conditions.

METHODS

Prospective data of 588 surgical patients were collected by means of Acute Physiology and Chronic Health Evaluation II (APACHE II), Dense Laboratory Whole Blood Applied Risk Estimation (DELAWARE), and Simplified Acute Physiology Score II (SAPS II) score systems. These patients were admitted to three different intensive care units over a period of 12 months. On the day of admission, predicted hospital survival and mortality were evaluated.

RESULTS

With a cutoff value of 0.6, the sensitivity of the APACHE II, DELAWARE, and SAPS II was at 0.19, 0.24, and 0.21; the specificity was at 0.98, 0.92, and 0.98; and the correct classification rate at 0.86, 0.83, and 0.86. The r(2) value was 0.35 for the APACHE II, 0.12 for the DELAWARE, and 0.21 for the SAPS II. The hospital mortality rate was overestimated in all three score systems.

CONCLUSIONS

The results of this first multicenter study comparing three risk score systems indicate that it is possible to establish a general risk score for surgical intensive care patients on admission date. Such a risk score is solely based on quality-controlled, low-cost routine laboratory parameters.

Balancing operating theatre and bed capacity in a cardiothoracic centre

Cardiothoracic surgery requires many expensive resources. This paper examines the balance between operating theatres and beds in a specialist facility providing elective heart and lung surgery. Without both operating theatre time and an Intensive Care bed a patient's surgery has to be postponed. While admissions can be managed, there are significant stochastic features, notably the cancellation of theatre procedures and patients' length of stay on the Intensive Care Unit. A simulation was developed, with clinical and management staff, to explore the interdependencies of resource availabilities and the daily demand. The model was used to examine options for expanding the capacity of the whole facility. Ideally the bed and theatre capacity should be well balanced but unmatched increases in either resource can still be beneficial. The study provides an example of a capacity planning problem in which there is uncertainty in the demand for two symbiotic resources.

Long-term comparison of a routine laboratory parameter-based severity score with APACHE II and SAPS II

BACKGROUND

Risk score models predicting mortality have tremendous value, but because of the additional effort involved, their clinical use remains low. The aim of this study is to compare three different scores that each requires different levels of effort during admission and throughout treatment: the Acute Physiology and Chronic Health Evaluation II (APACHE II), the Simplified Acute Physiology Score II (SAPS II), and the Dense Laboratory Whole Blood Applied Risk Estimation (DELAWARE) score. Of the three, only the DELAWARE is based solely on routine laboratory parameters.

METHODS

Prospective data of the three scores were collected for 268 surgical patients admitted to the intensive care unit over 1 year. The predicted hospital mortality and survival were evaluated for the first 14 days.

RESULTS

With a cutoff value of 0.65, the sensitivity of the DELAWARE was 71.6%, the specificity, 92.5%, and the correct classification rate, 87.3%. The APACHE II and SAPS II showed values of 41.2%/96.8%/86.2% and 62.7%/87.1%/82.5%, respectively. The r2 value was 0.884 for the DELAWARE, 0.876/0.814 for the APACHE II and SAPS II. Hospital mortality rate was overestimated by 20% to 65% in all scores. The discriminatory ability of the APACHE II and SAPS II increased throughout the course of treatment.

CONCLUSIONS

The routine laboratory-based DELAWARE provides a reliable, valid risk assessment of the surgical intensive care patient at admission. It also provides additional information without added effort or poor interobserver reliability, which leads to better data comparability. We have to state that until now the data have been collected in a single-center and their general validity is therefore limited. By the end of treatment, the SAPS II and APACHE II had increased discriminatory ability and are therefore useful as process parameters.

Trauma intensive care unit survival: how good is an educated guess?

BACKGROUND

Resource utilization in medicine is becoming a more and more urgent issue with ongoing national discussions on healthcare coverage. In the management of a trauma system, large amounts of resources and money are expended on individual patients in hope of a "great save." In addition, those of us caring for these patients are required to estimate outcomes daily to the family in an effort to choose the best course of care for an individual patient. Hence, we undertook a study to analyze the accuracy of outcomes predictions of various members of the healthcare team.

METHODS

During a period of 38 months (July 2005 to August 2008), an observational study of patients admitted to a Level I Trauma Center Intensive Care Unit (ICU) was undertaken. Institutional Review Board permission was obtained before starting the study. Only patients older than 18 years were included. Patients who were moribund or expected discharge within 72 hours were excluded.Our traumatized ICU patients are cared for by a multidisciplinary team consisting of a trauma/ICU attending, all of whom have additional certification in surgical critical care and who rotate through the ICU on a weekly basis, a surgical ICU fellow, residents and medical students of several levels of training who rotate on a monthly basis, trauma advanced-level practitioners who rotate weekly, and bedside ICU nurses who work routine shifts. Respiratory therapists, nutritionists, ICU pharmacists, and other members of the rounding team were not included in the study because they do not provide global patient care. Regardless of admitting physician, the patients are managed by the team, and our practice of care is similar across the group, based on protocols and consensus.For each of the study patients, a survey tool was filled out by the ICU rounding team on hospital day 1 and hospital day 3. The tool was completed by members of the team providing global care to the patient and varied depending on the members of the group at each day's rounds. All current and admission data on injuries, study and laboratory results, and current patient status were available to all members of the team. Each member was expected to fill out the survey tool independently, and the results of the tool were not discussed during rounds.Concurrently, data were collected by the ICU fellow and research nurse. These data and the results of the survey tools were entered in a database for analysis after patient discharge. A retrospective analysis was undertaken to analyze the relative accuracy of the care, team members' assessment, and actual survival. Statistical analysis was done using by-chance accuracy comparisons.

RESULTS

Two hundred twenty-three patients had 326 observations performed. Day 3 accuracy improved for most groups. In all groups, accuracy was found to be statistically significantly better than by-chance accuracy. Given that the majority of patients in the trauma population are survivors, sensitivity and positive predictive value of the observer's ability to predict death were also evaluated.

CONCLUSIONS

Although significantly better than chance prediction, the ability of members of the ICU team to predict survival of trauma patients remains poor, particularly on initial evaluation. A period of clinical observation improves the accuracy. Unfortunately, experience of the observer does not seem to improve accuracy of survival prediction. This data indicate that care must be taken when describing likely outcomes to patient family members.

Prognosis in critical care

Prognostic risk prediction models have been employed in the intensive care unit (ICU) setting since the 1980s and provide health care providers with important information to help inform decisions related to treatment and prognosis, as well as to compare outcomes across institutions. Prognostic models for critical care are among the most widely utilized and tested predictive models in healthcare. In this article, we review and compare mortality prediction models, including the APACHE (1981), SAPS (1984), APACHE-II (1985), MPM (1987), APACHE-III (1991), SAPS-II (1993), and MPM-II (1993). We emphasize the importance of model calibration in this domain. In addition, we present a brief review of the statistical methodology, multiple logistic regression, which underlies most of the models currently used in critical care.

Review article: scoring systems for assessing prognosis in critically ill adult cirrhotics

BACKGROUND

Cirrhotic patients admitted to intensive care units (ICU) still have poor outcomes. Some current ICU prognostic models [Acute Physiology and Chronic Health Evaluation (APACHE), Organ System Failure (OSF) and Sequential Organ Failure Assessment (SOFA)] were used to stratify cirrhotics into risk categories, but few cirrhotics were included in the original model development. Liver-specific scores [Child-Turcotte-Pugh (CTP) and model for end-stage liver disease (MELD)] could be useful in this setting.

AIM

To evaluate whether ICU prognostic models perform better compared with liver-disease specific ones in cirrhotics admitted to ICU.

METHODS

We performed a structured literature review identifying clinical studies focusing on prognosis and risk factors for mortality in adult cirrhotics admitted to ICU.

RESULTS

We found 21 studies (five solely dealing with gastrointestinal bleeding) published during the last 20 years (54-420 patients in each). APACHE II and III, SOFA and OSF had better discrimination for correctly predicting death compared with the CTP score. The MELD score was evaluated only in one study and had good predictive accuracy [receiver operator characteristic (ROC) curve: 0.81). Organ dysfunction models (OSF, SOFA) were superior compared with APACHE II and III (ROC curve: range 0.83-0.94 vs. 0.66-0.88 respectively). Cardiovascular, liver and renal system dysfunction were more frequently independently associated with mortality.

CONCLUSIONS

General-ICU models had better performance in cirrhotic populations compared with CTP score; OSF and SOFA had the best predictive ability. Further prospective and validation studies are needed.

Determinants of mortality in patients with severe sepsis

OBJECTIVE

To evaluate the relative importance of predictors of in-hospital mortality in severe sepsis and compare the performance of generic and disease-specific mortality prediction models.

METHODS

The author used data from all 826 patients receiving placebo in the Recombinant Human Activated Protein C Worldwide Evaluation in Severe Sepsis (PROWESS) trial. After a variety of clinical factors were examined for their univariate association with in-hospital mortality, logistic regression models incorporating successively more inclusive sets of predictors were created and compared. For each model, discrimination was assessed and the relative contribution of each model component to overall model explanatory power evaluated. The accuracy of using the Acute Physiology and Chronic Health Evaluation (APACHE) II score in isolation as an indicator of "high risk" was assessed by comparing model predictions from APACHE-only models to those of disease-specific models.

RESULTS

Age, a number of laboratory values, and APACHE II score were significant univariate predictors of mortality. In multivariable models, age and laboratory values contributed the most information to model predictions; the contribution of the APACHE II score, in particular, the acute physiology component, was modest at best. A risk model including only the total APACHE II score had a c-statistic of 0.686, whereas the best performing disease-specific model had a c-statistic of 0.787. Use of the APACHE II score alone to establish high risk versus low risk resulted in misclassification of 26% of patients.

CONCLUSIONS

Individual severe sepsis patient outcomes depend on an array of clinical predictors. Models incorporating sepsis disease-specific risk factors may predict mortality more accurately than generic ICU severity measures.

Impact of operator expertise on collection of the APACHE II score and on the derived risk of death and standardized mortality ratio

We assessed the impact of operator expertise on collection of the APACHE II score, the derived risk of death and standardized mortality ratio in 465 consecutive patients admitted to a multi-disciplinary tertiary hospital ICU. Research coordinators and junior clinical staff independently collected the APACHE II variables; experts (senior clinical staff) rescored 20% of the records. Agreement was moderate between junior clinical staff and research coordinators or senior clinical staff for most variables of the acute physiology score (weighted kappa<0.6); agreement between research coordinators and senior clinical staff data collectors was good (weighted kappa >0.75). The APACHE II score and its derived risk of death (ROD) were significantly lower using the junior clinical staff dataset compared to research coordinators and senior clinical staff (APACHE II score: 13.4+/-9.2 vs 16.8+/-8.5 vs 17.1+/-7 7, P<0.001; ROD: 14.7%+/-22.4% vs 21.6%+/-22.6% vs 20.8%+/-22.4%, P<0.01 respectively). The discriminative capacity was not altered by the lack of agreement (area under Receiver Operator Characteristic curve >0.8) but calibration of ROD from the junior clinical staff dataset was poor (Goodness-of-fit: P= 0.001). The standardized mortality ratio (SMR) was higher with the junior clinical staff dataset (SMR: 1.22, 95% CI: 0.96-1.52 vs 0.87, 95% CI: 0.70-1.06 vs 0.76, 95% CI: 0.40-1.3 calculated from junior clinical staff research coordinators and senior clinical staff datasets respectively). We conclude that the expertise of data collectors significantly influences the APACHE II score, the derived risk of death and the standardized mortality ratio. Given the importance of such scores, ICUs should be provided with sufficient resources to train and employ dedicated data collectors.

Application of analytic hierarchy process for measuring and comparing the global performance of intensive care units

PURPOSE

To develop a model for the global performance measurement of intensive care units (ICUs) and to apply that model to compare the services for quality improvement.

MATERIALS AND METHODS

Analytic hierarchy process, a multiple-attribute decision-making technique, is used in this study to evolve such a model. The steps consisted of identifying the critical success factors for the best performance of an ICU, identifying subfactors that influence the critical factors, comparing them pairwise, deriving their relative importance and ratings, and calculating the cumulative performance according to the attributes of a given ICU. Every step in the model was derived by group discussions, brainstorming, and consensus among intensivists.

RESULTS

The model was applied to 3 ICUs, 1 each in Barbados, Trinidad, and India in tertiary care teaching hospitals of similar setting. The cumulative performance rating of the Barbados ICU was 1.17 when compared with that of Trinidad and Indian ICU, which were 0.82 and 0.75, respectively, showing that the Trinidad and Indian ICUs performed 70% and 64% with respect to Barbados ICU. The model also enabled identifying specific areas where the ICUs did not perform well, which helped to improvise those areas.

CONCLUSIONS

Analytic hierarchy process is a very useful model to measure the global performance of an ICU.

A new tool for measurement of process‐based performance of multispecialty tertiary care hospitals

There is an increasing need of a model for the process-based performance measurement of multispecialty tertiary care hospitals for quality improvement. Analytic hierarchy process (AHP) is utilized in this study to evolve such a model. Each step in the model was derived by group-discussions and brainstorming sessions among experienced clinicians and managers. This tool was applied to two tertiary care teaching hospitals in Barbados and India. The model enabled identification of specific areas where neither hospital performed very well, and helped to suggest recommendations to improve those areas. AHP is recommended as a valuable tool to measure the process-based performance of multispecialty tertiary care hospitals.

Outcome evaluation in a surgical intensive care unit in Barbados

The performance of the Acute Physiology and Chronic Health Evaluation II scoring system was prospectively assessed in the surgical intensive care unit at the Queen Elizabeth Hospital, Barbados. A total of 309 patients admitted consecutively during a 2-year period (1999-2001) were evaluated. Demographic data, diagnosis, Acute Physiology and Chronic Health Evaluation II score, duration of stay and hospital outcome were recorded. The predicted mortality for every patient and the costs incurred were also calculated. The overall observed mortality rate was 15.9% while the mean predicted mortality rate for our case-mix was 16.4%, which is comparable to results from developed countries. The cost incurred per patient was much lower at $13,636 (Barbados), compared to the patients' cost in North America ($60,000 Barbados).

A comparison of predictive outcomes of APACHE II and SAPS II in a surgical intensive care unit

The Acute Physiologic Score and Chronic Health Evaluation (APACHE) II and the Simplified Acute Physiologic Scale (SAPS) II are two of the more commonly employed predictors of outcome and performance in the intensive care unit setting. However, controversy persists about whether the scores generated by these systems have similar predictive value. This study compared the predicted mortalities derived from APACHE II and SAPS II and contrasted them to the actual mortality in a surgical intensive care unit (SICU). Data for 1665 patients admitted to the SICU between July 1994 and August 1997 were entered into an SICU computerized database. From recorded demographic, hemodynamic, and laboratory data, APACHE II and SAPS II scores were obtained with corresponding predicted mortalities. Patients were stratified by age into categories of less than and greater than 65 years old. Predicted mortalities by APACHE II and SAPS II were compared for each group. An additional analysis included a comparison of survivors and nonsurvivors. There was no significant difference in predicted mortality between APACHE II and SAPS II in any of the groups. Actual mortality was 30 of 486 (6.2%) in patients less than 65 years of age and 73 of 1179 (6.2%) in patients 65 years of age or greater. The APACHE II and SAPS II predicted mortalities (mean +/- SD) for patients less than 65 years of age were 10.5% +/- 10.6% and 10.9% +/- 13.3%, respectively (P > .05). The APACHE II and SAPS II predicted mortalities in patients 65 years of age or greater were 19.1% +/- 17.8% and 18.7% +/- 21.0%, respectively (P > .05). Similarly, when patients were stratified by survival status, no significant difference was present between groups. However, in individual patients, a difference between APACHE II and SAPS II scores was often present. We conclude that although disparities between APACHE II and SAPS II predicted mortalities in individual patients may be significant, APACHE II and SAPS II have similar predictive value in a large SICU patient population. However, both APACHE II and SAPS II systems overestimate mortality in SICU patients. Based on our results, we conclude that either system can be used to measure quality of care in the SICU; however, neither system can be reliably applied to a single patient.

Prognostic factors in intensive care

Predicting the outcome of critical illness remains an evolving art despite many recent advances. This review article describes the tools currently employed, appraising each in turn. The subject is viewed from the perspective that physiological reserve and inflammatory response are the essential elements in assessing prognosis in patients with multi-organ dysfunction/failure, the most commonly encountered syndrome in intensive care practice.

Outcomes of trauma patients who survive prolonged lengths of stay in the intensive care unit

OBJECTIVE

To examine a subgroup of severely injured patients spending > or = 3 weeks in the intensive care unit (ICU) and to determine their disposition and eventual functional outcome.

METHODS

A retrospective review of our trauma registry and medical records over a 7-year period (January of 1991 to December of 1997) identified 115 patients with ICU length of stay (LOS) > or = 3 weeks. Variables selected included age, length of stay, injury severity score, injuries, disposition, and charges. Functional independence measures (FIM) were obtained in patients requiring inpatient rehabilitation and a written questionnaire (Rand 36-item Health Survey) was mailed to all patients alive at discharge.

RESULTS

Mean ICU length of stay of the 115 patients was 36 days (range, 21-106 days); mean age, 49 years (range, 4-89 years); 73 patients (63%) were males, 42 patients (37%) were females. Overall mortality was 22% (n = 25). The remaining 90 patients survived to discharge with the following disposition: rehabilitation facility 60% (n = 54), home with temporary disability 22% (n = 20), nursing home 8% (n = 7), home with permanent disability 4% (n = 4), transferred 6% (n = 5). Mean hospital charge was $193,000 (range, $77,000-$528,000). No variable or combination could predict outcome except age. Elderly patients (age > or = 75, n = 24) had an overall mortality of 42% (n = 10). Eight of 14 survivors fulfilled admission criteria and entered our rehabilitation facility. The remaining six elderly patients either went to nursing homes or were permanently disabled. Complete FIM scores were available on 47 of 54 patients who went to rehabilitation facility. The mean rehabilitation facility admission FIM score was 52, indicating either complete dependence or the need for moderate assistance. After they had remained at the rehabilitation facility for a mean of 48 days (range, 7-278 days), patients' FIM scores improved to a mean of 86, signifying minimal contact assistance or supervision only. Three-month follow-up FIM scores continued to improve to a mean of 101, a score denoting complete independence. Elderly patients within the rehabilitation facility fared as well as the younger group. For the Rand-36 survey, 47 of 90 patients or family members were contacted. Twelve patients died since discharge, leaving 35 patients to complete the survey. Despite excellent FIM scores, overall mean health was only fair to good, with limitations to activity and lack of energy cited as the main problems.

CONCLUSION

Despite tremendous resource utilization, the majority of trauma patients with prolonged ICU stays can eventually return to varying degrees of functional daily living and independence, but not to preinjury levels. A subgroup of severely injured elderly patients had a significantly higher mortality rate. However, elderly survivors that entered our rehabilitation facility fared as well as the younger patients.

Differences in mortality between elderly and younger adult trauma patients: geriatric status increases risk of delayed death

BACKGROUND

Elderly patients suffer higher mortality rates after trauma than younger patients. This increased mortality is attributable to age, preexisting disease, and complications as well as injury severity.

METHODS

Records from 5,139 adult patients from a Level I trauma center were retrospectively reviewed. Injury Severity Score (ISS), Revised Trauma Score (RTS), early mortality (<24 hours), and late mortality (>24 hours) were determined for elderly (> or =65 years) and younger (16-64 years) patients. Preexisting diseases and complications were identified by International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis coding.

RESULTS

Mortality in elderly patients was twice that in younger patients despite equivalent injury severity (p < 0.001), and elderly patients were more likely to suffer later death than younger patients (p < 0.005). The prevalence of preexisting disease was greater in the elderly, as was the incidence of complications. Using logistic regression, ISS, RTS, preexisting cardiovascular or liver disease, the development of cardiac, renal, or infectious complications, and geriatric status were all independently predictive of late mortality (p < 0.05).

CONCLUSION

Elderly trauma patients more frequently suffer late mortality than younger patients because of the combination of injury and increased preexisting disease and complications after injury. Aggressive treatment of the elderly trauma patient is warranted; however, in the face of significant preexisting disease or complications, survival is less likely. Predictive models of survival can be developed, taking into account preexisting disease and complications as well as admission parameters such as age, ISS, and RTS, and specific risk of mortality quantitated.