External validation of Acute Physiology and Chronic Health Evaluation IV in Dutch intensive care units and comparison with Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II

Comparative performance of intensive care mortality prediction models based on manually curated versus automatically extracted electronic health record data

INTRODUCTION

Benchmarking intensive care units for audit and feedback is frequently based on comparing actual mortality versus predicted mortality. Traditionally, mortality prediction models rely on a limited number of input variables and significant manual data entry and curation. Using automatically extracted electronic health record data may be a promising alternative. However, adequate data on comparative performance between these approaches is currently lacking.

METHODS

The AmsterdamUMCdb intensive care database was used to construct a baseline APACHE IV in-hospital mortality model based on data typically available through manual data curation. Subsequently, new in-hospital mortality models were systematically developed and evaluated. New models differed with respect to the extent of automatic variable extraction, classification method, recalibration usage and the size of collection window.

RESULTS

A total of 13 models were developed based on data from 5,077 admissions divided into a train (80%) and test (20%) cohort. Adding variables or extending collection windows only marginally improved discrimination and calibration. An XGBoost model using only automatically extracted variables, and therefore no acute or chronic diagnoses, was the best performing automated model with an AUC of 0.89 and a Brier score of 0.10.

DISCUSSION

Performance of intensive care mortality prediction models based on manually curated versus automatically extracted electronic health record data is similar. Importantly, our results suggest that variables typically requiring manual curation, such as diagnosis at admission and comorbidities, may not be necessary for accurate mortality prediction. These proof-of-concept results require replication using multi-centre data.

Predicting 30-day mortality in intensive care unit patients with ischaemic stroke or intracerebral haemorrhage

BACKGROUND

Stroke patients admitted to an intensive care unit (ICU) follow a particular survival pattern with a high short-term mortality, but if they survive the first 30 days, a relatively favourable subsequent survival is observed.

OBJECTIVES

The development and validation of two prognostic models predicting 30-day mortality for ICU patients with ischaemic stroke and for ICU patients with intracerebral haemorrhage (ICH), analysed separately, based on parameters readily available within 24 h after ICU admission, and with comparison with the existing Acute Physiology and Chronic Health Evaluation IV (APACHE-IV) model.

DESIGN

Observational cohort study.

SETTING

All 85 ICUs participating in the Dutch National Intensive Care Evaluation database.

PATIENTS

All adult patients with ischaemic stroke or ICH admitted to these ICUs between 2010 and 2019.

MAIN OUTCOME MEASURES

Models were developed using logistic regressions and compared with the existing APACHE-IV model. Predictive performance was assessed using ROC curves, calibration plots and Brier scores.

RESULTS

We enrolled 14 303 patients with stroke admitted to ICU: 8422 with ischaemic stroke and 5881 with ICH. Thirty-day mortality was 27% in patients with ischaemic stroke and 41% in patients with ICH. Important factors predicting 30-day mortality in both ischaemic stroke and ICH were age, lowest Glasgow Coma Scale (GCS) score in the first 24 h, acute physiological disturbance (measured using the Acute Physiology Score) and the application of mechanical ventilation. Both prognostic models showed high discrimination with an AUC 0.85 [95% confidence interval (CI), 0.84 to 0.87] for patients with ischaemic stroke and 0.85 (0.83 to 0.86) in ICH. Calibration plots and Brier scores indicated an overall good fit and good predictive performance. The APACHE-IV model predicting 30-day mortality showed similar performance with an AUC of 0.86 (95% CI, 0.85 to 0.87) in ischaemic stroke and 0.87 (0.86 to 0.89) in ICH.

CONCLUSION

We developed and validated two prognostic models for patients with ischaemic stroke and ICH separately with a high discrimination and good calibration to predict 30-day mortality within 24 h after ICU admission.

TRIAL REGISTRATION

Trial registration: Dutch Trial Registry ( https://www.trialregister.nl/ ); identifier: NTR7438.

Predictive Modeling Using Artificial Intelligence and Machine Learning Algorithms on Electronic Health Record Data: Advantages and Challenges

The rapid adoption of electronic health record (EHR) systems in US hospitals from 2008 to 2014 produced novel data elements for analysis. Concurrent innovations in computing architecture and machine learning (ML) algorithms have made rapid consumption of health data feasible and a powerful engine for clinical innovation. In critical care research, the net convergence of these trends has resulted in an exponential increase in outcome prediction research. In the following article, we explore the history of outcome prediction in the intensive care unit (ICU), the growing use of EHR data, and the rise of artificial intelligence and ML (AI) in critical care.

Comparison of APACHE II and APACHE IV score as predictors of mortality in patients with septic shock in intensive care unit: A prospective observational study

Background and Aims

Prediction of outcome in intensive care unit (ICU) patients is of imperative importance. Our aim was to assess and compare the performance of Acute Physiology and Chronic Health Evaluation (APACHE) II and APACHE IV scores in predicting mortality in adult patients suffering from septic shock admitted to our ICU.

Material and Methods

This was a prospective observational study conducted in a 14-bedded medical ICU of a tertiary care center from January 2019 to March 2020; 128 patients suffering from septic shock were included and APACHE II and IV scores were calculated. We also calculated the predicted and actual mortality rates and standardized mortality ratios. The receiver operating characteristic curves were used to assess discrimination.

Results

Out of the 128 patients, 63 patients (49.21%) died. The mean (± standard deviation) admission APACHE II score was 16.7 ± 5.53, while the mean APACHE IV score was 67.25 ± 25.99. The non-survivors had significantly higher APACHE II and IV scores when compared to those who survived (P < 0.001). APACHE II had a slightly better discriminative power (with the area under the Receiver operating characteristic (ROC) curve of 0.78) than APACHE IV (with the area under the ROC curve of 0.74). The mean predicted mortality rate (PMR) of the patient population calculated on the basis of the APACHE II scoring system was 22.46 ± 15.76, and the mean PMR calculated as per the APACHE IV scoring system was 11.64 ± 15.59.

Conclusion

Both APACHE II and APACHE IV underestimated mortality in septic shock patients. Both APACHE II and APACHE IV were comparable in differentiating survivors from non-survivors. However, there was a good correlation between the two models.

The Global Open Source Severity of Illness Score (GOSSIS)

OBJECTIVES

To develop and demonstrate the feasibility of a Global Open Source Severity of Illness Score (GOSSIS)-1 for critical care patients, which generalizes across healthcare systems and countries.

DESIGN

A merger of several critical care multicenter cohorts derived from registry and electronic health record data. Data were split into training (70%) and test (30%) sets, using each set exclusively for development and evaluation, respectively. Missing data were imputed when not available.

SETTING/PATIENTS

Two large multicenter datasets from Australia and New Zealand (Australian and New Zealand Intensive Care Society Adult Patient Database [ANZICS-APD]) and the United States (eICU Collaborative Research Database [eICU-CRD]) representing 249,229 and 131,051 patients, respectively. ANZICS-APD and eICU-CRD contributed data from 162 and 204 hospitals, respectively. The cohort included all ICU admissions discharged in 2014-2015, excluding patients less than 16 years old, admissions less than 6 hours, and those with a previous ICU stay.

INTERVENTIONS

Not applicable.

MEASUREMENTS AND MAIN RESULTS

GOSSIS-1 uses data collected during the ICU stay's first 24 hours, including extrema values for vital signs and laboratory results, admission diagnosis, the Glasgow Coma Scale, chronic comorbidities, and admission/demographic variables. The datasets showed significant variation in admission-related variables, case-mix, and average physiologic state. Despite this heterogeneity, test set discrimination of GOSSIS-1 was high (area under the receiver operator characteristic curve [AUROC], 0.918; 95% CI, 0.915-0.921) and calibration was excellent (standardized mortality ratio [SMR], 0.986; 95% CI, 0.966-1.005; Brier score, 0.050). Performance was held within ANZICS-APD (AUROC, 0.925; SMR, 0.982; Brier score, 0.047) and eICU-CRD (AUROC, 0.904; SMR, 0.992; Brier score, 0.055). Compared with GOSSIS-1, Acute Physiology and Chronic Health Evaluation (APACHE)-IIIj (ANZICS-APD) and APACHE-IVa (eICU-CRD), had worse discrimination with AUROCs of 0.904 and 0.869, and poorer calibration with SMRs of 0.594 and 0.770, and Brier scores of 0.059 and 0.063, respectively.

CONCLUSIONS

GOSSIS-1 is a modern, free, open-source inhospital mortality prediction algorithm for critical care patients, achieving excellent discrimination and calibration across three countries.

Factors associated with a persistent delirium in the intensive care unit: A retrospective cohort study

PURPOSE

To explore differences between ICU patients with persistent delirium (PD), non-persistent delirium (NPD) and no delirium (ND), and to determine factors associated with PD.

MATERIALS AND METHODS

Retrospective cohort study including all ICU adults admitted for ≥12 h (January 2015-February 2020), assessable for delirium and followed during their entire hospitalization. PD was defined as ≥14 days of delirium. Factors associated with PD were determined using multivariable logistic regression analysis.

RESULTS

Out of 10,295 patients, 3138 (30.5%) had delirium, and 284 (2.8%) had PD. As compared to NPD (n = 2854, 27.7%) and ND (n = 7157, 69.5%), PD patients were older, sicker, more physically restrained, longer comatose and mechanically ventilated, had a longer ICU and hospital stay, more ICU readmissions and a higher mortality rate. Factors associated with PD were age (adjusted odds ratio [aOR] 1.03; 95% confidence interval [CI] 1.02-1.04); emergency surgical (aOR 1.84; 95%CI 1.26-2.68) and medical (aOR 1.57; 95%CI 1.12-2.21) referral, mean Sequential Organ Failure Assessment (SOFA) score before delirium onset (aOR 1.18; 95%CI 1.13-1.24) and use of physical restraints (aOR 5.02; 95%CI 3.09-8.15).

CONCLUSIONS

Patients with persistent delirium differ in several characteristics and had worse short-term outcomes. Physical restraints were the most strongly associated with PD.

Validation of the Acute Physiology and Chronic Health Evaluation (APACHE) II and IV Score in COVID-19 Patients

Background

Severity scoring systems are inherent to ICU practice for multiple purposes. Acute Physiology and Chronic Health Evaluation (APACHE) scoring systems are designed for ICU mortality prediction. This study aims to validate APACHE IV in COVID-19 patients admitted to the ICU.

Methods

All COVID-19 patients admitted to the ICU between March 13, 2020, and October 17, 2020, were retrospectively analyzed. APACHE II and APACHE IV scores as well as SOFA scores were calculated within 24 hours after admission. Discrimination for mortality of all three scoring systems was assessed by receiver operating characteristic curves. Youden index was determined for the scoring system with the best discriminative performance. The Hosmer-Lemeshow goodness-of-fit test was used to assess calibration. All analyses were performed for both the overall population as in a subgroup treated with anti-Xa adjusted dosages of LMWHs.

Results

116 patients were admitted to our ICU during the study period. 13 were excluded for various reasons, leaving 103 patients in the statistical analysis of the overall population. 57 patients were treated with anti-Xa adjusted prophylactic dosages of LMWH and were supplementary analyzed in a subgroup analysis. APACHE IV had the best discriminative power of the three scoring systems, both in the overall population (APACHE IV ROC AUC 0.67 vs. APACHE II ROC AUC 0.63) as in the subgroup (APACHE IV ROC AUC 0.82 vs. APACHE II ROC AUC 0.7). This model exhibits good calibration. Hosmer-Lemeshow p values for APACHE IV were 0.9234 for the overall population and 0.8017 for the subgroup. Calibration p values of the APACHE II score were 0.1394 and 0.6475 for the overall versus subgroup, respectively.

Conclusions

APACHE IV provided the best discrimination and calibration of the considered scoring systems in critically ill COVID-19 patients, both in the overall group and in the subgroup with anti-Xa adjusted LMWH doses. Only in the subgroup analysis, discriminative abilities of APACHE IV were very good. This trial is registered with NCT04713852.

Long-Term Mortality Among ICU Patients With Stroke Compared With Other Critically Ill Patients

OBJECTIVES

Assessment of all-cause mortality of intracerebral hemorrhage and ischemic stroke patients admitted to the ICU and comparison to the mortality of other critically ill ICU patients classified into six other diagnostic subgroups and the general Dutch population.

DESIGN

Observational cohort study.

SETTING

All ICUs participating in the Dutch National Intensive Care Evaluation database.

PATIENTS

All adult patients admitted to these ICUs between 2010 and 2015; patients were followed until February 2017.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Of all 370,386 included ICU patients, 7,046 (1.9%) were stroke patients, 4,072 with ischemic stroke, and 2,974 with intracerebral hemorrhage. Short-term mortality in ICU-admitted stroke patients was high with 30 days mortality of 31% in ischemic stroke and 42% in intracerebral hemorrhage. In the longer term, the survival curve gradient among ischemic stroke and intracerebral hemorrhage patients stabilized. The gradual alteration of mortality risk after ICU admission was assessed using left-truncation with increasing minimum survival period. ICU-admitted stroke patients who survive the first 30 days after suffering from a stroke had a favorable subsequent survival compared with other diseases necessitating ICU admission such as patients admitted due to sepsis or severe community-acquired pneumonia. After having survived the first 3 months after ICU admission, multivariable Cox regression analyses showed that case-mix adjusted hazard ratios during the follow-up period of up to 3 years were lower in ischemic stroke compared with sepsis (adjusted hazard ratio, 1.21; 95% CI, 1.06-1.36) and severe community-acquired pneumonia (adjusted hazard ratio, 1.57; 95% CI, 1.39-1.77) and in intracerebral hemorrhage patients compared with these groups (adjusted hazard ratio, 1.14; 95% CI, 0.98-1.33 and adjusted hazard ratio, 1.49; 95% CI, 1.28-1.73).

CONCLUSIONS

Stroke patients who need intensive care treatment have a high short-term mortality risk, but this alters favorably with increasing duration of survival time after ICU admission in patients with both ischemic stroke and intracerebral hemorrhage, especially compared with other populations of critically ill patients such as sepsis or severe community-acquired pneumonia patients.

Characteristics and outcomes of patients admitted to adult intensive care units in Hong Kong: a population retrospective cohort study from 2008 to 2018

Background

Globally, mortality rates of patients admitted to the intensive care unit (ICU) have decreased over the last two decades. However, evaluations of the temporal trends in the characteristics and outcomes of ICU patients in Asia are limited. The objective of this study was to describe the characteristics and risk adjusted outcomes of all patients admitted to publicly funded ICUs in Hong Kong over a 11-year period. The secondary objective was to validate the predictive performance of Acute Physiology And Chronic Health Evaluation (APACHE) IV for ICU patients in Hong Kong.

Methods

This was an 11-year population-based retrospective study of all patients admitted to adult general (mixed medical-surgical) intensive care units in Hong Kong public hospitals. ICU patients were identified from a population electronic health record database. Prospectively collected APACHE IV data and clinical outcomes were analysed.

Results

From 1 April 2008 to 31 March 2019, there were a total of 133,858 adult ICU admissions in Hong Kong public hospitals. During this time, annual ICU admissions increased from 11,267 to 14,068, whilst hospital mortality decreased from 19.7 to 14.3%. The APACHE IV standard mortality ratio (SMR) decreased from 0.81 to 0.65 during the same period. Linear regression demonstrated that APACHE IV SMR changed by − 0.15 (95% CI − 0.18 to − 0.11) per year (Pearson’s R = − 0.951, p < 0.001). Observed median ICU length of stay was shorter than that predicted by APACHE IV (1.98 vs. 4.77, p < 0.001). C-statistic for APACHE IV to predict hospital mortality was 0.889 (95% CI 0.887 to 0.891) whilst calibration was limited (Hosmer–Lemeshow test p < 0.001).

Conclusions

Despite relatively modest per capita health expenditure, and a small number of ICU beds per population, Hong Kong consistently provides a high-quality and efficient ICU service. Number of adult ICU admissions has increased, whilst adjusted mortality has decreased over the last decade. Although APACHE IV had good discrimination for hospital mortality, it overestimated hospital mortality of critically ill patients in Hong Kong.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40560-020-00513-9.

Peripherally inserted central catheter placement in a multidisciplinary intensive care unit: A preliminary study demonstrating safety and procedural time in critically ill subjects

OBJECTIVE

Peripherally inserted central catheters are a popular means of obtaining central venous access in critically ill patients. However, there is limited data regarding the rapidity of the peripherally inserted central catheter procedure in the presence of acute illness or obesity, both of which may impede central venous catheter placement. We aimed to determine the feasibility, safety, and duration of peripherally inserted central catheter placement in critically ill patients, including obese patients and patients in shock.

METHODS

This retrospective cohort study was performed using data on 55 peripherally inserted central catheters placed in a 30-bed multidisciplinary intensive care unit in Mayo Clinic Hospital, Phoenix, Arizona. Information on the time required to complete each step of the peripherally inserted central catheter procedure, associated complications, and patient characteristics was obtained from a prospectively assembled internal quality assurance database created through random convenience sampling.

RESULTS

The Median Procedure Time, beginning with the first needle puncture and ending when the procedure is complete, was 14 (interquartile range: 9-20) min. Neither critical illness nor obesity resulted in a statistically significant increase in the time required to complete the peripherally inserted central catheter procedure. Three (5.5%) minor complications were observed.

CONCLUSION

Critical illness and obesity do not delay the acquisition of vascular access when placing a peripherally inserted central catheter. Concerns of delayed vascular access in critically ill patients should not deter a physician from selecting a peripherally inserted central catheter to provide vascular access when it would otherwise be appropriate.

Study protocol and pilot results of an observational cohort study evaluating effect of red blood cell transfusion on oxygenation and mitochondrial oxygen tension in critically ill patients with anaemia: the INsufficient Oxygenation in the Intensive Care Unit (INOX ICU-2) study

INTRODUCTION

The recently developed protoporphyrin IX-triple state lifetime technique measures mitochondrial oxygenation tension (mitoPO2) in vivo at the bedside. MitoPO2might be an early indicator of oxygen disbalance in cells of critically ill patients and therefore may support clinical decisions regarding red blood cell (RBC) transfusion. We aim to investigate the effect of RBC transfusion and the associated changes in haemoglobin concentration on mitoPO2 and other physiological measures of tissue oxygenation and oxygen balance in critically ill patients with anaemia. We present the protocol and pilot results for this study.

METHODS AND ANALYSIS

We perform a prospective multicentre observational study in three mixed intensive care units in the Netherlands with critically ill patients with anaemia in whom an RBC transfusion is planned. The skin of the anterior chest wall of the patients is primed with a 5-aminolevulinic acid patch for 4 hours for induction of mitochondrial protoporphyrin-IX to enable measurements of mitoPO2, which is done with the COMET monitoring device. At multiple predefined moments, before and after RBC transfusion, we assess mitoPO2 and other physiological parameters of oxygen balance and tissue oxygenation. Descriptive statistics will be used to describe the data. A linear mixed-effect model will be used to study the association between RBC transfusion and mitoPO2 and other traditional parameters of oxygenation, oxygen delivery and oxygen balance. Missing data will be imputed using multiple imputation methods.

ETHICS AND DISSEMINATION

The institutional ethics committee of each participating centre approved the study (reference P16.303), which will be conducted according to the 1964 Helsinki declaration and its later amendments. The results will be submitted for publication in peer-reviewed journals and presented at scientific conferences.

TRIAL REGISTRATION NUMBER

NCT03092297.

Adjusting for Disease Severity Across ICUs in Multicenter Studies

Supplemental Digital Content is available in the text.

Objectives:

To compare methods to adjust for confounding by disease severity during multicenter intervention studies in ICU, when different disease severity measures are collected across centers.

Design:

In silico simulation study using national registry data.

Setting:

Twenty mixed ICUs in The Netherlands.

Subjects:

Fifty-five–thousand six-hundred fifty-five ICU admissions between January 1, 2011, and January 1, 2016.

Interventions:

None.

Measurements and Main Results:

To mimic an intervention study with confounding, a fictitious treatment variable was simulated whose effect on the outcome was confounded by Acute Physiology and Chronic Health Evaluation IV predicted mortality (a common measure for disease severity). Diverse, realistic scenarios were investigated where the availability of disease severity measures (i.e., Acute Physiology and Chronic Health Evaluation IV, Acute Physiology and Chronic Health Evaluation II, and Simplified Acute Physiology Score II scores) varied across centers. For each scenario, eight different methods to adjust for confounding were used to obtain an estimate of the (fictitious) treatment effect. These were compared in terms of relative (%) and absolute (odds ratio) bias to a reference scenario where the treatment effect was estimated following correction for the Acute Physiology and Chronic Health Evaluation IV scores from all centers. Complete neglect of differences in disease severity measures across centers resulted in bias ranging from 10.2% to 173.6% across scenarios, and no commonly used methodology—such as two-stage modeling or score standardization—was able to effectively eliminate bias. In scenarios where some of the included centers had (only) Acute Physiology and Chronic Health Evaluation II or Simplified Acute Physiology Score II available (and not Acute Physiology and Chronic Health Evaluation IV), either restriction of the analysis to Acute Physiology and Chronic Health Evaluation IV centers alone or multiple imputation of Acute Physiology and Chronic Health Evaluation IV scores resulted in the least amount of relative bias (0.0% and 5.1% for Acute Physiology and Chronic Health Evaluation II, respectively, and 0.0% and 4.6% for Simplified Acute Physiology Score II, respectively). In scenarios where some centers used Acute Physiology and Chronic Health Evaluation II, regression calibration yielded low relative bias too (relative bias, 12.4%); this was not true if these same centers only had Simplified Acute Physiology Score II available (relative bias, 54.8%).

Conclusions:

When different disease severity measures are available across centers, the performance of various methods to control for confounding by disease severity may show important differences. When planning multicenter studies, researchers should make contingency plans to limit the use of or properly incorporate different disease measures across centers in the statistical analysis.

Clinical and Epidemiological Characteristics of Severe Acute Adult Poisoning Cases in Martinique: Implicated Toxic Exposures and Their Outcomes

The epidemiology of severe acute poisonings in the French overseas departments of the Americas remains poorly reported. The main objective of this study was to determine the epidemiology and characteristics of severe acutely poisoned adult patients.

METHODS

A retrospective descriptive study was conducted from 1 January 2000 to 31 December 2010 in severely poisoned patients presenting to the emergency department (ED) of the University Hospital of Martinique, and the general public hospitals of Lamentin and Trinité.

RESULTS

During the study period, 291 patients were admitted for severe poisoning, giving an incidence rate of 7.7 severe cases/100,000 inhabitants. The mean age was 46 ± 19 years and 166 (57%) were male. Psychiatric disorders were recorded in 143 (49.8%) patients. Simplified Acute Psychological Score (SAPS II) at admission was 39 ± 23 points and Poisoning Severity Score (PSS) was 2.7 ± 0.8 points. Death was recorded in 30 (10.3%) patients and hospital length of stay was 6 ± 7 days. The mode of intoxication was intentional self-poisoning in 87% of cases and drug overdose was recorded in 13% of cases. The toxic agent involved was a therapeutic drug in 58% and a chemical product in 52% of cases. The predominant clinical manifestations were respiratory failure (59%), hemodynamic failure (27%), neurologic failure (45%), gastrointestinal manifestations (27%), and renal failure (11%). Polypnea, shock, ventricular fibrillation or tachycardia, and gastro-intestinal disorders were the main symptoms associated with death. The main biological abnormalities associated with death in our patients were metabolic acidosis, hypokalemia, hyperlactatemia, hypocalcemia, renal injury, rhabdomyolysis, increased aspartate aminotransferases, and thrombocytopenia. Extracorporal membrane oxygenation (ECMO) was used in three patients and specific antidotes were used in 21% of patients.

CONCLUSIONS

Acute poisonings remain a major public health problem in Martinique with different epidemiological characteristics to those in mainland France, with a high incidence of poisoning by rural and household toxins.

Mortality prediction by SOFA score in ICU-patients after cardiac surgery; comparison with traditional prognostic-models

Background

There are many prognostic models and scoring systems in use to predict mortality in ICU patients. The only general ICU scoring system developed and validated for patients after cardiac surgery is the APACHE-IV model. This is, however, a labor-intensive scoring system requiring a lot of data and could therefore be prone to error. The SOFA score on the other hand is a simpler system, has been widely used in ICUs and could be a good alternative.

The goal of the study was to compare the SOFA score with the APACHE-IV and other ICU prediction models.

Methods

We investigated, in a large cohort of cardiac surgery patients admitted to Dutch ICUs, how well the SOFA score from the first 24 h after admission, predict hospital and ICU mortality in comparison with other recalibrated general ICU scoring systems. Measures of discrimination, accuracy, and calibration (area under the receiver operating characteristic curve (AUC), Brier score, R², and Ĉ-statistic) were calculated using bootstrapping. The cohort consisted of 36,632 Patients from the Dutch National Intensive Care Evaluation (NICE) registry having had a cardiac surgery procedure for which ICU admission was necessary between January 1st, 2006 and June 31st, 2018.

Results

Discrimination of the SOFA-, APACHE-IV-, APACHE-II-, SAPS-II-, MPM₂₄-II - models to predict hospital mortality was good with an AUC of respectively: 0.809, 0.851, 0.830, 0.850, 0.801. Discrimination of the SOFA-, APACHE-IV-, APACHE-II-, SAPS-II-, MPM₂₄-II - models to predict ICU mortality was slightly better with AUCs of respectively: 0.809, 0.906, 0.892, 0.919, 0.862. Calibration of the models was generally poor.

Conclusion

Although the SOFA score had a good discriminatory power for hospital- and ICU mortality the discriminatory power of the APACHE-IV and SAPS-II was better. The SOFA score should not be preferred as mortality prediction model above traditional prognostic ICU-models.

A nationwide overview of 1-year mortality in cardiac arrest patients admitted to intensive care units in the Netherlands between 2010 and 2016

AIM

Worldwide, cardiac arrest (CA) remains a major cause of death. Most post-CA patients are admitted to the intensive care unit (ICU). The aim of this study is to describe mortality rates and possible changes in mortality rates in patients with CA admitted to the ICU in the Netherlands between 2010 and 2016.

METHODS

In this study, we included all adult CA patients registered in the National Intensive Care Evaluation (NICE) registry who were admitted to ICUs in the Netherlands between 2010 and 2016. The primary outcome was 1-year mortality which was analysed by Cox regression. The secondary outcomes were ICU mortality and hospital mortality. Hospital mortality was analysed by binary logistic regression analysis. Patients were stratified by whether they experienced in-hospital cardiac arrest (IHCA) or out-of-hospital cardiac arrest (OHCA). Finally, the outcome over calendar time was assessed for both groups.

RESULTS

We included 26,056 CA patients: 10,618 (40.8%) IHCA patients and 14,482 (55.6%) OHCA patients. The 1-year mortality rate was 57.5%: 59% for IHCA and 56.4% for OHCA, p < 0.01. This mortality rate remained stable between 2010 and 2016 for IHCA (p = 0.31) and declined for OHCA patients (p = 0.01). The hospital mortality rate was 50.3%: 50.5% for IHCA and 50.2% for OHCA, p = 0.66. This mortality rate remained stable between 2010-2016 for IHCA (p = 0.21) and decreased for OHCA patients (p < 0.01). An additional analysis with calendar year as a continuous variable showed a mortality decline of 1.56% per calendar year for 1-year mortality.

CONCLUSION

This nationwide registry cohort study reported a 57.5% 1-year mortality rate for CA patients admitted to the ICU between 2010 and 2016. We reported a decline in 1-year mortality for OHCA patients in these years.

Validation the performance of New York Sepsis Severity Score compared with Sepsis Severity Score in predicting hospital mortality among sepsis patients

PURPOSE

The aim of this study was to compare the performance of the New York Sepsis Severity Score (NYSSS) with the Sepsis Severity Score (SSS) and Acute Physiology and Chronic Health Evaluation and Simplified Acute Physiology Scores for predicting mortality in sepsis patients.

METHOD

A retrospective analysis was conducted in the intensive care unit. The primary outcome was in-hospital mortality.

RESULTS

Overall 1680 sepsis patients were enrolled. The hospital mortality rate was 44.4%. The NYSSS underestimated actual mortality with standard mortality ratio (SMR) of 1.28 (95%CI 1.19-1.38). However, the SSS slightly overestimated the actual mortality with an SMR of 0.94 (0.88-1.01). The NYSSS had moderate discrimination with an AUC of 0.772 (0.750-0.794), in contrast to the SSS which had good discrimination with an AUC of 0.889 (0.873-0.904). The AUC of the SSS was statistically higher than that of the NYSSS. The AUCs of both the NYSSS and SSS were significantly lower than other standard severity scores. The calibrations for all severity scores were poor. The SSS had better overall performance than the NYSSS (Brier score 0.149 and 0.201, respectively).

CONCLUSION

The SSS had better discrimination and overall performance than the NYSSS. However, both sepsis severity scores were poorly calibrated.

Performance of the Acute Physiology and Chronic Health Evaluation II (APACHE II) in the prediction of hospital mortality in a mixed ICU in Singapore

Background:

The Acute Physiology and Chronic Health Evaluation II (APACHE II) is used to quantify disease severity and hospital mortality risk in critically ill patients. It is widely used in intensive care units (ICUs) in Singapore, but its prognostic validity remains questionable as it has not been thoroughly assessed by established statistical methods.

Objectives:

This study aimed to: (a) evaluate the discrimination and calibration accuracy of the APACHE II in the prediction of hospital mortality in a mixed ICU, and (b) customise the APACHE II in an effort to maximise its prognostic performance.

Methods:

A prospective cohort study was conducted and all adult patients with >24 h of ICU admission in a tertiary care institution in Singapore were included. The outcome measure was hospital mortality, and all patients were followed-up until hospital discharge or death for up to one year after ICU admission.

Results:

There were 503 patients, and their mean (SD) age and APACHE II score were 61.2 (15.8) years and 24.5 (8.2), respectively. Hospital mortality was 31%, and no patients were lost to follow-up. The APACHE II has good discrimination (receiver operating characteristic: 0.76) but poor calibration (Hosmer–Lemeshow C test: <0.001). Customisation did not significantly improve calibration accuracy.

Conclusions:

The APACHE II and its customised version should not be used in the local setting as they both have poor calibration. There is an urgent need for larger studies to perform second-level customisation or to develop a new prognostic model tailored to the Singapore critical care setting.

Feasibility and Accuracy of a Nonmedical Research Person in Assimilation and Calculation of Acute Physiologic Assessment and Chronic Health Evaluation Scores in an Indian Intensive Care Unit

Background

The Physiologic Assessment and Chronic Health Evaluation (APACHE) score assimilation and calculation, as well as other demographic data collection, is inherent to research and nonresearch related needs of intensive care. There may be a role for well-trained nonmedical personnel to collect this vital material to enhance research and the standard of care in the Intensive Care Units (ICUs) in countries that are poorly funded and resourced in terms of medical personnel.

Aims

The aim of this study is to verify the interrater reliability of a trained nonmedical personnel and ICU trainee in the collection and calculation APACHE scores.

Materials and Methods

In a prospective study, two raters who were blinded, one a trained nonmedical ward clerk and another an ICU trainee, assimilated data and calculated APACHE scores for 60 consecutive patients admitted to two tertiary mixed ICUs (with a total of 19 beds). Primary outcomes were to assess interrater and interclass correlation as well as the agreement of scores between the two raters.

Results

There was an excellent correlation of APACHE scores (Kappa coefficient of 0.92) and Bland-Altman plot depicted overall good agreement with low bias between raters.

Conclusions

A well-trained and supervised nonmedical research person can assimilate and calculate APACHE II scores with good agreement with an ICU trainee. This may help in deriving data from medically understaffed ICUs in India, thus promoting much-needed research from such ICUs.

A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran

Background: Clinical assessment of disease severity is an important part of medical practice for prediction of mortality and morbidity in Intensive Care Unit (ICU). A disease severity scoring system can be used as guidance for clinicians for objective assessment of disease outcomes and estimation of the chance of recovery. This study aimed to evaluate the hypothesis that the mortality and length of stay in emergency ICUs predicted by APACHE-IV is different to the real rates of mortality and length of stay observed in our emergency ICU in Iran.

Methods: This was a retrospective cohort study conducted on the data of 839 consecutive patients admitted to the emergency ICU of Nemazi Hospital, Shiraz, Iran, during 2012-2015. The relevant variables were used to calculate APACHE-IV.  Length of stay and death or discharge, Glasgow coma score, and acute physiology score were also evaluated. Moreover, the accuracy of APACHE-IV for mortality was assessed using area under the Receiver Operator Characteristic (ROC) curve.

Results: Of the studied patients, 157 died and 682 were discharged (non-survivors and survivors, respectively). The length of stay in the ICU was 10.98±14.60, 10.22 ± 14.21 and 14.30±15.80 days for all patients, survivors, and non-survivors, respectively. The results showed that APACHE-IV model underestimated length of stay in our emergency ICU (p<0.001). In addition, the overall observed mortality was 17.8%, while the predicted mortality by APACHE-IV model was 21%. Therefore, there was an overestimation of predicted mortality by APACHE-IV model, with an absolute difference of 3.2% (p=0.036).

Conclusion: The findings showed that APACHE-IV was a poor predictor of length of stay and mortality rate in emergency ICU. Therefore, specific models based on big sample sizes of Iranian patients are required to improve accuracy of predictions.

Assessing contemporary intensive care unit outcome: development and validation of the Australian and New Zealand Risk of Death admission model

The Australian and New Zealand Risk of Death (ANZROD) model currently used for benchmarking intensive care units (ICUs) in Australia and New Zealand utilises physiological data collected up to 24 hours after ICU admission to estimate the risk of hospital mortality. This study aimed to develop the Australian and New Zealand Risk of Death admission (ANZROD₀) model to predict hospital mortality using data available at presentation to ICU and compare its performance with the ANZROD in Australian and New Zealand hospitals. Data pertaining to all ICU admissions between 1 January 2006 and 31 December 2015 were extracted from the Australian and New Zealand Intensive Care Society Adult Patient Database. Hospital mortality was modelled using logistic regression with development (two-thirds) and validation (one-third) datasets. All predictor variables available at ICU admission were considered for inclusion in the ANZROD₀ model. Model performance was assessed using Brier score, standardised mortality ratio and area under the receiver operating characteristic curve. The relationship between ANZROD₀ and ANZROD predicted risk of death was assessed using linear regression. After standard exclusions, 1,097,416 patients were available for model development and validation. Observed mortality was 9.5%. Model performance measures (Brier score, standardised mortality ratio and area under the receiver operating characteristic curve) for the ANZROD₀ and ANZROD in the validation dataset were 0.069, 1.0 and 0.853; 0.057, 1.0 and 0.909, respectively. There was a strong positive correlation between the mortality predictions with an overall R² of 0.73. We found that the ANZROD₀ model had acceptable calibration and discrimination. Predictions from the models had high correlations in all major diagnostic groups, with the exception of cardiac surgery and possibly trauma and sepsis.

The Ability of the Acute Physiology and Chronic Health Evaluation (APACHE) IV Score to Predict Mortality in a Single Tertiary Hospital

Background

The Acute Physiology and Chronic Health Evaluation (APACHE) II model has been widely used in Korea. However, there have been few studies on the APACHE IV model in Korean intensive care units (ICUs). The aim of this study was to compare the ability of APACHE IV and APACHE II in predicting hospital mortality, and to investigate the ability of APACHE IV as a critical care triage criterion.

Methods

The study was designed as a prospective cohort study. Measurements of discrimination and calibration were performed using the area under the receiver operating characteristic curve (AUROC) and the Hosmer-Lemeshow goodness-of-fit test respectively. We also calculated the standardized mortality ratio (SMR).

Results

The APACHE IV score, the Charlson Comorbidity index (CCI) score, acute respiratory distress syndrome, and unplanned ICU admissions were independently associated with hospital mortality. The calibration, discrimination, and SMR of APACHE IV were good (H = 7.67, P = 0.465; C = 3.42, P = 0.905; AUROC = 0.759; SMR = 1.00). However, the explanatory power of an APACHE IV score >93 alone on hospital mortality was low at 44.1%. The explanatory power was increased to 53.8% when the hospital mortality was predicted using a model that considers APACHE IV >93 scores, medical admission, and risk factors for CCI >3 coincidentally. However, the discriminative ability of the prediction model was unsatisfactory (C index <0.70).

Conclusions

The APACHE IV presented good discrimination, calibration, and SMR for hospital mortality.

Variation in rates of ICU readmissions and post-ICU in-hospital mortality and their association with ICU discharge practices

BACKGROUND

Variation in intensive care unit (ICU) readmissions and in-hospital mortality after ICU discharge may indicate potential for improvement and could be explained by ICU discharge practices. Our objective was threefold: (1) describe variation in rates of ICU readmissions within 48 h and post-ICU in-hospital mortality, (2) describe ICU discharge practices in Dutch hospitals, and (3) study the association between rates of ICU readmissions within 48 h and post-ICU in-hospital mortality and ICU discharge practices.

METHODS

We analysed data on 42,040 admissions to 82 (91.1%) Dutch ICUs in 2011 from the Dutch National Intensive Care Evaluation (NICE) registry to describe variation in standardized ICU readmission and post-ICU mortality rates using funnel-plots. We send a questionnaire to all Dutch ICUs. 75 ICUs responded and their questionnaire data could be linked to 38,498 admissions in the NICE registry. Generalized estimation equations analyses were used to study the association between ICU readmissions and post-ICU mortality rates and the identified discharge practices, i.e. (1) ICU discharge criteria; (2) bed managers; (3) early discharge planning; (4) step-down facilities; (5) medication reconciliation; (6) verbal and written handover; (7) monitoring of post-ICU patients; and (8) consulting ICU nurses. In all analyses, the outcomes were corrected for patient-related confounding factors.

RESULTS

The standardized rate of ICU readmissions varied between 0.14 and 2.67 and 20.8% of the hospitals fell outside the 95% control limits and 3.6% outside the 99.8% control limits. The standardized rate of post-ICU mortality varied between 0.07 and 2.07 and 17.1% of the hospitals fell outside the 95% control limits and 4.9% outside the 99.8% control limits. We could not demonstrate an association between the eight ICU discharge practices and rates of ICU readmissions or post-ICU in-hospital mortality. Implementing a higher number of ICU discharge practices was also not associated with better patient outcomes.

CONCLUSIONS

We found both variation in patient outcomes and variation in ICU discharge practices between ICUs. However, we found no association between discharge practices and rates of ICU readmissions or post-ICU mortality. Further research is necessary to find factors, which may influence these patient outcomes, in order to improve quality of care.

Efficacy and safety of procalcitonin guidance in reducing the duration of antibiotic treatment in critically ill patients: a randomised, controlled, open-label trial

BACKGROUND

In critically ill patients, antibiotic therapy is of great importance but long duration of treatment is associated with the development of antimicrobial resistance. Procalcitonin is a marker used to guide antibacterial therapy and reduce its duration, but data about safety of this reduction are scarce. We assessed the efficacy and safety of procalcitonin-guided antibiotic treatment in patients in intensive care units (ICUs) in a health-care system with a comparatively low use of antibiotics.

METHODS

We did a prospective, multicentre, randomised, controlled, open-label intervention trial in 15 hospitals in the Netherlands. Critically ill patients aged at least 18 years, admitted to the ICU, and who received their first dose of antibiotics no longer than 24 h before inclusion in the study for an assumed or proven infection were eligible to participate. Patients who received antibiotics for presumed infection were randomly assigned (1:1), using a computer-generated list, and stratified (according to treatment centre, whether infection was acquired before or during ICU stay, and dependent on severity of infection [ie, sepsis, severe sepsis, or septic shock]) to receive either procalcitonin-guided or standard-of-care antibiotic discontinuation. Both patients and investigators were aware of group assignment. In the procalcitonin-guided group, a non-binding advice to discontinue antibiotics was provided if procalcitonin concentration had decreased by 80% or more of its peak value or to 0·5 μg/L or lower. In the standard-of-care group, patients were treated according to local antibiotic protocols. Primary endpoints were antibiotic daily defined doses and duration of antibiotic treatment. All analyses were done by intention to treat. Mortality analyses were completed for all patients (intention to treat) and for patients in whom antibiotics were stopped while being on the ICU (per-protocol analysis). Safety endpoints were reinstitution of antibiotics and recurrent inflammation measured by C-reactive protein concentrations and they were measured in the population adhering to the stopping rules (per-protocol analysis). The study is registered with ClinicalTrials.gov, number NCT01139489, and was completed in August, 2014.

FINDINGS

Between Sept 18, 2009, and July 1, 2013, 1575 of the 4507 patients assessed for eligibility were randomly assigned to the procalcitonin-guided group (761) or to standard-of-care (785). In 538 patients (71%) in the procalcitonin-guided group antibiotics were discontinued in the ICU. Median consumption of antibiotics was 7·5 daily defined doses (IQR 4·0-12·7) in the procalcitonin-guided group versus 9·3 daily defined doses (5·0-16·6) in the standard-of-care group (between-group absolute difference 2·69, 95% CI 1·26-4·12, p<0·0001). Median duration of treatment was 5 days (3-9) in the procalcitonin-guided group and 7 days (4-11) in the standard-of-care group (between-group absolute difference 1·22, 0·65-1·78, p<0·0001). Mortality at 28 days was 149 (20%) of 761 patients in the procalcitonin-guided group and 196 (25%) of 785 patients in the standard-of-care group (between-group absolute difference 5·4%, 95% CI 1·2-9·5, p=0·0122) according to the intention-to-treat analysis, and 107 (20%) of 538 patients in the procalcitonin-guided group versus 121 (27%) of 457 patients in the standard-of-care group (between-group absolute difference 6·6%, 1·3-11·9, p=0·0154) in the per-protocol analysis. 1-year mortality in the per-protocol analysis was 191 (36%) of 538 patients in the procalcitonin-guided and 196 (43%) of 457 patients in the standard-of-care groups (between-group absolute difference 7·4, 1·3-13·8, p=0·0188).

INTERPRETATION

Procalcitonin guidance stimulates reduction of duration of treatment and daily defined doses in critically ill patients with a presumed bacterial infection. This reduction was associated with a significant decrease in mortality. Procalcitonin concentrations might help physicians in deciding whether or not the presumed infection is truly bacterial, leading to more adequate diagnosis and treatment, the cornerstones of antibiotic stewardship.

FUNDING

Thermo Fisher Scientific.

To develop a regional ICU mortality prediction model during the first 24 h of ICU admission utilizing MODS and NEMS with six other independent variables from the Critical Care Information System (CCIS) Ontario, Canada

Background

Intensive care unit (ICU) scoring systems or prediction models evolved to meet the desire of clinical and administrative leaders to assess the quality of care provided by their ICUs. The Critical Care Information System (CCIS) is province-wide data information for all Ontario, Canada level 3 and level 2 ICUs collected for this purpose. With the dataset, we developed a multivariable logistic regression ICU mortality prediction model during the first 24 h of ICU admission utilizing the explanatory variables including the two validated scores, Multiple Organs Dysfunctional Score (MODS) and Nine Equivalents Nursing Manpower Use Score (NEMS) followed by the variables age, sex, readmission to the ICU during the same hospital stay, admission diagnosis, source of admission, and the modified Charlson Co-morbidity Index (CCI) collected through the hospital health records.

Methods

This study is a single-center retrospective cohort review of 8822 records from the Critical Care Trauma Centre (CCTC) and Medical-Surgical Intensive Care Unit (MSICU) of London Health Sciences Centre (LHSC), Ontario, Canada between 1 Jan 2009 to 30 Nov 2012. Multivariable logistic regression on training dataset (n = 4321) was used to develop the model and validate by bootstrapping method on the testing dataset (n = 4501). Discrimination, calibration, and overall model performance were also assessed.

Results

The predictors significantly associated with ICU mortality included: age (p < 0.001), source of admission (p < 0.0001), ICU admitting diagnosis (p < 0.0001), MODS (p < 0.0001), and NEMS (p < 0.0001). The variables sex and modified CCI were not significantly associated with ICU mortality. The training dataset for the developed model has good discriminating ability between patients with high risk and those with low risk of mortality (c-statistic 0.787). The Hosmer and Lemeshow goodness-of-fit test has a strong correlation between the observed and expected ICU mortality (χ² = 5.48; p > 0.31). The overall optimism of the estimation between the training and testing data set ΔAUC = 0.003, indicating a stable prediction model.

Conclusions

This study demonstrates that CCIS data available after the first 24 h of ICU admission at LHSC can be used to create a robust mortality prediction model with acceptable fit statistic and internal validity for valid benchmarking and monitoring ICU performance.

Electronic supplementary material

The online version of this article (doi:10.1186/s40560-016-0143-6) contains supplementary material, which is available to authorized users.

Risk factors for and prediction of mortality in critically ill medical-surgical patients receiving heparin thromboprophylaxis

Background

Previous studies have suggested that prediction models for mortality should be adjusted for additional risk factors beyond the Acute Physiology and Chronic Health Evaluation (APACHE) score. Our objective was to identify risk factors independent of APACHE II score and construct a prediction model to improve the predictive accuracy for hospital and intensive care unit (ICU) mortality.

Methods

We used data from a multicenter randomized controlled trial (PROTECT, Prophylaxis for Thromboembolism in Critical Care Trial) to build a new prediction model for hospital and ICU mortality. Our primary outcome was all-cause 60-day hospital mortality, and the secondary outcome was all-cause 60-day ICU mortality.

Results

We included 3746 critically ill non-trauma medical–surgical patients receiving heparin thromboprophylaxis (43.3 % females) in this study. The new model predicting 60-day hospital mortality incorporated APACHE II score (main effect: hazard ratio (HR) = 0.97 for per-point increase), body mass index (BMI) (main effect: HR = 0.92 for per-point increase), medical admission versus surgical (HR = 1.67), use of inotropes or vasopressors (HR = 1.34), acetylsalicylic acid or clopidogrel (HR = 1.27) and the interaction term between APACHE II score and BMI (HR = 1.002 for per-point increase). This model had a good fit to the data and was well calibrated and internally validated. However, the discriminative ability of the prediction model was unsatisfactory (C index < 0.65). Sensitivity analyses supported the robustness of these findings. Similar results were observed in the new prediction model for 60-day ICU mortality which included APACHE II score, BMI, medical admission and invasive mechanical ventilation.

Conclusion

Compared with the APACHE II score alone, the new prediction model increases data collection, is more complex but does not substantially improve discriminative ability.

Trial registration: ClinicalTrials.gov Identifier: NCT00182143

Do Serially Recorded Prognostic Scores Predict Outcome Better Than One-Time Recorded Score on Admission? A Prospective Study in Adult Intensive Care Patients

OBJECTIVES

The prognosticating ability of one-time recorded Acute Physiology and Chronic Health Evaluation (APACHE) IV score was compared with serially recorded Mortality Prediction Model (MPM) II scores.

DESIGN AND METHODS

A prospective observational study was conducted for a period of 6 months. Acute Physiology and Chronic Health Evaluation IV score was recorded during the first day on intensive care unit (ICU) admission. Mortality Prediction Model II was recorded on admission, 24, 48, and 72 hours. Predicted mortality was compared with observed mortality. The systems were calibrated and tested for discriminant functions.

RESULTS

One hundred and fifty patients were studied. The observed mortality was 21.3%. The mean predicted hospital mortality by APACHE IV was 20.6%. The mean predicted hospital mortality rate by serial MPM II measurements was 27.7%, 24.3%, 25.5%, and 25.8%. The area under the receiver-operating characteristic curve was 0.87 for APACHE IV and 0.82, 0.84, 0.85, and 0.89 for MPM II series. Both systems calibrated well with similar degree of goodness of fit.

CONCLUSION

Acute Physiology and Chronic Health Evaluation IV on admission predicted hospital mortality better than serially recorded MPM, which overestimated mortality. Also, APACHE IV had a slightly better discrimination compared to MPM II on admission. One-time recording of APACHE IV on admission may be sufficient for prognostication of ICU patients rather than serial MPM scores.

Standardised mortality ratio based on the sum of age and percentage total body surface area burned is an adequate quality indicator in burn care: An exploratory review

Standardised Mortality Ratio (SMR) based on generic mortality predicting models is an established quality indicator in critical care. Burn-specific mortality models are preferred for the comparison among patients with burns as their predictive value is better. The aim was to assess whether the sum of age (years) and percentage total body surface area burned (which constitutes the Baux score) is acceptable in comparison to other more complex models, and to find out if data collected from a separate burn centre are sufficient for SMR based quality assessment. The predictive value of nine burn-specific models was tested by comparing values from the area under the receiver-operating characteristic curve (AUC) and a non-inferiority analysis using 1% as the limit (delta). SMR was analysed by comparing data from seven reference sources, including the North American National Burn Repository (NBR), with the observed mortality (years 1993-2012, n=1613, 80 deaths). The AUC values ranged between 0.934 and 0.976. The AUC 0.970 (95% CI 0.96-0.98) for the Baux score was non-inferior to the other models. SMR was 0.52 (95% CI 0.28-0.88) for the most recent five-year period compared with NBR based data. The analysis suggests that SMR based on the Baux score is eligible as an indicator of quality for setting standards of mortality in burn care. More advanced modelling only marginally improves the predictive value. The SMR can detect mortality differences in data from a single centre.

Survival and quality of life after extracorporeal life support for refractory cardiac arrest: A case series

OBJECTIVES

Extracorporeal life support (ECLS) is an emerging option to treat selected patients with cardiac arrest refractory to cardiopulmonary resuscitation (CPR). Our primary objective was to determine the mortality at 30 days and at hospital discharge among adult patients receiving veno-arterial ECLS for refractory cardiac arrest. Our secondary objectives were to determine the 1-year survival and the health-related quality of life, and to examine factors associated with 30-day mortality.

METHODS

In a retrospective, single-center investigation within a tertiary referral center, we analyzed the prospectively collected data of 49 patients rescued from refractory cardiac arrest through emergent implantation of ECLS (E-CPR) (18.1% of our overall ECLS activity, 2005-2013), implanted in-hospital and during ongoing external cardiac massage in all cases. A prospective follow-up with administration of the Medical Outcomes Study 36-Item Short-Form Health Survey (SF-36) questionnaire was performed.

RESULTS

The mean age was 47.6 ± 1.6 years; out-of-hospital cardiac arrest occurred in 12% of cases; average low-flow time was 47.2 ± 33 minutes; causes of cardiac arrest were heart disease (61.2%), trauma (14.3%), respiratory disease (4.1%), sepsis (2%), and miscellaneous (18.4%).

PRIMARY OBJECTIVE

Rates of survival at E-CPR explantation and at 30 days were 42.9% and 36.7%, respectively; brain death occurred in 24.5% of cases.

SECONDARY OBJECTIVES

Increased simplified acute physiology score; higher serum lactate levels and lower body temperature at the time of implantation were associated with 30-day mortality. Bridge to heart transplantation or implantation of a long-term ventricular assist device was performed in 8.2%. No deaths occurred during the follow-up after discharge (36.7% survival; average follow-up was 15.6 ± 19.2 months). The average Physical Component Summary and Mental Component Summary scores (SF-36 questionnaire) were, respectively, 45.2 ± 6.8 and 48.3 ± 7.7 among survivors.

CONCLUSIONS

Extracorporeal cardiopulmonary resuscitation is a viable treatment for selected patients with cardiac arrest refractory to CPR. In our series, approximately one third of rescued patients were alive at 6 months and presented quality-of-life scores comparable to those previously observed in patients treated with ECLS.

Comparison of acute physiology and chronic health evaluation II and acute physiology and chronic health evaluation IV to predict intensive care unit mortality

Context:

Clinical assessment of severity of illness is an essential component of medical practice to predict the outcome of critically ill-patient. Acute Physiology and Chronic Health Evaluation (APACHE) model is one of the widely used scoring systems.

Aims:

This study was designed to evaluate the Performance of APACHE II and IV scoring systems in our Intensive Care Unit (ICU).

Settings and Design:

A prospective study in 6 bedded ICU, including 76 patients all above 15 years.

Subjects and Methods:

APACHE II and APACHE IV scores were calculated based on the worst values in the first 24 h of admission. All enrolled patients were followed, and outcome was recorded as survivors or nonsurvivors.

Statistical Analysis Used:

SPSS version 17.

Results:

The mean APACHE score was significantly higher among nonsurvivors than survivors (P < 0.005). Discrimination for APACHE II and APACHE IV was fair with area under receiver operating characteristic curve of 0.73 and 0.79 respectively. The cut-off point with best Youden index for APACHE II was 17 and for APACHE IV was 85. Above cut-off point, mortality was higher for both models (P < 0.005). Hosmer–Lemeshow Chi-square coefficient test showed better calibration for APACHE II than APACHE IV. A positive correlation was seen between the models with Spearman's correlation coefficient of 0.748 (P < 0.01).

Conclusions:

Discrimination was better for APACHE IV than APACHE II model however Calibration was better for APACHE II than APACHE IV model in our study. There was good correlation between the two models observed in our study.

Comparison of regression methods for modeling intensive care length of stay

Intensive care units (ICUs) are increasingly interested in assessing and improving their performance. ICU Length of Stay (LoS) could be seen as an indicator for efficiency of care. However, little consensus exists on which prognostic method should be used to adjust ICU LoS for case-mix factors. This study compared the performance of different regression models when predicting ICU LoS. We included data from 32,667 unplanned ICU admissions to ICUs participating in the Dutch National Intensive Care Evaluation (NICE) in the year 2011. We predicted ICU LoS using eight regression models: ordinary least squares regression on untransformed ICU LoS,LoS truncated at 30 days and log-transformed LoS; a generalized linear model with a Gaussian distribution and a logarithmic link function; Poisson regression; negative binomial regression; Gamma regression with a logarithmic link function; and the original and recalibrated APACHE IV model, for all patients together and for survivors and non-survivors separately. We assessed the predictive performance of the models using bootstrapping and the squared Pearson correlation coefficient (R2), root mean squared prediction error (RMSPE), mean absolute prediction error (MAPE) and bias. The distribution of ICU LoS was skewed to the right with a median of 1.7 days (interquartile range 0.8 to 4.0) and a mean of 4.2 days (standard deviation 7.9). The predictive performance of the models was between 0.09 and 0.20 for R2, between 7.28 and 8.74 days for RMSPE, between 3.00 and 4.42 days for MAPE and between -2.99 and 1.64 days for bias. The predictive performance was slightly better for survivors than for non-survivors. We were disappointed in the predictive performance of the regression models and conclude that it is difficult to predict LoS of unplanned ICU admissions using patient characteristics at admission time only.

Validation of the APACHE IV model and its comparison with the APACHE II, SAPS 3, and Korean SAPS 3 models for the prediction of hospital mortality in a Korean surgical intensive care unit

Background

The Acute Physiology and Chronic Health Evaluation (APACHE) IV model has not yet been validated in Korea. The aim of this study was to compare the ability of the APACHE IV with those of APACHE II, Simplified Acute Physiology Score (SAPS) 3, and Korean SAPS 3 in predicting hospital mortality in a surgical intensive care unit (SICU) population.

Methods

We retrospectively reviewed electronic medical records for patients admitted to the SICU from March 2011 to February 2012 in a university hospital. Measurements of discrimination and calibration were performed using the area under the receiver operating characteristic curve (AUC) and the Hosmer-Lemeshow test, respectively. We calculated the standardized mortality ratio (SMR, actual mortality predicted mortality) for the four models.

Results

The study included 1,314 patients. The hospital mortality rate was 3.3%. The discriminative powers of all models were similar and very reliable. The AUCs were 0.80 for APACHE IV, 0.85 for APACHE II, 0.86 for SAPS 3, and 0.86 for Korean SAPS 3. Hosmer and Lemeshow C and H statistics showed poor calibration for all of the models (P < 0.05). The SMRs of APACHE IV, APACHE II, SAPS 3, and Korean SAPS 3 were 0.21, 0.11 0.23, 0.34, and 0.25, respectively.

Conclusions

The APACHE IV revealed good discrimination but poor calibration. The overall discrimination and calibration of APACHE IV were similar to those of APACHE II, SAPS 3, and Korean SAPS 3 in this study. A high level of customization is required to improve calibration in this study setting.

A calibration study of SAPS II with Norwegian intensive care registry data

Background

Mortality prediction is important in intensive care. The Simplified Acute Physiology Score (SAPS) II is a tool for predicting such mortality. However, the original SAPS II is poorly calibrated to current intensive care unit (ICU) populations because it draws on data, which is more than 20 years old. We aimed to improve the calibration of SAPS II using data from the Norwegian Intensive Care Registry (NIR). This is the first recalibration of SAPS II for Nordic data.

Methods

A first-level customization was applied to improve calibration of the original SAPS II model (Model A). NIR data used covered more than 90% of adult patients admitted to ICUs in Norway from 2008 to 2010 (n = 30712).

Results

The modified SAPS II, Model B, outperformed the original Model A with respect to calibration. Model B gave more accurate predictions of mortality than Model A (Hosmer–Lemeshow's C: 22.01 vs. 689.07; Brier score: 0.120 vs. 0.131; Cox's calibration regression: α = −0.093 vs. −0.747, β = 0.921 vs. 0.735, (α|β = 1) = −0.009 vs. −0.630). The standardized mortality ratio was 0.73 [95% confidence interval (CI) of 0.70–0.76] for Model A and 0.99 (95% CI of 0.95–1.04) for Model B. Discrimination was good for both models (area under receiver operating characteristic curve = 0.83 for both models).

Conclusions

As expected, Model B is better calibrated than Model A, and both models have similar uniformity of fit and equal discrimination. Introducing Model B into Norwegian ICUs may improve precision in decision-making. Units will have a more realistic benchmark for the assessment of ICU performance. Mortality risk estimates from Model B are better than previous SAPS II estimates have been.

APACHE IV is superior to MELD scoring system in predicting prognosis in patients after orthotopic liver transplantation

This study aims to compare the efficiency of APACHE IV with that of MELD scoring system for prediction of the risk of mortality risk after orthotopic liver transplantation (OLT). A retrospective cohort study was performed based on a total of 195 patients admitted to the ICU after orthotopic liver transplantation (OLT) between February 2006 and July 2009 in Guangzhou, China. APACHE IV and MELD scoring systems were used to predict the postoperative mortality after OLT. The area under the receiver operating characteristic curve (AUC) and the Hosmer-Lemeshow C statistic were used to assess the discrimination and calibration of APACHE IV and MELD, respectively. Twenty-seven patients died during hospitalization with a mortality rate of 13.8%. The mean scores of APACHE IV and MELD were 42.32 ± 21.95 and 18.09 ± 10.55, respectively, and APACHE IV showed better discrimination than MELD; the areas under the receiver operating characteristic curve for APACHE IV and MELD were 0.937 and 0.694 (P < 0.05 for both models), which indicated that the prognostic value of APACHE IV was relatively high. Both models were well-calibrated (The Hosmer-Lemeshow C statistics were 1.568 and 6.818 for APACHE IV and MELD, resp.; P > 0.05 for both). The respective Youden indexes of APACHE IV, MELD, and combination of APACHE IV with MELD were 0.763, 0.430, and 0.545. The prognostic value of APACHE IV is high but still underestimates the overall hospital mortality, while the prognostic value of MELD is poor. The function of the APACHE IV is, thus, better than that of the MELD.

Risk prediction of hospital mortality for adult patients admitted to Australian and New Zealand intensive care units: development and validation of the Australian and New Zealand Risk of Death model

PURPOSE

The purpose of this study is to develop and validate a new mortality prediction model (Australian and New Zealand Risk of Death [ANZROD]) for Australian and New Zealand intensive care units (ICUs) and compare its performance with the existing Acute Physiology and Chronic Health Evaluation (APACHE) III-j.

MATERIALS AND METHODS

All ICU admissions from 2004 to 2009 were extracted from the Australian and New Zealand Intensive Care Society Adult Patient Database. Hospital mortality was modeled using logistic regression with training (two third) and validation (one third) data sets. Predictor variables included APACHE III score components, source of admission to ICU and hospital, lead time, elective surgery, treatment limitation, ventilation status, and APACHE III diagnoses. Model performance was assessed by standardized mortality ratio, Hosmer-Lemeshow C and H statistics, Brier score, Cox calibration regression, area under the receiver operating characteristic curve, and calibration curves.

RESULTS

There were 456605 patients available for model development and validation. Observed mortality was 11.3%. Performance measures (standardized mortality ratio, Hosmer-Lemeshow C and H statistics, and receiver operating characteristic curve) for the ANZROD and APACHE III-j model in the validation data set were 1.01, 104.9 and 111.4, and 0.902; 0.84, 1596.6 and 2087.3, and 0.885, respectively.

CONCLUSIONS

The ANZROD has better calibration; discrimination compared with the APACHE III-j. Further research is required to validate performance over time and in specific subgroups of ICU population.

Prediction of long-term mortality in ICU patients: model validation and assessing the effect of using in-hospital versus long-term mortality on benchmarking

PURPOSE

To analyze the influence of using mortality 1, 3, and 6 months after intensive care unit (ICU) admission instead of in-hospital mortality on the quality indicator standardized mortality ratio (SMR).

METHODS

A cohort study of 77,616 patients admitted to 44 Dutch mixed ICUs between 1 January 2008 and 1 July 2011. Four Acute Physiology and Chronic Health Evaluation (APACHE) IV models were customized to predict in-hospital mortality and mortality 1, 3, and 6 months after ICU admission. Models' performance, the SMR and associated SMR rank position of the ICUs were assessed by bootstrapping.

RESULTS

The customized APACHE IV models can be used for prediction of in-hospital mortality as well as for mortality 1, 3, and 6 months after ICU admission. When SMR based on mortality 1, 3 or 6 months after ICU admission was used instead of in-hospital SMR, 23, 36, and 30% of the ICUs, respectively, received a significantly different SMR. The percentages of patients discharged from ICU to another medical facility outside the hospital or to home had a significant influence on the difference in SMR rank position if mortality 1 month after ICU admission was used instead of in-hospital mortality.

CONCLUSIONS

The SMR and SMR rank position of ICUs were significantly influenced by the chosen endpoint of follow-up. Case-mix-adjusted in-hospital mortality is still influenced by discharge policies, therefore SMR based on mortality at a fixed time point after ICU admission should preferably be used as a quality indicator for benchmarking purposes.

Reliability of in-hospital mortality as a quality indicator in clinical quality registries. A case study in an intensive care quality register

OBJECTIVES

Errors in the registration or extraction of patient outcome data, such as in-hospital mortality, may lower the reliability of the quality indicator that uses this (partly) incorrect data. Our aim was to measure the reliability of in-hospital mortality registration in the Dutch National Intensive Care Evaluation (NICE) registry.

METHODS

We linked data of the NICE registry with an insurance claims database, resulting in a list of discrepancies in in-hospital mortality. Eleven Intensive Care Units (ICUs) were visited where local data sources were investigated to find the true in-hospital mortality status of the discrepancies and to identify the causes of the data errors in the NICE registry. Original and corrected Standardized Mortality Ratios (SMRs) were calculated to determine if conclusions about quality of care changed compared to the national benchmark.

RESULTS

In eleven ICUs, 23,855 records with 460 discrepancies were identified of which 255 discrepancies (1.1% of all linked records) were due to incorrect in-hospital mortality registration in the NICE registry. Two programming errors in computer software of six ICUs caused 78% of errors, the remainder was caused by manual transcription errors and failure to record patient outcomes. For one ICU the performance became concordant with the national benchmark after correction, instead of being better.

CONCLUSIONS

The reliability of in-hospital mortality registration in the NICE registry was good. This was reflected by the low number of data errors and by the fact that conclusions about the quality of care were only affected for one ICU due to systematic data errors. We recommend that registries frequently verify the software used in the registration process, and compare mortality data with an external source to assure consistent quality of data.

A strategy to enhance the safety and efficiency of handovers of ICU patients: study protocol of the pICUp study

Background

To use intensive care unit (ICU) facilities efficiently and ensure high quality of care, an optimal patient flow is necessary. Discharging patients relieves the pressure on ICU beds but the risk of premature discharge must be managed carefully. Suboptimal patient discharge may result in ICU readmissions and in patients’ death.

The aim of this study is to obtain insight into the safety and efficiency of current ICU discharge practices and into barriers and facilitators to the implementation of effective ICU discharge interventions, and to develop an implementation strategy tailored to the barriers and facilitators identified.

Methods/design

This study exists of five phases. Phase A: analysis of routinely registered data on variation in ICU readmissions and hospital mortality after ICU discharge of all ICUs participating in the Dutch National Intensive Care Evaluation registry (n = 83). Phase B: systematic review of effective interventions aiming to improve the efficiency and safety of the ICU discharge process. Phase C: assessing the intervention adherence with a questionnaire survey among all Dutch ICUs (n = 90). Phase D: assessing barriers and facilitators to the implementation of effective ICU discharge interventions with a questionnaire survey among all Dutch intensivists (n = 700). The questionnaire will be based on barriers and facilitators identified by focus groups (n = 4) and individual interviews with professionals of ICUs and general wards and adult discharged ICU patients (n = 25 to 30). Phase E: systematic development of an implementation strategy based on the sampled data in phase A to D, and effective implementation strategies from the literature using the intervention mapping method.

Discussion

Using theory and empirical data, an implementation strategy will be developed to improve the safety and efficiency of the ICU discharge process. The developed strategy will be evaluated in a subsequent study. The knowledge obtained in this study should be used for further implementation of ICU discharge interventions, and can be used for implementation of handover interventions in other healthcare transition settings.

Use of an abnormal laboratory value-drug combination alert to detect drug-induced thrombocytopenia in critically Ill patients

PURPOSE

The aim of this study was to assess the performance of a commercially available clinical decision support system (CDSS) drug-laboratory result alert in detecting drug-induced thrombocytopenia in critically ill patients.

MATERIALS AND METHODS

Adult patients admitted to the medical and cardiac intensive care unit during an 8-week period and identified by 1 of 3 signals in the CDSS, TheraDoc, were eligible. Alerts were generated when the patient had a low platelet count and was ordered a potentially causal drug. Patients were evaluated in real time for the occurrence of an adverse drug reaction using 3 causality instruments. Positive predictive values were calculated for the alert.

RESULTS

Sixty-four patients with a mean age of 54 years met the inclusion criteria, generating 350 alerts. Positive predictive values were 0.36, 0.83, and 0.40 for signals 1, 2, and 3, respectively. Overall, there were 137 adverse drug reactions identified in the 350 alerts, with heparin, vancomycin, and famotidine as the 3 most common potential causes.

CONCLUSIONS

A commercial CDSS drug-laboratory alert is effective at identifying drug-induced thrombocytopenia in the intensive care unit and may improve patient safety. Compared with previous studies, the combination alert performs better than alerts based exclusively on laboratory values and should be considered to reduce alert fatigue.

Effectiveness of classroom based crew resource management training in the intensive care unit: study design of a controlled trial

BACKGROUND

Crew resource management (CRM) has the potential to enhance patient safety in intensive care units (ICU) by improving the use of non-technical skills. However, CRM evaluation studies in health care are inconclusive with regard to the effect of this training on behaviour and organizational outcomes, due to weak study designs and the scarce use of direct observations. Therefore, the aim of this study is to determine the effectiveness and cost-effectiveness of CRM training on attitude, behaviour and organization after one year, using a multi-method approach and matched control units. The purpose of the present article is to describe the study protocol and the underlying choices of this evaluation study of CRM in the ICU in detail.

METHODS/DESIGN

Six ICUs participated in a paired controlled trial, with one pre-test and two post test measurements (respectively three months and one year after the training). Three ICUs were trained and compared to matched control ICUs. The 2-day classroom-based training was delivered to multidisciplinary groups. Typical CRM topics on the individual, team and organizational level were discussed, such as situational awareness, leadership and communication. All levels of Kirkpatrick's evaluation framework (reaction, learning, behaviour and organisation) were assessed using questionnaires, direct observations, interviews and routine ICU administration data.

DISCUSSION

It is expected that the CRM training acts as a generic intervention that stimulates specific interventions. Besides effectiveness and cost-effectiveness, the assessment of the barriers and facilitators will provide insight in the implementation process of CRM.

TRIAL REGISTRATION

Netherlands Trial Register (NTR): NTR1976.

Caution when using prognostic models: a prospective comparison of 3 recent prognostic models

PURPOSE

Prognostic models have been developed to estimate mortality and to compare outcomes in different intensive care units. However, these models need to be validated before their use in different populations. In this study, we assessed the performance of 3 recently developed general prognostic models (Acute Physiologic and Chronic Health Evaluation [APACHE] IV, Simplified Acute Physiology Score [SAPS] 3 and Mortality Probability Model III [MPM(0)-III]) in a population admitted at 3 medical-surgical Brazilian intensive care units.

MATERIALS AND METHODS

All patients admitted from July 2008 to December 2009 were evaluated for inclusion in the study. Standardized mortality ratios were calculated for all models. Calibration was assessed by the Hosmer-Lemeshow goodness-of-fit test. Discrimination was evaluated using the area under the receiver operator curve.

RESULTS

A total of 5780 patients were included. Inhospital mortality was 9.1%. Discrimination was very good for all models (area under the receiver operator curve for APACHE IV, SAPS 3 and MPM(0)-III was 0.883, 0.855 and 0.840, respectively). APACHE IV showed better discrimination than SAPS 3 and MPM(0)-III (P < .001 for both comparisons). All models calibrated poorly and overestimated hospital mortality (Hosmer-Lemeshow statistic was 53.7, 134.2, 226.6 for APACHE IV, MPM(0)-III, and SAPS 3, respectively; P < .001 for all).

CONCLUSIONS

In this study, all models showed poor calibration, while discrimination was very good for all of them. As this has been a common finding in validation studies, caution is warranted when using prognostic models for benchmarking.