Which risk-adjustment index performs better in predicting 30-day mortality? A systematic review and meta-analysis

Comparative performance of three comorbidity scores in predicting survival after the elective repair of abdominal aortic aneurysms

BACKGROUND

We aimed to study the discriminative power of 3 comorbidity scores for predicting 5-year survival after the elective repair of aorto-iliac aneurysms (AAA).

METHODS

444 patients with AAA undergoing elective repair (33% open and 67% endovascular) between 2000 and 2020 were reviewed. The Charlson Comorbidity Index (CCI) and subsequent adjustments by Schneeweiss, Quan and Armitage, the Modified Frailty Index (MFI) and the American Society of Anesthesiologists Score (ASA) were calculated from preoperative data. Their association with 5-year survival was analyzed using Cox regression models and their discriminative power and its changes with C statistics and Net Reclassification Index (NRI).

RESULTS

All comorbidity scores were associated with survival after adjusting by age, sex and type of surgical repair: original CCI HR=1.24, P<0.001; Schneeweiss CCI HR=1.23, P<0.001; Quan CCI HR=1.27, P<0.001, Armitage CCI HR=1.46, P<0.001, MFI HR=1.39, P<0.001 and ASA HR=1.68 (P=0.04) and 2.86 (P=0.01) for classes III and IV, respectively. Associated C statistics were of 0.64, 0.65, 0.65, 0.64, 0.61 and 0.59, respectively. Compared with the original CCI, models based on Schneeweiss CCI and Armitage CCI provided minor improvements in NRI (0.32 and 0.23), and the model based on ASA showed lower C statistics (P=0.014) and NRI (-0.30).

CONCLUSIONS

Established comorbidity scores, such as CCI, MFI or ASA, are all associated with 5-year survival after the elective repair of AAAs, being ASA the worst of them. However, their predictive power is in no case sufficient to identify, by themselves, those patients who may not be eligible for intervention on the basis of life expectancy.

Predicting mortality in the very old: a machine learning analysis on claims data

Machine learning (ML) may be used to predict mortality. We used claims data from one large German insurer to develop and test differently complex ML prediction models, comparing them for their (balanced) accuracy, but also the importance of different predictors, the relevance of the follow-up period before death (i.e. the amount of accumulated data) and the time distance of the data used for prediction and death. A sample of 373,077 insured very old, aged 75 years or above, living in the Northeast of Germany in 2012 was drawn and followed over 6 years. Our outcome was whether an individual died in one of the years of interest (2013-2017) or not; the primary metric was (balanced) accuracy in a hold-out test dataset. From the 86,326 potential variables, we used the 30 most important ones for modeling. We trained a total of 45 model combinations: (1) Three different ML models were used; logistic regression (LR), random forest (RF), extreme gradient boosting (XGB); (2) Different periods of follow-up were employed for training; 1-5 years; (3) Different time distances between data used for prediction and the time of the event (death/survival) were set; 0-4 years. The mortality rate was 9.15% in mean per year. The models showed (balanced) accuracy between 65 and 93%. A longer follow-up period showed limited to no advantage, but models with short time distance from the event were more accurate than models trained on more distant data. RF and XGB were more accurate than LR. For RF and XGB sensitivity and specificity were similar, while for LR sensitivity was significantly lower than specificity. For all three models, the positive-predictive-value was below 62% (and even dropped to below 20% for longer time distances from death), while the negative-predictive-value significantly exceeded 90% for all analyses. The utilization of and costs for emergency transport as well as emergency and any hospital visits as well as the utilization of conventional outpatient care and laboratory services were consistently found most relevant for predicting mortality. All models showed useful accuracies, and more complex models showed advantages. The variables employed for prediction were consistent across models and with medical reasoning. Identifying individuals at risk could assist tailored decision-making and interventions.

Intestinal failure in Finland: prevalence and characteristics of an adult patient population

OBJECTIVES

Details of intestinal failure in the Finnish adult population are unknown. This study aimed to specify the intestinal failure prevalence and to clinically characterize the patient population in Finland.

METHODS

All Finnish healthcare units with the potential of providing parenteral support received an electronic survey to report whether they had patient(s) aged ≥18 years on long-term (≥120 days) parenteral support due to intestinal failure. Patient details came from patient records. IBM SPSS v.25 was used to analyze descriptive statistics.

RESULTS

Of the 74 patients, 52 were included after confirming parenteral support indication from the records. The adult intestinal failure prevalence for 2017 was 11.7 per million, 95% confidence interval: 8.9-15.3. Most patients were women (69%), and the median age was 62 (45-72) years. Short bowel syndrome was the most frequent intestinal failure mechanism (73%), and surgical complication the most frequent underlying diagnosis (29%). Of patients, 66% represented the clinical classification category parenteral nutrition 1 or parenteral nutrition 2. Median Charlson Comorbidity Index was one (0-2.8); hypertension (37%) and diabetes (23%) were the most frequent comorbidities. Patients received seven (3.5-7) parenteral support infusions weekly, and eight patients (15%) were on fluids and electrolytes only. The median duration of parenteral support was 27.5 (11.3-57.3) months. Ten patients ceased parenteral support during 2017 after a median of 20.0 (9.0-40.3) parenteral support months. Eight weaned off parenteral support, one ran out of catheter sites, and one died.

CONCLUSION

Prevalence and patient characteristics of adult intestinal failure in Finland are similar to those in other Western countries.

Development and Validation of a 30-Day In-hospital Mortality Model Among Seriously Ill Transferred Patients: a Retrospective Cohort Study

BACKGROUND

Predicting the risk of in-hospital mortality on admission is challenging but essential for risk stratification of patient outcomes and designing an appropriate plan-of-care, especially among transferred patients.

OBJECTIVE

Develop a model that uses administrative and clinical data within 24 h of transfer to predict 30-day in-hospital mortality at an Academic Health Center (AHC).

DESIGN

Retrospective cohort study. We used 30 putative variables in a multiple logistic regression model in the full data set (n = 10,389) to identify 20 candidate variables obtained from the electronic medical record (EMR) within 24 h of admission that were associated with 30-day in-hospital mortality (p < 0.05). These 20 variables were tested using multiple logistic regression and area under the curve (AUC)-receiver operating characteristics (ROC) analysis to identify an optimal risk threshold score in a randomly split derivation sample (n = 5194) which was then examined in the validation sample (n = 5195).

PARTICIPANTS

Ten thousand three hundred eighty-nine patients greater than 18 years transferred to the Indiana University (IU)-Adult Academic Health Center (AHC) between 1/1/2016 and 12/31/2017.

MAIN MEASURES

Sensitivity, specificity, positive predictive value, C-statistic, and risk threshold score of the model.

KEY RESULTS

The final model was strongly discriminative (C-statistic = 0.90) and had a good fit (Hosmer-Lemeshow goodness-of-fit test [X² (8) =6.26, p = 0.62]). The positive predictive value for 30-day in-hospital death was 68%; AUC-ROC was 0.90 (95% confidence interval 0.89-0.92, p < 0.0001). We identified a risk threshold score of -2.19 that had a maximum sensitivity (79.87%) and specificity (85.24%) in the derivation and validation sample (sensitivity: 75.00%, specificity: 85.71%). In the validation sample, 34.40% (354/1029) of the patients above this threshold died compared to only 2.83% (118/4166) deaths below this threshold.

CONCLUSION

This model can use EMR and administrative data within 24 h of transfer to predict the risk of 30-day in-hospital mortality with reasonable accuracy among seriously ill transferred patients.

Predictors of medium- and long-term mortality in elderly patients with acute pulmonary embolism

Introduction

Data on medium- and long-term prognostic factors for death in elderly patients with acute Pulmonary Embolism (APE) are lacking. The present study aimed to assess sPESI score and the Charlson Comorbidity Index (CCI) as medium- and long-term predictors of mortality in elderly patients with haemodinamically stable APE.

Methods

All consecutive patients aged≥65 years old, evaluated at the emergency department (ED) of our hospital from 2010 through 2014, with a final diagnosis of APE, were included in this retrospective cohort study.

Results

Study population:162 patients, female:36.5%, median age:79 years old, 74% presented a sPESI score>0, and 61% a CCI≥ 1. All causes mortality: 19.8%, 23.5%, 26.5%, 32.1% and 48.2% at 3, 6 months, 1, 2 and 5 years after APE. Univariate regression analysis: CCI≥1 was associated with a higher mortality at 3, 6 months, 1, 2 and 5 years. Multivariate Cox analysis: CCI≥1 associated with increased mortality at 3 months (HR:4.29; IC95%:1.46-12.59), 6 months (HR:5.33; IC95%:1.84-15.44), 1 year (HR:4.87; IC95%:1.87-12.70), 2 years (HR:3.78; IC95%:1.74-8.25), and 5 years (HR:2.30; IC95%:1.33-3.99). sPESI score≥1 was not found to be related to an increased medium-or long-term mortality. Negative predictive values (IC95%) of CCI≥1 were 93.65% (87.61-99.69), 93.65% (87.61-99.69), 92.06% (85.37-98.76), 87.3% (79.05-95.55) and 71.61% (60.13-83.1) for mortality at 3, 6 months, 1, 2 and 5 years.

Conclusion

In elderly patients with a confirmed normotensive APE, unlike sPESI score, CCI showed to be an independent prognostic factor for medium- and long-term mortality. In these patients, after the acute phase following a PE event, the assessment of the comorbidities burden represents the most appropriate approach for predicting medium- and long-term mortality.

Variability in usual care fluid resuscitation and risk-adjusted outcomes for mechanically ventilated patients in shock

RATIONALE

There remains significant controversy regarding the optimal approach to fluid resuscitation for patients in shock. The magnitude of care variability in shock resuscitation, the confounding effects of disease severity and comorbidity, and the relative impact on sepsis survival are poorly understood.

OBJECTIVE

To evaluate usual care variability and determine the differential effect of observed and predicted fluid resuscitation volumes on risk-adjusted hospital mortality for mechanically ventilated patients in shock.

METHODS

We performed a retrospective outcome analysis of mechanically ventilated patients admitted to intensive care units using the 2013 Premier Hospital Database (Premier, Inc.). Observed and predicted hospital mortality were evaluated by observed and predicted day 1 fluid administration, using the difference in predicted and observed outcomes to adjust for disease severity between groups. Both predictive models were validated using a second large administrative database (Truven Health Analytics Inc.). Secondary outcomes included duration of mechanical ventilation, hospital and ICU length of stay, and cost.

RESULTS

Among 33,831 patients, observed hospital mortality was incrementally higher than predicted for each additional liter of day 1 fluid beginning at 7 L (40.9% vs. 37.2%, p = 0.008). Compared to patients that received expected (± 1.5 L predicted) day 1 fluid volumes, greater-than-expected fluid resuscitation was associated with increased risk-adjusted hospital mortality (52.3% vs. 45.0%, p < 0.0001) among all patients with shock and among a subgroup of shock patients with comorbid conditions predictive of lower fluid volume administration (47.1% vs. 41.5%, p < 0.0001). However, in patients with shock but without such conditions, both greater-than-expected (57.5% vs. 49.2%, p < 0.0001) and less-than-expected (52.1% vs. 49.2%, p = 0.037) day 1 fluid resuscitation were associated with increased risk-adjusted hospital mortality.

CONCLUSIONS

Highly variable day 1 fluid resuscitation was associated with a non-uniform impact on risk-adjusted hospital mortality among distinct subgroups of mechanically ventilated patients with shock. These findings support closer evaluation of fluid resuscitation strategies that include broadly applied fluid volume targets in the early phase of shock resuscitation.