Comparing Mortality Prediction by Quick Sequential Organ Failure Assessment with Emergency Physician Judgment

Comparing In-Hospital Mortality Prediction by Senior Emergency Resident's Judgment and Prognostic Models in the Emergency Department

Background

A comparison of emergency residents' judgments and two derivatives of the Sequential Organ Failure Assessment (SOFA), namely, the mSOFA and the qSOFA, was conducted to determine the accuracy of predicting in-hospital mortality among critically ill patients in the emergency department (ED).

Methods

A prospective cohort research was performed on patients over 18 years of age presented to the ED. We used logistic regression to develop a model for predicting in-hospital mortality by using qSOFA, mSOFA, and residents' judgment scores. We compared the accuracy of prognostic models and residents' judgment in terms of the overall accuracy of the predicted probabilities (Brier score), discrimination (area under the ROC curve), and calibration (calibration graph). Analyses were carried out using R software version R-4.2.0.

Results

In the study, 2,205 patients with median age of 64 (IQR: 50-77) years were included. There were no significant differences between the qSOFA (AUC 0.70; 95% CI: 0.67-0.73) and physician's judgment (AUC 0.68; 0.65-0.71). Despite this, the discrimination of mSOFA (AUC 0.74; 0.71-0.77) was significantly higher than that of the qSOFA and residents' judgments. Additionally, the AUC-PR of mSOFA, qSOFA, and emergency resident's judgments was 0.45 (0.43-0.47), 0.38 (0.36-0.40), and 0.35 (0.33-0.37), respectively. The mSOFA appears stronger in terms of overall performance: 0.13 vs. 0.14 and 0.15. All three models showed good calibration.

Conclusion

The performance of emergency residents' judgment and the qSOFA was the same in predicting in-hospital mortality. However, the mSOFA predicted better-calibrated mortality risk. Large-scale studies should be conducted to determine the utility of these models.

Mortality Prediction in Hip Fracture Patients: Physician Assessment Versus Prognostic Models

OBJECTIVES

To evaluate 2 prognostic models for mortality after a fracture of the hip, the Nottingham Hip Fracture Score and Hip Fracture Estimator of Mortality Amsterdam and to compare their predictive performance to physician assessment of mortality risk in hip fracture patients.

DESIGN

Prospective cohort study.

SETTING

Two level-2 trauma centers located in the Netherlands.

PATIENTS

Two hundred forty-four patients admitted to the Emergency Departments of both hospitals with a fractured hip.

INTERVENTION

Data used in both prediction models were collected at the time of admission for each individual patient, as well as predictions of mortality by treating physicians.

MAIN OUTCOME MEASURES

Predictive performances were evaluated for 30-day, 1-year, and 5-year mortality. Discrimination was assessed with the area under the curve (AUC); calibration with the Hosmer-Lemeshow goodness-of-fit test and calibration plots; clinical usefulness in terms of accuracy, sensitivity, and specificity.

RESULTS

Mortality was 7.4% after 30 days, 22.1% after 1 year, and 59.4% after 5 years. There were no statistically significant differences in discrimination between the prediction methods (AUC 0.73-0.80). The Nottingham Hip Fracture Score demonstrated underfitting for 30-day mortality and failed to identify the majority of high-risk patients (sensitivity 33%). The Hip fracture Estimator of Mortality Amsterdam showed systematic overestimation and overfitting. Physicians were able to identify most high-risk patients for 30-day mortality (sensitivity 78%) but with some overestimation. Both risk models demonstrated a lack of fit when used for 1-year and 5-year mortality predictions.

CONCLUSIONS

In this study, prognostic models and physicians demonstrated similar discriminating abilities when predicting mortality in hip fracture patients. Although physicians overestimated mortality, they were better at identifying high-risk patients and at predicting long-term mortality.

LEVEL OF EVIDENCE

Prognostic Level II. See Instructions for Authors for a complete description of levels of evidence.

Mortality prediction upon hospital admission - the value of clinical assessment: A retrospective, matched cohort study

Accurate prediction of mortality upon hospital admission is of great value, both for the sake of patients and appropriate resources' allocation. A myriad of assessment tools exists for this purpose. The evidence relating to the comparative value of clinical assessment versus established indexes are scarce. We analyzed the accuracy of a senior physician's clinical assessment in a retrospective cohort of patients in a crude, general patients' population and later on a propensity matched patients' population. In one department of internal medicine in a tertiary hospital, of 9891 admitted patients, 973 (10%) were categorized as prone to death in a 6-months' duration by a senior physician. The risk of death was significantly higher for these patients [73.1% vs 14.1% mortality within 180 days; hazard ratio (HR) = 7.58; confidence intervals (CI) 7.02-8.19, P < .001]. After accounting for multiple, other patients' variables associated with increased risk of mortality, the correlation remained significant (HR = 3.25; CI 2.85-3.71, P < .001). We further performed a propensity matching analysis (a subgroup of 710 patients, subdivided to two groups with 355 patients each): survival rates were as low as 45% for patients categorized as prone to death compared to 78% in patients who weren't categorized as such (P < .001). Reliance on clinical evaluation, done by an experienced senior physician, is an appropriate tool for mortality prediction upon hospital admission, achieving high accuracy rates.

Quick sequential organ failure assessment score combined with other sepsis-related risk factors to predict in-hospital mortality: Post-hoc analysis of prospective multicenter study data

This study aimed to assess the value of quick sequential organ failure assessment (qSOFA) combined with other risk factors in predicting in-hospital mortality in patients presenting to the emergency department with suspected infection. This post-hoc analysis of a prospective multicenter study dataset included 34 emergency departments across Japan (December 2017 to February 2018). We included adult patients (age ≥16 years) who presented to the emergency department with suspected infection. qSOFA was calculated and recorded by senior emergency physicians when they suspected an infection. Different types of sepsis-related risk factors (demographic, functional, and laboratory values) were chosen from prior studies. A logistic regression model was used to assess the predictive value of qSOFA for in-hospital mortality in models based on the following combination of predictors: 1) qSOFA-Only; 2) qSOFA+Age; 3) qSOFA+Clinical Frailty Scale (CFS); 4) qSOFA+Charlson Comorbidity Index (CCI); 5) qSOFA+lactate levels; 6) qSOFA+Age+CCI+CFS+lactate levels. We calculated the area under the receiver operating characteristic curve (AUC) and other key clinical statistics at Youden's index, where the sum of sensitivity and specificity is maximized. Following prior literature, an AUC >0.9 was deemed to indicate high accuracy; 0.7-0.9, moderate accuracy; 0.5-0.7, low accuracy; and 0.5, a chance result. Of the 951 patients included in the analysis, 151 (15.9%) died during hospitalization. The AUC for predicting in-hospital mortality was 0.627 (95% confidence interval [CI]: 0.580-0.673) for the qSOFA-Only model. Addition of other variables only marginally improved the model's AUC; the model that included all potentially relevant variables yielded an AUC of only 0.730 (95% CI: 0.687-0.774). Other key statistic values were similar among all models, with sensitivity and specificity of 0.55-0.65 and 0.60-0.75, respectively. In this post-hoc data analysis from a prospective multicenter study based in Japan, combining qSOFA with other sepsis-related risk factors only marginally improved the model's predictive value.

Predicting intensive care unit admission and death for COVID-19 patients in the emergency department using early warning scores

AIMS

To identify the most accurate early warning score (EWS) for predicting an adverse outcome in COVID-19 patients admitted to the emergency department (ED).

METHODS

In adult consecutive patients admitted (March 1-April 15, 2020) to the ED of a major referral centre for COVID-19, we retrospectively calculated NEWS, NEWS2, NEWS-C, MEWS, qSOFA, and REMS from physiological variables measured on arrival. Sensitivity, specificity, positive (PPV) and negative predictive value (NPV), and the area under the receiver operating characteristic (AUROC) curve of each EWS for predicting admission to the intensive care unit (ICU) and death at 48 h and 7 days were calculated.

RESULTS

We included 334 patients (119 [35.6%] females, median age 66 [54-78] years). At 7 days, the rates of ICU admission and death were 56/334 (17%) and 26/334 (7.8%), respectively. NEWS was the most accurate predictor of ICU admission within 7 days (AUROC 0.783 [95% CI, 0.735-0.826]; sensitivity 71.4 [57.8-82.7]%; NPV 93.1 [89.8-95.3]%), while REMS was the most accurate predictor of death within 7 days (AUROC 0.823 [0.778-0.863]; sensitivity 96.1 [80.4-99.9]%; NPV 99.4[96.2-99.9]%). Similar results were observed for ICU admission and death at 48 h. NEWS and REMS were as accurate as the triage system used in our ED. MEWS and qSOFA had the lowest overall accuracy for both outcomes.

CONCLUSION

In our single-centre cohort of COVID-19 patients, NEWS and REMS measured on ED arrival were the most sensitive predictors of 7-day ICU admission or death. EWS could be useful to identify patients with low risk of clinical deterioration.

Prognostic performance of endothelial biomarkers to early predict clinical deterioration of patients with suspected bacterial infection and sepsis admitted to the emergency department

BACKGROUND

The objective of this study was to evaluate the ability of endothelial biomarkers to early predict clinical deterioration of patients admitted to the emergency department (ED) with a suspected sepsis. This was a prospective, multicentre, international study conducted in EDs. Adult patients with suspected acute bacterial infection and sepsis were enrolled but only those with confirmed infection were analysed. The kinetics of biomarkers and organ dysfunction were collected at T0, T6 and T24 hours after ED admission to assess prognostic performances of sVEGFR2, suPAR and procalcitonin (PCT). The primary outcome was the deterioration within 72 h and was defined as a composite of relevant outcomes such as death, intensive care unit admission and/or SOFA score increase validated by an independent adjudication committee.

RESULTS

After adjudication of 602 patients, 462 were analysed including 124 who deteriorated (27%). On admission, those who deteriorated were significantly older (73 [60-82] vs 63 [45-78] y-o, p < 0.001) and presented significantly higher SOFA scores (2.15 ± 1.61 vs 1.56 ± 1.40, p = 0.003). At T0, sVEGFR2 (5794 [5026-6788] vs 6681 [5516-8059], p < 0.0001), suPAR (6.04 [4.42-8.85] vs 4.68 [3.50-6.43], p < 0.0001) and PCT (7.8 ± 25.0 vs 5.4 ± 17.9 ng/mL, p = 0.001) were associated with clinical deterioration. In multivariate analysis, low sVEGFR2 expression and high suPAR and PCT levels were significantly associated with early deterioration, independently of confounding parameters (sVEGFR2, OR = 1.53 [1.07-2.23], p < 0.001; suPAR, OR = 1.57 [1.21-2.07], p = 0.003; PCT, OR = 1.10 [1.04-1.17], p = 0.0019). Combination of sVEGFR2 and suPAR had the best prognostic performance (AUC = 0.7 [0.65-0.75]) compared to clinical or biological variables.

CONCLUSIONS

sVEGFR2, either alone or combined with suPAR, seems of interest to predict deterioration of patients with suspected bacterial acute infection upon ED admission and could help front-line physicians in the triage process.