Index blood tests and national early warning scores within 24 hours of emergency admission can predict the risk of in-hospital mortality: a model development and validation study

An Evaluation of the National Early Warning Score 2 and the Laboratory Data Decision Tree Early Warning Score in Predicting Mortality in Geriatric Patients

BACKGROUND

Due to the high rate of geriatric patient visits, scoring systems are needed to predict increasing mortality rates.

OBJECTIVE

In this study, we aimed to investigate the in-hospital mortality prediction power of the National Early Warning Score 2 (NEWS2) and the Laboratory Data Decision Tree Early Warning Score (LDT-EWS), which consists of frequently performed laboratory parameters.

METHODS

We retrospectively analyzed 651 geriatric patients who visited the emergency department (ED), were not discharged on the same day from ED, and were hospitalized. The patients were categorized according to their in-hospital mortality status. The NEWS2 and LDT-EWS values of these patients were calculated and compared on the basis of deceased and living patients.

RESULTS

Median (interquartile range [IQR]) NEWS2 and LDT-EWS values of the 127 patients who died were found to be statistically significantly higher than those of the patients who survived (NEWS2: 5 [3-8] vs. 3 [1-5]; p < 0.001; LDT-EWS: 8 [7-10] vs. 6 [5-8]; p < 0.001). In the receiver operating characteristic curve analysis, the NEWS2, LDT-EWS, and NEWS2+LDT-EWS-formed by the sum of the two scoring systems-resulted in 0.717, 0.705, and 0.775 area under curve values, respectively.

CONCLUSIONS

The NEWS2 and LDT-EWS were found to be valuable for predicting in-hospital mortality in geriatric patients. The power of the NEWS2 to predict in-hospital mortality increased when used with the LDT-EWS.

Copeptin combined with National Early Warning Score for predicting survival in elderly critical ill patients at emergency department

OBJECTIVE

Copeptin, reflecting vasopressin release, as well as the National Early Warning Score (NEWS), reflecting the severity of critical illness, might qualify for survival prediction in elderly patients with critical illness. This prospective observational study aims at assessing the predictive value of copeptin combined with NEWS on the prognosis of elderly critical ill patients at emergency department (ED).

METHODS

We analyzed serum copeptin levels and the NEWS at admission to the ED in a prospective, single-center, and observational study comprising 205 elderly patients with critical illness. Death within 30 days after admission to the ED was the primary end point.

RESULTS

The serum copeptin levels and the NEWS in the non-survivor patients group were higher than those in the survivor group [30.35 (14.20, 38.91) vs 17.53 (13.01, 25.20), P = 0.001 and 9.0 (7.0-10.0) vs 7.0 (6.0-8.0), P = 0.001]. Multivariate logistic regression analysis showed that copeptin, NEWS and copeptin combined with NEWS were all independent risk factors for 30-day mortality in elderly patients with critical illness. Copeptin, NEWS and copeptin combined with NEWS all performed well in predicting 30-day survival, with area under the ROC curve (AUC) values of 0.766 (95%CI, 0.702-0.822), 0.797 (95%CI, 0.744-0.877) and 0.854 (95%CI, 0.798-0.899) respectively. Using the Z test to compare the areas under the above three curves, copeptin combined with NEWS showed a higher predictive value for 30-day survival (P < 0.05). As we calculated, the optimal cut-off values of copeptin and NEWS using the Youden index were 19.78 pg/mL and 8.5 points, respectively. Risk stratification analysis showed that patients with both copeptin levels higher than 19.78 pg/mL and NEWS points higher than 8.5 points had the highest risk of death.

CONCLUSIONS

Copeptin combined with NEWS have a stronger predictive power on the prognosis of elderly patients with critical illness at ED, comparing to either factor individually.

Performance of universal early warning scores in different patient subgroups and clinical settings: a systematic review

OBJECTIVE

To assess predictive performance of universal early warning scores (EWS) in disease subgroups and clinical settings.

DESIGN

Systematic review.

DATA SOURCES

Medline, CINAHL, Embase and Cochrane database of systematic reviews from 1997 to 2019.

INCLUSION CRITERIA

Randomised trials and observational studies of internal or external validation of EWS to predict deterioration (mortality, intensive care unit (ICU) transfer and cardiac arrest) in disease subgroups or clinical settings.

RESULTS

We identified 770 studies, of which 103 were included. Study designs and methods were inconsistent, with significant risk of bias (high: n=16 and unclear: n=64 and low risk: n=28). There were only two randomised trials. There was a high degree of heterogeneity in all subgroups and in national early warning score (I2=72%-99%). Predictive accuracy (mean area under the curve; 95% CI) was highest in medical (0.74; 0.74 to 0.75) and surgical (0.77; 0.75 to 0.80) settings and respiratory diseases (0.77; 0.75 to 0.80). Few studies evaluated EWS in specific diseases, for example, cardiology (n=1) and respiratory (n=7). Mortality and ICU transfer were most frequently studied outcomes, and cardiac arrest was least examined (n=8). Integration with electronic health records was uncommon (n=9).

CONCLUSION

Methodology and quality of validation studies of EWS are insufficient to recommend their use in all diseases and all clinical settings despite good performance of EWS in some subgroups. There is urgent need for consistency in methods and study design, following consensus guidelines for predictive risk scores. Further research should consider specific diseases and settings, using electronic health record data, prior to large-scale implementation.

PROSPERO REGISTRATION NUMBER

PROSPERO CRD42019143141.

Development and validation of early warning score system: A systematic literature review

OBJECTIVES

This review aims to: 1) evaluate the quality of model reporting, 2) provide an overview of methodology for developing and validating Early Warning Score Systems (EWSs) for adult patients in acute care settings, and 3) highlight the strengths and limitations of the methodologies, as well as identify future directions for EWS derivation and validation studies.

METHODOLOGY

A systematic search was conducted in PubMed, Cochrane Library, and CINAHL. Only peer reviewed articles and clinical guidelines regarding developing and validating EWSs for adult patients in acute care settings were included. 615 articles were extracted and reviewed by five of the authors. Selected studies were evaluated based on the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist. The studies were analyzed according to their study design, predictor selection, outcome measurement, methodology of modeling, and validation strategy.

RESULTS

A total of 29 articles were included in the final analysis. Twenty-six articles reported on the development and validation of a new EWS, while three reported on validation and model modification. Only eight studies met more than 75% of the items in the TRIPOD checklist. Three major techniques were utilized among the studies to inform their predictive algorithms: 1) clinical-consensus models (n = 6), 2) regression models (n = 15), and 3) tree models (n = 5). The number of predictors included in the EWSs varied from 3 to 72 with a median of seven. Twenty-eight models included vital signs, while 11 included lab data. Pulse oximetry, mental status, and other variables extracted from electronic health records (EHRs) were among other frequently used predictors. In-hospital mortality, unplanned transfer to the intensive care unit (ICU), and cardiac arrest were commonly used clinical outcomes. Twenty-eight studies conducted a form of model validation either within the study or against other widely-used EWSs. Only three studies validated their model using an external database separate from the derived database.

CONCLUSION

This literature review demonstrates that the characteristics of the cohort, predictors, and outcome selection, as well as the metrics for model validation, vary greatly across EWS studies. There is no consensus on the optimal strategy for developing such algorithms since data-driven models with acceptable predictive accuracy are often site-specific. A standardized checklist for clinical prediction model reporting exists, but few studies have included reporting aligned with it in their publications. Data-driven models are subjected to biases in the use of EHR data, thus it is particularly important to provide detailed study protocols and acknowledge, leverage, or reduce potential biases of the data used for EWS development to improve transparency and generalizability.

Predicting Adverse Outcomes for Febrile Patients in the Emergency Department Using Sparse Laboratory Data: Development of a Time Adaptive Model

BACKGROUND

A timely decision in the initial stages for patients with an acute illness is important. However, only a few studies have determined the prognosis of patients based on insufficient laboratory data during the initial stages of treatment.

OBJECTIVE

This study aimed to develop and validate time adaptive prediction models to predict the severity of illness in the emergency department (ED) using highly sparse laboratory test data (test order status and test results) and a machine learning approach.

METHODS

This retrospective study used ED data from a tertiary academic hospital in Seoul, Korea. Two different models were developed based on laboratory test data: order status only (OSO) and order status and results (OSR) models. A binary composite adverse outcome was used, including mortality or hospitalization in the intensive care unit. Both models were evaluated using various performance criteria, including the area under the receiver operating characteristic curve (AUC) and balanced accuracy (BA). Clinical usefulness was examined by determining the positive likelihood ratio (PLR) and negative likelihood ratio (NLR).

RESULTS

Of 9491 eligible patients in the ED (mean age, 55.2 years, SD 17.7 years; 4839/9491, 51.0% women), the model development cohort and validation cohort included 6645 and 2846 patients, respectively. The OSR model generally exhibited better performance (AUC=0.88, BA=0.81) than the OSO model (AUC=0.80, BA=0.74). The OSR model was more informative than the OSO model to predict patients at low or high risk of adverse outcomes (P<.001 for differences in both PLR and NLR).

CONCLUSIONS

Early-stage adverse outcomes for febrile patients could be predicted using machine learning models of highly sparse data including test order status and laboratory test results. This prediction tool could help medical professionals who are simultaneously treating the same patient share information, lead dynamic communication, and consequently prevent medical errors.

Combining National Early Warning Score With Soluble Urokinase Plasminogen Activator Receptor (suPAR) Improves Risk Prediction in Acute Medical Patients: A Registry-Based Cohort Study

Supplemental Digital Content is available in the text.

Objectives:

Soluble urokinase plasminogen activator receptor is a prognostic biomarker associated with critical illness, disease progression, and risk of mortality. We aimed to evaluate whether soluble urokinase plasminogen activator receptor adds prognostic value to a vital sign-based score for clinical monitoring of patient risk (National Early Warning Score) in acute medical patients.

Design:

Registry-based observational cohort study of consecutively admitted acute medical patients.

Setting:

The Acute Medical Unit, Copenhagen University Hospital Amager and Hvidovre, Hvidovre, Denmark.

Patients:

Acute medical patients admitted between November 18, 2013, and September 30, 2015.

Interventions:

None.

Measurements and Main Results:

Of 17,312 included patients, admission National Early Warning Score was available for 16,244 (93.8%). During follow-up, 587 patients (3.4%) died in-hospital, 859 (5.0%) within 30 days, and 1,367 (7.9%) within 90 days. High soluble urokinase plasminogen activator receptor was significantly associated with in-hospital-, 30-day-, and 90-day mortality within all National Early Warning Score groups, in particular in patients with a low National Early Warning Score; for 30-day mortality, mortality rate ratios ranged from 3.45 (95% CI, 2.91–4.10) for patients with National Early Warning Score 0–1, to 1.86 (95% CI, 1.47–2.34) for patients with National Early Warning Score greater than or equal to 9 for every doubling in soluble urokinase plasminogen activator receptor (log₂-transformed). Combining National Early Warning Score, age, and sex with soluble urokinase plasminogen activator receptor improved prediction of in-hospital-, 30-day-, and 90-day mortality, increasing the area under the curve (95% CI) for 30-day mortality from 0.86 (0.85–0.87) to 0.90 (0.89–0.91), p value of less than 0.0001, with a negative predictive value of 99.0%.

Conclusions:

The addition of soluble urokinase plasminogen activator receptor to National Early Warning Score significantly improved risk prediction of both low- and high-risk acute medical patients. Patients with low National Early Warning Score but elevated soluble urokinase plasminogen activator receptor had mortality risks comparable to that of patients with higher National Early Warning Score.

Development and validation of a novel computer-aided score to predict the risk of in-hospital mortality for acutely ill medical admissions in two acute hospitals using their first electronically recorded blood test results and vital signs: a cross-sectional study

OBJECTIVES

There are no established mortality risk equations specifically for emergency medical patients who are admitted to a general hospital ward. Such risk equations may be useful in supporting the clinical decision-making process. We aim to develop and externally validate a computer-aided risk of mortality (CARM) score by combining the first electronically recorded vital signs and blood test results for emergency medical admissions.

DESIGN

Logistic regression model development and external validation study.

SETTING

Two acute hospitals (Northern Lincolnshire and Goole NHS Foundation Trust Hospital (NH)-model development data; York Hospital (YH)-external validation data).

PARTICIPANTS

Adult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic National Early Warning Score(s) and blood test results recorded on admission.

RESULTS

The risk of in-hospital mortality following emergency medical admission was 5.7% (NH: 1766/30 996) and 6.5% (YH: 1703/26 247). The C-statistic for the CARM score in NH was 0.87 (95% CI 0.86 to 0.88) and was similar in an external hospital setting YH (0.86, 95% CI 0.85 to 0.87) and the calibration slope included 1 (0.97, 95% CI 0.94 to 1.00).

CONCLUSIONS

We have developed a novel, externally validated CARM score with good performance characteristics for estimating the risk of in-hospital mortality following an emergency medical admission using the patient's first, electronically recorded, vital signs and blood test results. Since the CARM score places no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

Predicting in-hospital mortality and unanticipated admissions to the intensive care unit using routinely collected blood tests and vital signs: Development and validation of a multivariable model

AIM

The National Early Warning System (NEWS) is based on vital signs; the Laboratory Decision Tree Early Warning Score (LDT-EWS) on laboratory test results. We aimed to develop and validate a new EWS (the LDTEWS:NEWS risk index) by combining the two and evaluating the discrimination of the primary outcome of unanticipated intensive care unit (ICU) admission or in-hospital mortality, within 24 h.

METHODS

We studied emergency medical admissions, aged 16 years or over, admitted to Oxford University Hospitals (OUH) and Portsmouth Hospitals (PH). Each admission had vital signs and laboratory tests measured within their hospital stay. We combined LDT-EWS and NEWS values using a linear time-decay weighting function imposed on the most recent blood tests. The LDTEWS:NEWS risk index was developed using data from 5 years of admissions to PH, and validated on a year of data from both PH and OUH. We tested the risk index's ability to discriminate the primary outcome using the c-statistic.

RESULTS

The development cohort contained 97,933 admissions (median age = 73 years) of which 4723 (4.8%) resulted inhospital death and 1078 (1.1%) in unanticipated ICU admission. We validated the risk index using data from PH (n = 21,028) and OUH (n = 16,383). The risk index showed a higher discrimination in the validation sets (c-statistic value (95% CI)) (PH, 0.901 (0.898-0.905); OUH, 0.916 (0.911-0.921)), than NEWS alone (PH, 0.877 (0.873-0.882); OUH, 0.898 (0.893-0.904)).

CONCLUSIONS

The LDTEWS:NEWS risk index increases the ability to identify patients at risk of deterioration, compared to NEWS alone.

Development and validation of a prognostic model for predicting 30-day mortality risk in medical patients in emergency department (ED)

The primary aim of this prospective study is to develop and validate a new prognostic model for predicting the risk of mortality in Emergency Department (ED) patients. The study involved 1765 patients in the development cohort and 1728 in the validation cohort. The main outcome was mortality up to 30 days after admission. Potential risk factors included clinical characteristics, vital signs, and routine haematological and biochemistry tests. The Bayesian Model Averaging method within the Cox’s regression model was used to identify independent risk factors for mortality. In the development cohort, the incidence of 30-day mortality was 9.8%, and the following factors were associated with a greater risk of mortality: male gender, increased respiratory rate and serum urea, decreased peripheral oxygen saturation and serum albumin, lower Glasgow Coma Score, and admission to intensive care unit. The area under the receiver operating characteristic curve for the model with the listed factors was 0.871 (95% CI, 0.844–0.898) in the development cohort and 0.783 (95% CI, 0.743–0.823) in the validation cohort. Calibration analysis found a close agreement between predicted and observed mortality risk. We conclude that the risk of mortality among ED patients could be accurately predicted by using common clinical signs and biochemical tests.

Association between routine laboratory tests and long-term mortality among acutely admitted older medical patients: a cohort study

BACKGROUND

Older people have the highest incidence of acute medical admissions. Old age and acute hospital admissions are associated with a high risk of adverse health outcomes after discharge, such as reduced physical performance, readmissions and mortality. Hospitalisations in this population are often by acute admission and through the emergency department. This, along with the rapidly increasing proportion of older people, warrants the need for clinically feasible tools that can systematically assess vulnerability in older medical patients upon acute hospital admission. These are essential for prioritising treatment during hospitalisation and after discharge. Here we explore whether an abbreviated form of the FI-Lab frailty index, calculated as the number of admission laboratory test results outside of the reference interval (FI-OutRef) was associated with long term mortality among acutely admitted older medical patients. Secondly, we investigate other markers of aging (age, total number of chronic diagnoses, new chronic diagnoses, and new acute admissions) and their associations with long-term mortality.

METHODS

A cohort study of acutely admitted medical patients aged 65 or older. Survival time within a 3 years post-discharge follow up period was used as the outcome. The associations between the markers and survival time were investigated by Cox regression analyses. For analyses, all markers were grouped by quartiles.

RESULTS

A total of 4,005 patients were included. Among the 3,172 patients without a cancer diagnosis, mortality within 3 years was 39.9%. Univariate and multiple regression analyses for each marker showed that all were significantly associated with post-discharge survival. The changes between the estimates for the FI-OutRef quartiles in the univariate- and the multiple analyses were negligible. Among all the markers investigated, FI-OutRef had the highest hazard ratio of the fourth quartile versus the first quartile: 3.45 (95% CI: 2.83-s4.22, P < 0.001).

CONCLUSION

Among acutely admitted older medical patients, FI-OutRef was strongly associated with long-term mortality. This association was independent of age, sex, and number of chronic diagnoses, new chronic diagnoses, and new acute admissions. Hence FI-OutRef could be a biomarker of advancement of aging within the acute care setting.

Should predictive scores based on vital signs be used in the same way as those based on laboratory data? A hypothesis generating retrospective evaluation of in-hospital mortality by four different scoring systems

BACKGROUND

few studies have compared the discrimination of predictive scores of in-hospital mortality that used vital signs with those using laboratory results in different patient populations.

METHODS

a hypothesis generating retrospective observational cohort study. A score that only used vital signs was compared with three other scores that used laboratory changes in 44,985 medical and 20,432 surgical patients.

RESULTS

the discrimination of the score based only on vital signs was highest for the prediction of in-hospital death within 24h. In contrast the, albeit lower, discrimination of scores based only on laboratory data remained constant for the prediction of death up to 30 days after hospital admission. Moreover, the discrimination of scores based only on laboratory data was higher in surgical than in medical patients.

CONCLUSION

in acutely ill medical patients a vital sign based score appears to predict mortality within 24h better than scores using laboratory data. This may be because in acutely ill patients vital sign changes indicate how well a patient is responding to a current insult. In contrast, for patients without acute illness laboratory data may be a more valuable indication of the patient's capacity to respond to insults in the future.

Validation of the National Early Warning Score in the prehospital setting

BACKGROUND

Early intervention and response to deranged physiological parameters in the critically ill patient improves outcomes. A National Early Warning Score (NEWS) based on physiological observations has been developed for use throughout the National Health Service (NHS) in the UK. Although a good predictor of mortality and deterioration in inpatients, its performance in the prehospital setting is largely untested. This study aimed to assess the validity of the NEWS in unselected prehospital patients.

METHODS

All clinical observations taken by emergency ambulance crews transporting patients to a single hospital were collated along with information relating to hospital outcome over a two month period. The performance of the NEWS in identifying the endpoints of 48h and 30 day mortality, intensive care unit (ICU) admission, and a combined endpoint of 48h mortality or ICU admission was analysed.

RESULTS

1684 patients were analysed. All three of the primary endpoints and the combined endpoint were associated with higher NEWS scores (p<0.01 for each). The medium-risk NEWS group was associated with a statistically significant increase in ICU admission (RR=2.466, 95% CI 1.0-6.09), but not in-hospital mortality relative to the low risk group. The high risk NEWS group had significant increases in 48h mortality (RR 35.32 [10.08-123.7]), 30 day mortality (RR 6.7 [3.79-11.88]), and ICU admission (5.43 [2.29-12.89]). Similar results were noted when trauma and non-trauma patients were analysed separately.

CONCLUSIONS

Elevated NEWS among unselected prehospital patients is associated with a higher incidence of adverse outcomes. Calculation of prehospital NEWS may facilitate earlier recognition of deteriorating patients, early involvement of senior Emergency Department staff and appropriate critical care.

Establishing an evidence base for critical laboratory value thresholds

OBJECTIVES

Critical values denote laboratory test results indicating a life-threatening situation. The outcomes of this premise have not been rigorously evaluated.

METHODS

Five years of inpatient admissions were examined for critical or "near-critical" results (total admissions = 165,066; total test results = 872,503). In-hospital mortality was examined as a function of time and degree of test result abnormality.

RESULTS

Some critical value thresholds appropriately identified patients at risk for death (eg, elevated potassium). Other thresholds were too conservative (elevated hematocrit, hemoglobin) or not conservative enough (elevated lactate). Mortality risk for most critical values was time dependent, but some critical values showed no temporal effect on mortality (elevated activated partial thromboplastin time [APTT], international normalized ratio [INR], and glucose). Following an initial critical result, further worsening was associated with increased mortality. Prior hospital admission within 30 days was a predictor of lower mortality for some (elevated APTT, INR, potassium, and sodium; low glucose, hematocrit, hemoglobin, and potassium) but not other critical values (elevated lactate, glucose, hematocrit, and hemoglobin; low sodium).

CONCLUSIONS

Only a subset of laboratory critical value thresholds was optimally chosen for increased risk of in-hospital mortality, with a time urgency for most but not all critical values. For many tests, a prior hospital admission imparted a decreased risk of in-hospital death.