Triage of patients with fever: The Manchester triage system's predictive validity for sepsis or septic shock and seven-day mortality

Interpretable machine learning for predicting sepsis risk in emergency triage patients

The study aimed to develop and validate a sepsis prediction model using structured electronic medical records (sEMR) and machine learning (ML) methods in emergency triage. The goal was to enhance early sepsis screening by integrating comprehensive triage information beyond vital signs. This retrospective cohort study utilized data from the MIMIC-IV database. Two models were developed: Model 1 based on vital signs alone, and Model 2 incorporating vital signs, demographic characteristics, medical history, and chief complaints. Eight ML algorithms were employed, and model performance was evaluated using metrics such as AUC, F1 Score, and calibration curves. SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) methods were used to enhance model interpretability. The study included 189,617 patients, with 5.95% diagnosed with sepsis. Model 2 consistently outperformed Model 1 across most algorithms. In Model 2, Gradient Boosting achieved the highest AUC of 0.83, followed by Extra Tree, Random Forest, and Support Vector Machine (all 0.82). The SHAP method provided more comprehensible explanations for the Gradient Boosting algorithm. Modeling with comprehensive triage information using sEMR and ML methods was more effective in predicting sepsis at triage compared to using vital signs alone. Interpretable ML enhanced model transparency and provided sepsis prediction probabilities, offering a feasible approach for early sepsis screening and aiding healthcare professionals in making informed decisions during the triage process.

Validation and comparison of triage-based screening strategies for sepsis

OBJECTIVE

This study sought to externally validate and compare proposed methods for stratifying sepsis risk at emergency department (ED) triage.

METHODS

This nested case/control study enrolled ED patients from four hospitals in Utah and evaluated the performance of previously-published sepsis risk scores amenable to use at ED triage based on their area under the precision-recall curve (AUPRC, which balances positive predictive value and sensitivity) and area under the receiver operator characteristic curve (AUROC, which balances sensitivity and specificity). Score performance for predicting whether patients met Sepsis-3 criteria in the ED was compared to patients' assigned ED triage score (Canadian Triage Acuity Score [CTAS]) with adjustment for multiple comparisons.

RESULTS

Among 2000 case/control patients, 981 met Sepsis-3 criteria on final adjudication. The best performing sepsis risk scores were the Predict Sepsis version #3 (AUPRC 0.183, 95 % CI 0.148-0.256; AUROC 0.859, 95 % CI 0.843-0.875) and Borelli scores (AUPRC 0.127, 95 % CI 0.107-0.160, AUROC 0.845, 95 % CI 0.829-0.862), which significantly outperformed CTAS (AUPRC 0.038, 95 % CI 0.035-0.042, AUROC 0.650, 95 % CI 0.628-0.671, p < 0.001 for all AUPRC and AUROC comparisons). The Predict Sepsis and Borelli scores exhibited sensitivity of 0.670 and 0.678 and specificity of 0.902 and 0.834, respectively, at their recommended cutoff values and outperformed Systemic Inflammatory Response Syndrome (SIRS) criteria (AUPRC 0.083, 95 % CI 0.070-0.102, p = 0.052 and p = 0.078, respectively; AUROC 0.775, 95 % CI 0.756-0.795, p < 0.001 for both scores).

CONCLUSIONS

The Predict Sepsis and Borelli scores exhibited improved performance including increased specificity and positive predictive values for sepsis identification at ED triage compared to CTAS and SIRS criteria.

Enhancing triage accuracy: The influence of nursing education on risk prediction

AIM

This study aimed to compare the performance in risk prediction of various outcomes between specially trained triage nurses and the Manchester Triage System (MTS).

DESIGN

Prospective observational study.

METHODS

The study was conducted from June 1st to December 31st, 2023, at the Emergency Department of Merano Hospital. Triage nurses underwent continuous training through dedicated courses and daily audits. We compared the risk stratification performed by expert nurses with that of MTS on various outcomes such as mortality, hospitalisation, and urgency defined by the physicians. Comparisons were made using the Areas Under the Receiver Operating Characteristic curve (AUROC).

RESULTS

The agreement in code classification between the MTS and the expert nurse was very low. The AUROC curve analysis showed that the expert nurse outperformed the MTS in all outcomes. The triage nurse's experience led to statistically significant better stratification in admission rates, ICU admissions, and all outcomes based on the physician's assessment.

CONCLUSIONS

The continuous training of nurses enables them to achieve better risk prediction compared to standardized triage systems like MTS, emphasizing the utility and necessity of implementing continuous training pathways for these highly specialised personnel.

Human intelligence versus Chat-GPT: who performs better in correctly classifying patients in triage?

INTRODUCTION

Chat-GPT is rapidly emerging as a promising and potentially revolutionary tool in medicine. One of its possible applications is the stratification of patients according to the severity of clinical conditions and prognosis during the triage evaluation in the emergency department (ED).

METHODS

Using a randomly selected sample of 30 vignettes recreated from real clinical cases, we compared the concordance in risk stratification of ED patients between healthcare personnel and Chat-GPT. The concordance was assessed with Cohen's kappa, and the performance was evaluated with the area under the receiver operating characteristic curve (AUROC) curves. Among the outcomes, we considered mortality within 72 h, the need for hospitalization, and the presence of a severe or time-dependent condition.

RESULTS

The concordance in triage code assignment between triage nurses and Chat-GPT was 0.278 (unweighted Cohen's kappa; 95% confidence intervals: 0.231-0.388). For all outcomes, the ROC values were higher for the triage nurses. The most relevant difference was found in 72-h mortality, where triage nurses showed an AUROC of 0.910 (0.757-1.000) compared to only 0.669 (0.153-1.000) for Chat-GPT.

CONCLUSIONS

The current level of Chat-GPT reliability is insufficient to make it a valid substitute for the expertise of triage nurses in prioritizing ED patients. Further developments are required to enhance the safety and effectiveness of AI for risk stratification of ED patients.

Diagnosis of sepsis in the emergency department: a real-world analysis from Lombardy, Italy

Early identification of sepsis is particularly important in the emergency department (ED). However, data on the diagnosis of sepsis in the ED are scanty, especially within the Italian context. To quantify sepsis incidence and recognition in the ED from Lombardy, Italy, we used EUOL data from the Regional Emergency Agency for the years 2017-2022. Sepsis was identified based on the ED discharge diagnosis; recognized sepsis cases were those assigned to a high-priority code at triage, while unrecognized ones were those assigned to a low priority code. Odds ratios (ORs) for sepsis recognition according to various patient characteristics were estimated using multivariable mixed-effects logistic regression models. The rate of sepsis diagnosis in ED was 1.9 per 1000 (6626 patients) in 2017 and increased to 3.4 per 1000 in 2022 (11,508 patients). In 2022, 67% of sepsis cases were correctly identified. Death in the ED was more frequent in patients with recognized sepsis (10.4%) than in those with unrecognized sepsis (2.3%). The probability of sepsis being recognized at ED admission was higher in men (multivariable OR: 1.06), in individuals with advanced age (OR: 1.71 for age ≥ 90 years vs < 60), and in those with access to the second (OR: 1.48) and third ED level (OR: 1.87). Conversely, it was lower in patients arriving at the ED through autonomous transportation (OR: 0.36). This large real-world analysis indicates an increase in sepsis cases referred to the ED in recent years. About one-third of sepsis cases are not correctly identified at triage, although more severe cases appear to be promptly recognized.

Electrocardiogram interpretation during nurse triage improves the performance of the triage system in patients with cardiovascular symptoms - A prospective observational study

BACKGROUND

An immediate ECG on arrival of a patient with cardiovascular symptoms in the ED may anticipate the need for life-saving intervention. The aim was to evaluate whether ECG interpretation during nurse triage can improve triage system performance in patients with cardiovascular symptoms.

METHODS

All patients who required an assessment for cardiovascular symptoms were considered for this observational study. During triage assessment, the nurses assessed the patient's level of urgency applying the MTS, then again after this evaluation (confirming or modifying the level of urgency based on personal clinical experience) and after interpretation of the patient's ECG. The main study outcome was the diagnosis of an acute cardiovascular event.

RESULTS

Of the 1211 patients in the study, 10.5% presented the main study outcome. ECG interpretation in triage exhibited a nurse-physician agreement of 92.9% (p<0.001). increased patient priority in 7.5% of cases and reduced it in 39.6%. The discriminatory ability of the triage system had an area under the ROC of 0.712and 0.845 after ECG interpretation. ECG interpretation improved the baseline assessment of priority, with an NRI of 60.1% (p<0.001).

CONCLUSIONS

ECG interpretation in triage can be a simple and safe tool that improves the assessment of patient priority.

Daily triage audit can improve nurses' triage stratification: A pre-post study

AIMS

The objective was to evaluate whether the error rate in the application of the triage system decreased after the introduction of daily auditing, and it was also evaluated if the agreement rate between physician and nurse on triage priority levels increased after the introduction of daily auditing and if the error-related variables in the pre-intervention period changed in the post-intervention period.

DESIGN

A quasi-experimental study was performed with a pre-post design, between June 2019 and June 2021 in one emergency department.

METHODS

The accuracy and error rate of triage in the pre- and post-intervention period were compared. Univariate and multivariate logistic regression analyses were performed to explore the relationships between the variables related to the error. The comparison between the priority level assigned by the physician and the triage nurse was analysed using Cohen's K.

RESULTS

Nine hundred four patients were enrolled in the pre-intervention period and 869 in the post-intervention period. The error rate in the pre-intervention period was 23.3% and in the post-intervention period was 9.7%. The concordance between the degree of priority expressed by the physician and the nurse varied from a quadratically weighted Cohen's K of 0.447 in the pre-intervention period to 0.881 in the post-intervention period.

CONCLUSION

Daily auditing is a clinical procedure that improves the nurse's application of the triage system. Daily auditing has reduced errors by the nurse, improving performance and concordance with the physician.

IMPACT

Triage systems are a key point for the stratification of the priority level of patients and it is therefore evident that they maintain high-quality standards. Through the practice of daily auditing, not only a reduction in the error rate, which ensures patient safety, but also an improvement in triage performance has been demonstrated.

NO PATIENT OR PUBLIC CONTRIBUTION

The study did not involve any patients during its conduction.

The Manchester Triage System's performance in clinical risk prioritisation of patients presenting with headache in emergency department: A retrospective observational study

OBJECTIVE

Non-traumatic headache is a frequent reason for visits to the emergency department (ED). We evaluated the performance of the Manchester Triage System (MTS) in prioritising patients presenting to the ED with non-traumatic headache.

METHODS

In this single-centre observational retrospective study, we compared the association of MTS priority classification codes with a final diagnosis of a severe neurological condition requiring timely management (ischaemic or haemorrhagic stroke, subarachnoid haemorrhage, cerebral sinus venous thrombosis, central nervous system infection or brain tumour). The study was conducted and reported according to the STROBE statement. The overall prioritisation accuracy of MTS was estimated by the area under the receiver operating characteristic (ROC) curve. The correctness of triage prediction was estimated based on the "very urgent" MTS grouping. An undertriage was defined as a patient with an urgent and severe neurological who received a low priority/urgency MTS code (green/yellow).

RESULTS

Over 30 months, 3002 triage evaluations of non-traumatic headache occurred (1.7% of ED visits). Of these, 2.3% (68/3002) were eventually diagnosed with an urgent and severe neurological condition. The MTS had an acceptable prioritisation accuracy, with an area under the ROC curve of 0.734 (95% CI 0.668-0.799). The sensitivity of the MTS for urgent codes (yellow, orange and red) was 79.4% (95% CI 74.5-84.3), with a specificity of 54.1% (95% CI 52.9-55.3). The triage prediction was incorrect in only 6.3% (190/3002) of patients with headache.

CONCLUSION

The MTS is a safe and accurate tool for prioritising patients with non-traumatic headache in the ED. However, MTS may need further specific tools for evaluating the more complicated symptoms and for correctly identifying patients with urgent and severe underlying pathologies.

RELEVANCE TO CLINICAL PRACTICE

The triage nurse using MTS may need additional tools to improve the assessment of patients with headache, although MTS provides a good safety profile.

Triage assessment of transitory loss of consciousness in the emergency department-A retrospective observational study

AIM

To establish how the Manchester Triage System can correctly prioritize patients admitted to the emergency department for transitory loss of consciousness in relation to their risk of presenting severe acute disease.

DESIGN

A observational retrospective study.

METHODS

A total of 2291 patients who required a triage evaluation for a transitory loss of consciousness at the emergency department of Merano Hospital between 1 January 2017 and 30 June 2019 were considered. Transitory loss of consciousness was classified according to European Society of Cardiology guidelines. The baseline characteristics of the patients were collected and divided according to the priority level assigned at triage into two different study groups: high priority (red/orange) and low priority (blue/green/yellow). The composite outcome of the study was defined as the diagnosis of a severe acute disease.

RESULTS

Of the patients enrolled, 17% (390/2291) had a high-priority code and 83% (1901/2291) received a low-priority code. Overall, a severe acute disease was present in 16.9% of patients (387/2291). The Manchester Triage System had a sensitivity of 42.4%, a specificity of 88.1% and an accuracy of 80.4% for predicting severe acute disease. The discriminatory ability had an area under the receiver operating characteristic curve of 0.651 (CI 95%: 0.618-0.685).

CONCLUSIONS

Despite the good specificity, the low sensitivity does not currently allow the Manchester Triage System to completely exclude patients with a severe acute disease who presented in the emergency department for a transitory loss of consciousness. Therefore, it is important to develop precise nursing tools or assessments that can improve triage performance.

IMPACT

The assessment of a complex symptom can create difficulties in the stratification of patients in triage, assigning low-priority codes to patients with a severe disease. Additional tools are needed to allow the correct triage assessment of patients presenting with transitory loss of consciousness.

The ROX index can be a useful tool for the triage evaluation of COVID-19 patients with dyspnoea

Aim

To assess whether the application of a non‐invasive tool, such as ratio of oxygen saturation (ROX) index, during triage can identify patients with COVID‐19 at high risk of developing acute respiratory distress syndrome (ARDS).

Design

A multi‐centre, observational, retrospective study.

Methods

Only COVID‐19 positive patients who required an emergency department evaluation for dyspnoea were considered. The primary objective of the study was to compare the ROX value obtained during triage with the medical diagnosis of ARDS and intubation in 72 h of the triage evaluation. The ROX index value was also compared with objective outcomes, such as the pressure of arterial O₂ (PaO₂)/fraction of inspired oxygen (FiO₂) ratio and the lung parenchyma volume involved in COVID‐19‐related inflammatory processes, based on 3D reconstructions of chest computed tomography (CT).

Results

During the study period, from 20 March 2020 until 31 May 2020, a total of 273 patients with confirmed SARS‐CoV‐2 infection were enrolled. The predictive ability of ROX for the risk of developing ARDS in 72 h after triage evaluation was associated with an area under the receiver operating characteristic (AUROC) of 0.845 (0.797–0.892, p < 0.001), whereas the AUROC value was 0.727 (0.634–0.821, p < 0.001) for the risk of intubation. ROX values were strongly correlated with PaO₂/FiO₂ values (r = 0.650, p < 0.001), decreased ROX values were associated with increased percentages of lung involvement based on 3D CT reconstruction (r = −0.371, p < 0.001).

Conclusion

The ROX index showed a good ability to identify triage patients at high evolutionary risk. Correlations with objective but more invasive indicators (PaO₂/FiO₂ and CT) confirmed the important role of ROX in identifying COVID‐19 patients with extensive pathological processes.

Impact

During the difficult triage evaluation of COVID‐19 patients, the ROX index can help the nurse to identify the real severity of the patient. The triage systems could integrate the ROX in the rapid patient assessment to stratify patients more accurately.

Acute abdominal pain in triage: A retrospective observational study of the Manchester triage system's validity

OBJECTIVE

Roughly 5% to 10% of patients admitted to the emergency department suffer from acute abdominal pain. Triage plays a key role in patient stratification, identifying patients who need prompt treatment versus those who can safely wait. In this regard, the aim of this study was to estimate the performance of the Manchester Triage System in classifying patients with acute abdominal pain.

METHODS

A total of 9,851 patients admitted at the Emergency Department of the Merano Hospital with acute abdominal pain were retrospectively enrolled between 1 January 2017 and 30 June 2019. The study was conducted and reported according to the STROBE statement. The sensitivity and specificity of the Manchester Triage System were estimated by verifying the triage classification received by the patients and their survival at seven days or the need for acute surgery within 72 h after emergency department access.

RESULTS

Among the patients with acute abdominal pain (median age 50 years), 0.4% died within seven days and 8.9% required surgery within 72 hours. The sensitivity was 44.7% (29.9-61.5), specificity was 95.4% (94.9-95.8), and negative predictive value was 99.7% (99.2-100) in relation to death at seven days.

CONCLUSIONS

The Manchester Triage System shows good specificity and negative predictive value. However, its sensitivity was low due to the amount of incorrect triage prediction in patients with high-priority codes (red/orange), suggesting overtriage in relation to seven-day mortality. This may be a protective measure for the patient. In contrast, the need for acute surgery within 72 h was affected by under-triage.

RELEVANCE TO CLINICAL PRACTICE

The triage nurse using Manchester Triage System can correctly prioritise the majority of patients with acute abdominal pain, especially in low acuity patients. The Manchester Triage System is safe and does not underestimate the severity of the patients.