Assessing sensitivity and specificity of the Manchester Triage System in the evaluation of acute coronary syndrome in adult patients in emergency care: a systematic review

The Diagnostic Evaluation of the SINEH Cardiopulmonary Triage Scale and the Emergency Severity Index in the Emergency Department: A Comparative Study

Introduction

The diagnostic evaluation of the emergency severity index (ESI) in the triage of patients with cardiopulmonary complaints has a high sensitivity but a low specificity in the emergency department (ED). Therefore, triage scales with more accurate diagnostic evaluation are required. As a result, accuracy of the SINEH triage scale (SinTS) and the ESI was compared to compare mistriage of critically ill patients with cardiopulmonary complaints.

Methods

This descriptive, analytical and cross-sectional study was conducted between December 2022 and April 2023. In this study, two nurses independently examined each patient using two triage scales. The admission unit and length of hospital stay were also recorded. The outcome was classified as high-risk admission (cardiac care unit and intensive care unit) and low-risk admission (internal unit or discharge from the ED). Undertriage and overtiage were defined as high-risk admission with triage level 3 and 4 and low-risk admission with triage level 1 or 2, respectively. A panel of experts evaluated content validity of SinTS and kappa designating agreement on relevance reported. The inter-rater reliability of two scales was also reported.

Results

Finally, the study included 145 patients. The average age of the patients studied was 61.35 years. SinTS has a total mistriage of 29.63%, with 4.13% being undertriage and 25.5% being overtriage. In ESI, the total mistriage is 66.8%, with 1.3% being undertriage and 65.5% being overtriage. The undertriage of the two scales did not differ significantly by admission unit (p=0.26), but the overtriage of the two methods did (p=0.001). The sensitivity, specificity, and accuracy of SinTS were 86.3%, 63.37%, and 72.27%, respectively, while those of ESI were 95.4%, 5.94%, and 32.79%, respectively.

Conclusion

SINEH triage scale has achieved the optimal accuracy in recognizing the acuity of the patients with chest pain and dyspnea by using SpO2, pressure of end-tidal carbon dioxide, troponin I, and peak expiratory flow. When triaging patients with chest pain and dyspnea, SinTS may exhibit a higher level of accuracy compared to ESI. More research is needed to improve accuracy of triage scales in patient with cardiopulmonary complaints.

Determining factors of service priority in the risk classification of patients with chest pain

OBJECTIVE

To verify the determining factors of high priority in the risk classification and the outcomes of the care provided to adult patients with chest pain.

METHOD

Retrospective study, conducted at an emergency referral service of a public hospital in the interior of São Paulo State, analyzing the records of care performed in the risk classification in 181 medical records of patients with the symptom of chest pain, between August and November 2020.

RESULTS

Individuals were most males 98 (54.1%), with moderate pain 133 (73.5), high priority for care 135 (74.5%) and who sought the service spontaneously 139 (76.8%). Of those classified as high priority, 47 (34.8%) were referred to the emergency room and, of these, 27 (17.0%) remained hospitalized. Female gender (p=0.0198; OR=0.40; CI=0.189-0.866) was independently associated with high priority of care.

CONCLUSION

Despite the priority classification, few participants were referred to the emergency room and required hospitalization. Female gender was a protective factor in the risk classification as high priority.

An AI-Enabled Dynamic Risk Stratification for Emergency Department Patients with ECG and CXR Integration

Emergency department (ED) triage scale determines the priority of patient care and foretells the prognosis. However, the information retrieved from the initial assessment is limited, hindering the risk identification accuracy of triage. Therefore, we sought to develop a 'dynamic' triage system as secondary screening, using artificial intelligence (AI) techniques to integrate information from initial assessment data and subsequent examinations. This retrospective cohort study included 134,112 ED visits with at least one electrocardiography (ECG) and chest X-ray (CXR) in a medical center from 2012 to 2022. Additionally, an independent community hospital provided 45,614 ED visits as an external validation set. We trained an eXtreme gradient boosting (XGB) model using initial assessment data to predict all-cause mortality in 7 days. Two deep learning models (DLMs) using ECG and CXR were trained to stratify mortality risks. The dynamic triage levels were based on output from the XGB-triage and DLMs from ECG and CXR. During the internal and external validation, the area under the receiver operating characteristic curve (AUC) of the XGB-triage model was >0.866; furthermore, the AUCs of DLMs using ECG and CXR were >0.862 and >0.886, respectively. The dynamic triage scale provided a higher C-index (0.914-0.920 vs. 0.827-0.843) than the original one and demonstrated better predictive ability for 5-year mortality, 30-day ED revisit, and 30-day discharge. The AI-based risk scale provides a more accurate and dynamic stratification of mortality risk in ED patients, particularly in identifying patients who tend to be overlooked due to atypical symptoms.

Myocardial injury defined as elevated high-sensitivity cardiac troponin T is associated with higher mortality in patients seeking care at emergency departments with acute dyspnea

BACKGROUND

Elevated levels of cardiac troponin T has been observed in patients seeking care at the emergency department (ED) presenting with chest pain but without myocardial infarction (MI). The clinical importance of this observation remains, however, still unclear. Our main aim was to study the role of cardiac troponin T in patients admitted to the emergency department with acute dyspnea, a group of patients with a high cardiovascular comorbidity, but no primary acute MI.

POPULATION AND METHODS

Patients from the age of 18 seeking care at the ED for dyspnea, without an acute cardiac syndrome, and with a recorded assessment of high-sensitivity cardiac troponin T (hs-cTnT), were included (n = 1001). Patients were categorized into 3 groups by hs-cTnT level, i.e. <15, 15-100 and > 100 µg/l. Cox regression with Hazard Ratios (HRs) and 95% Confidence Intervals (CI) for 3-months mortality was performed, with adjustment for sex, age, respiratory frequency, saturation, CHF, renal disease, and BMI.

RESULTS

Fully adjusted HRs (95% CI) for 3-month mortality, with hs-cTnT < 15 µg/l as reference level, showed for hs-cTnT 15-100 a HR of 3.682 (1.729-7.844), and for hs-cTnT > 100 a HR of 10.523 (4.465-24.803).

CONCLUSION

Elevated hs-cTnT seems to be a relevant marker of poor prognosis in patients with acute dyspnea without MI and warrants further validation and clinical testing.

Implementation of an Evidence-Based Prenotification Process for Patients With Stroke to Improve Neurological Outcomes

ABSTRACT

BACKGROUND: Endovascular mechanical thrombectomy can improve clinical outcomes in eligible patients with acute ischemic stroke (AIS), but its efficacy is time dependent. This quality improvement project aimed to examine whether a revised evidence-based neurological deficit algorithm initiated in the emergency department could reduce door-to-groin puncture time to less than 90 minutes and improve neurological outcomes in AIS patients who received mechanical thrombectomy. METHODS: Retrospective chart reviews occurred between September 2020 and April 2021, which included 25 patients. Clinical and time data were collected from AIS patients who were 18 years and older, presented for care with AIS symptoms, and deemed candidates for thrombectomy for a period of 6 months. A revised neurological deficit algorithm was initiated, and education was presented to appropriate staff. Postintervention chart reviews occurred from August 2021 to January 2022, which included 25 patients. RESULTS: Door-to-groin puncture time did not improve to less than 90 minutes; however, there was a slight improvement in time from 106 minutes in the preintervention to 98 minutes in the postintervention (P = .534). Although the outcome measures were not clinically significant, there was a statistically significant decrease in response time to acute stroke call down (P < .01). Door-to-computed tomography also improved, which was 14.42 minutes for the preintervention group and 5.25 minutes for the postintervention group (P < .001). Finally, the mean National Institutes of Health Stroke Scale on discharge for the preintervention group was 11.92, and that of the postintervention group was 6.05 on discharge (P < .01). CONCLUSIONS: Implementation of the revised neurological deficit algorithm did not decrease the door-to-groin puncture time to less than 90 minutes. After implementation of the revised neurologic deficit algorithm, there were single variable improvements in several benchmarks, and this is a starting point for future quality improvement projects.

Improving the performance of a triage scale for chest pain patients admitted to emergency departments: combining cardiovascular risk factors and electrocardiogram

Background

The triage of patients presenting with chest pain on admission to the emergency department uses scales based on patient clinical presentation or an electrocardiogram (ECG). These scales have different sensitivity and specificity. Although a good sensitivity allows for the prompt identification of high-risk patients, specificity prevent ED overcrowding. Moreover, ECG at triage avoids missing ST elevation myocardial infarction, which requires urgent revascularization. Our study therefore aimed to investigate whether a scale combining ECG and cardiovascular risk factors (CVRF) improves the diagnostic performance of ED chest pain triage scale.

Methods and results

In this prospective single-center observational study involving 505 patients, the standard ECG-based FRENCH scale was compared to a scale combining the ECG-based FRENCH scale and the patients CVRF. The new scale was called the “modified” FRENCH. The accuracy of patient CVRF collection was evaluated by comparing the results of triage nurses and ED physicians.

Compared with the standard FRENCH scale, the modified FRENCH scale had an increased sensitivity (61% versus 75%) but a decrease in specificity (76% versus 64%) resulting in a similar diagnostic performance. Using CVRF collected by the ED physicians, the modified FRENCH scale had a sensitivity of 87% and a specificity of 56% with a significant improvement in his diagnostic performance compared with standard FRENCH scales. This improvement can be explained by an accurate collection of the CVRF by physicians compared with nurses, as suggested by the weak to moderate correlation between their respective data collection.

Conclusion

In conclusion, combining ECG and accurately collected cardiovascular risks factor improves the diagnostic performance of the ECG based chest pain triage in the ED.

Trial registration

Trial registration number: NCT03913767.

Development and Validation of an Artificial Intelligence Electrocardiogram Recommendation System in the Emergency Department

The machine learning-assisted electrocardiogram (ECG) is increasingly recognized for its unprecedented capabilities in diagnosing and predicting cardiovascular diseases. Identifying the need for ECG examination early in emergency department (ED) triage is key to timely artificial intelligence-assisted analysis. We used machine learning to develop and validate a clinical decision support tool to predict ED triage patients’ need for ECG. Data from 301,658 ED visits from August 2017 to November 2020 in a tertiary hospital were divided into a development cohort, validation cohort, and two test cohorts that included admissions before and during the COVID-19 pandemic. Models were developed using logistic regression, decision tree, random forest, and XGBoost methods. Their areas under the receiver operating characteristic curves (AUCs), positive predictive values (PPVs), and negative predictive values (NPVs) were compared and validated. In the validation cohort, the AUCs were 0.887 for the XGBoost model, 0.885 for the logistic regression model, 0.878 for the random forest model, and 0.845 for the decision tree model. The XGBoost model was selected for subsequent application. In test cohort 1, the AUC was 0.891, with sensitivity of 0.812, specificity of 0.814, PPV of 0.708 and NPV of 0.886. In test cohort 2, the AUC was 0.885, with sensitivity of 0.816, specificity of 0.812, PPV of 0.659, and NPV of 0.908. In the cumulative incidence analysis, patients not receiving an ECG yet positively predicted by the model had significantly higher probability of receiving the examination within 48 h compared with those negatively predicted by the model. A machine learning model based on triage datasets was developed to predict ECG acquisition with high accuracy. The ECG recommendation can effectively predict whether patients presenting at ED triage will require an ECG, prompting subsequent analysis and decision-making in the ED.

Effect of the Emergency Department Assessment of Chest Pain Score on the Triage Performance in Patients With Chest Pain

The sensitivity of triage systems in identifying acute cardiovascular events in patients presented to the emergency department with chest pain is not optimal. Recently, a clinical score, the Emergency Department Assessment of Chest Pain Score (EDACS), has been proposed for a rapid assessment without additional instruments. To evaluate whether the integration of EDACS into triage evaluation of patients with chest pain can improve the triage's predictive validity for an acute cardiovascular event, a single-center prospective observational study was conducted. This study involved all patients who needed a triage admission for chest pain between January 1, 2020, and December 31, 2020. All enrolled patients first underwent a standard triage assessment and then the EDACS was calculated. The primary outcome of the study was the presence of an acute cardiovascular event. The discriminatory ability of EDACS in triage compared with standard triage assessment was evaluated by comparing the areas under the receiver operating characteristic curve, decision curve analysis, and net reclassification improvement. The study involved 1,596 patients, of that 7.3% presented the study outcome. The discriminatory ability of triage presented an area under the receiver operating characteristic curve of 0.688 that increased to 0.818 after the application of EDACS in the triage assessment. EDACS improved the baseline assessment of priority assigned in triage, with a net reclassification improvement of 33.6% (p <0.001), and the decision curve analyses demonstrated that EDACS in triage resulted in a clear net clinical benefit. In conclusion, the results of the study suggest that EDACS has a good discriminatory capacity for acute cardiovascular events and that its implementation in routine triage may improve triage performance in patients with chest pain.

Accuracy of telephone triage in primary care patients with chest discomfort: a cross-sectional study

OBJECTIVE

To assess the accuracy of semi-automatic assisted telephone triage in patients with acute chest discomfort against the diagnosis of acute coronary syndrome (ACS) or other life-threatening events (LTEs).

METHODS

A cross-sectional study was performed of telephone conversations with 2023 patients with acute chest discomfort (pain, pressure, tightness or discomfort) who called out-of-hours services for primary care (OHS-PC) between 2014 and 2016. Sensitivity, specificity, positive and negative predicted values were calculated for a high urgency (patient seen within one hour) against the diagnoses of ACS and other LTEs. Diagnoses were retrieved from the patients' medical records in general practice, including hospital specialists' discharge letters.

RESULTS

Of 2023 patients who called because of chest discomfort, 227 (11.2%) had an ACS (men 14.9%, women 8.2%) and 58 (2.9%) had another LTE (men 3.6%, women 2.3%). The sensitivity and specificity of a high Netherlands Triage System (NTS) urgency allocation against ACS/other LTEs were 0.73 (95% CI 0.68 to 0.78) and 0.43 (95% CI 0.40 to 0.45), respectively. In 13.2% of the calls the triage nurse overruled the NTS urgency, mostly by upscaling (11.0%). The sensitivity and specificity of the final urgency allocation were 0.86 (95% CI 0.81 to 0.90) and 0.34 (95% CI 0.32 to 0.37). The positive and negative predictive values of the final urgency were 0.18 (95% CI 0.17 to 0.19) and 0.94 (95% CI 0.92 to 0.95), respectively.

CONCLUSIONS

The semi-automatic triage NTS tool underestimated the urgency in 27% of patients with ACS/other LTEs. Overruling by triage nurses improved safety, but still 14% of men and women with ACS/other LTEs received too low urgency, while efficiency remained poor.

TRIAL REGISTRATION NUMBER

NTR7331.

Accuracy of telephone triage in patients suspected of transient ischaemic attack or stroke: a cross-sectional study

BACKGROUND

The Netherlands Triage Standard (NTS) is a widely used decision support tool for telephone triage at Dutch out-of-hours primary care services (OHS-PC), which, however, has never been validated against clinical outcomes. We aimed to determine the accuracy of the NTS urgency allocation for patients with neurological symptoms suggestive of a transient ischaemic attack (TIA) or stroke, with the clinical outcomes TIA, stroke, and other (neurologic) life-threatening events (LTEs) as the reference.

METHOD

A cross-sectional study of telephone triage recordings of patients with neurological symptoms calling the OHS-PC between 2014 and 2016.The allocated NTS urgencies were derived from the electronic medical records of the OHS-PC. The clinical outcomes were retrieved from the electronic medical records of the patients' own general practitioners. The accuracy of a high NTS urgency allocation (medical help within 3 h) was calculated in terms of sensitivity, specificity, positive and negative predictive values (PPV and NPV) with the clinical outcomes TIA/stroke/other LTEs as the reference.

RESULTS

Of 1269 patients, 635 (50.0%) received the diagnosis TIA/stroke (34.2% TIA/minor stroke, 15.8% major ischaemic or haemorrhagic stroke), and 4.8% other LTEs. For TIA/stroke/other LTEs, the sensitivity and specificity of the NTS urgency allocation were 0.72 (95%CI 0.68-0.75) and 0.48 (95%CI 0.43-0.52), and the PPV and NPV were 0.62 (95%CI 0.60-0.64) and 0.58 (95%CI 0.54-0.62).

CONCLUSIONS

The NTS decision support tool used in Dutch OHS-PC performed poor to moderately regarding safety (sensitivity) and efficiency (specificity) in allocating adequate urgencies to patients with and without TIA/stroke/other LTEs.

TRIAL REGISTRATION

The Netherlands National Trial Register, identification number NTR7331 /Trial NL7134 .

Limited reliability of experts' assessment of telephone triage in primary care patients with chest discomfort

OBJECTIVE

Root cause analyses of serious adverse events (SAE) in out-of-hours primary care (OHS-PC) often point to errors in telephone triage. Such analyses are, however, hampered by hindsight bias. We assessed whether experts, blinded to the outcome, recognize (un)safety of triage of patients with chest discomfort, and we quantified inter-rater reliability.

STUDY DESIGN AND SETTING

This is a case-control study with triage recordings from 2013-2017 at OHS-PC. Cases were missed acute coronary syndromes (ACSs, considered as SAE). These cases were age- and gender-matched 1:8 with the controls, sampled from the remainder of people calling for chest discomfort. Fifteen experts listened to the recordings and rated the safety of triage. We calculated sensitivity and specificity of recognizing an ACS and the intraclass correlation.

RESULTS

In total, 135 calls (15 SAE, 120 matched controls) were relistened. The experts identified ACSs with a sensitivity of 0.86 (95% CI: 0.71-0.95) and a specificity of 0.51 (95% CI: 0.43-0.58). Cases were rated significantly more often as unsafe than the controls (73.3% vs. 22.5%, P < 0.001). The inter-rater reliability for safety was poor: ICC 0.16 (95% CI: 0.00-0.32).

CONCLUSIONS

Blinded experts rated calls of missed ACSs more often as unsafe than matched control calls, but with a low level of agreement among the experts.

Obstetric triage systems: a systematic review of measurement properties (Clinimetric)

BACKGROUND

Since labor and delivery units often serve as emergency units for pregnant women, the use of obstetric triage systems with poor or inadequate quality can lead to unintended consequences such as over and under-triage and so a waste of humans and financial resources. Therefore, this systematic review was conducted to evaluate the measurement properties of obstetric triage tools.

METHODS

PubMed, EMBASE, and Medline were searched to identify studies in October 2018 and were updated in May 2019. The risk of bias COSMIN checklist was used to evaluate the quality of the studies. The quality of every measurement property was appraised by the update criteria of COSMIN. Evidence quality was judged using the modified GRADE approach.

RESULTS

A total of 444 studies were retrieved in initial search. Six studies evaluating 4 tools were included in this study. All the included studies reported only content validity and reliability. The quality of evidence varied from very low to moderate. The quality of content validity and reliability of the included tools was sufficient except for the reliability of the maternal-fetal triage index. The obstetric triage acuity scale (OTAS) was found to have higher reliability than other tools.

CONCLUSIONS

Due to insufficient evidence, the conclusions about the quality of measurement properties of each obstetric triage tool may be uncertain. This review emphasizes the necessity for further studies with robust methodological quality on the measurement properties of obstetric triage tools.

Effectiveness of Manchester Triage System in risk prioritisation of patients with pulmonary embolism who present dyspnoea, chest pain or collapse

BACKGROUND

Recognising patients with pulmonary embolism continues to be a clinical challenge. In the Emergency Department, up to 50% of patients with pulmonary embolism can be delayed or even misdiagnosed. The ability of a triage system to correctly prioritise suspected embolism in these patients is fundamental for determining diagnostic-therapeutic procedures.

AIM

To verify the effectiveness of the Manchester Triage System in risk prioritisation of patients with pulmonary embolism who present with dyspnoea, chest pain, or collapse.

METHODS

In this observational, retrospective, study the sensitivity, specificity, and positive and negative predictive values of the Manchester Triage System were calculated using the triage classification for these patients, and their established diagnoses. The analysis included baseline characteristics and triage evaluations.

RESULTS

During the two-year study period, 7055 patients were enrolled. Pulmonary embolism episodes were 2.1% of all cases, while severe pulmonary embolisms were 0.8%. The estimated specificity of the Manchester Triage System was 72.5% (CI 95%, 71.5-73.6), and the negative predictive value was 98.1% (CI 95%, 97.7-98.5). The results suggest that clinical characteristics leading to a high Manchester Triage System priority are similar to those characterising a pulmonary embolism episode.

CONCLUSIONS

Although pulmonary embolism is difficult to diagnose, the Manchester Triage System is an effective tool for prioritising patients with symptoms of this disease.

Optimisation of telephone triage of callers with symptoms suggestive of acute cardiovascular disease in out-of-hours primary care: observational design of the Safety First study

INTRODUCTION

In the Netherlands, the 'Netherlands Triage Standard' (NTS) is frequently used as digital decision support system for telephone triage at out-of-hours services in primary care (OHS-PC). The aim of the NTS is to guarantee accessible, efficient and safe care. However, there are indications that current triage is inefficient, with overestimation of urgency, notably in suspected acute cardiovascular disease. In addition, in primary care settings the NTS has only been validated against surrogate markers, and diagnostic accuracy with clinical outcomes as the reference is unknown. In the Safety First study, we address this gap in knowledge by describing, understanding and improving the diagnostic process and urgency allocation in callers with symptoms suggestive of acute cardiovascular disease, in order to improve both efficiency and safety of telephone triage in this domain.

METHODS AND ANALYSIS

An observational study in which 3000 telephone triage recordings (period 2014-2016) will be analysed. Information is collected from the recordings including caller and symptom characteristics and urgency allocation. The callers' own general practitioners are contacted for the final diagnosis of each contact. We included recordings of callers with symptoms suggestive of acute coronary syndrome (ACS) or transient ischaemic attack (TIA)/stroke. With univariable and multivariable logistic regression analyses the diagnostic accuracy of caller and symptom characteristics will be analysed in terms of predictive values with urgency level, and ACS and TIA/stroke as outcomes, respectively. To further improve our understanding of the triage process at OHS-PC, we will carry out additional studies applying both quantitative and qualitative methods: (i) case-control study on serious adverse events (SAE), (ii) conversation analysis study and (iii) interview study with triage nurses.

ETHICS AND DISSEMINATION

The Medical Ethics Committee Utrecht, the Netherlands endorsed this study (National Trial Register identification: NTR7331). Results will be disseminated at scientific conferences, regional educational sessions and publication in peer-reviewed journals.

Evaluation of triage quality in the emergency department: a scoping review protocol

REVIEW OBJECTIVE/QUESTIONS

The objective of this scoping review is to explore the existing literature on the evaluation of the quality of triage for patients of all ages and medical conditions in emergency departments (EDs).The question for this review is: How is triage in the ED evaluated? More specifically, we are interested in answering the following sub-questions.

Sensitivity and specificity of the Manchester Triage System in risk prioritization of patients with acute myocardial infarction who present with chest pain

Background:

The purpose of the Manchester Triage System is to clinically prioritize each patient seeking care in an emergency department. Patients with suspected acute myocardial infarction who have typical symptoms including chest pain should be classified in the highest priority groups, requiring immediate medical assistance or care within 10 min. As such, the Manchester Triage System should present adequate sensitivity and specificity.

Aims:

This study estimated the sensitivity and specificity of the Manchester Triage System in the triage of patients with chest pain related to the diagnosis of acute myocardial infarction, and the associations between the performance of the Manchester Triage System and selected variables.

Methods:

This was an observational, analytical, cross-sectional, retrospective study. The sensitivity and specificity of the Manchester Triage System were estimated by verifying the triage classification received by these patients and their established medical diagnoses.

Results:

The sample was composed of 10,087 triage episodes, in which 139 (1.38%) patients had a diagnosis of acute myocardial infarction. In 49 episodes, confirmation of medical diagnosis was not possible. The estimated sensitivity of the Manchester Triage System was 44.60% (36.18–53.27%) and the estimated specificity was 91.30% (90.73–91.85%). Of the 10,038 episodes in which the diagnosis of acute myocardial infarction was confirmed or excluded, 938 patients (9.34%) received an incorrect classification – undertriage or overtriage.

Conclusion:

This study showed that the specificity of the Manchester Triage System was very good. However, the low sensitivity based on the Manchester Triage System indicated that patients in high priority categories were undertriaged, leading to longer wait times and associated increased risks of adverse events.