Interrater Reliability, Accuracy, and Triage Time Pre- and Post-implementation of a Real-Time Electronic Triage Decision-Support Tool

Emergency department triage decision-making by registered nurses: An instrument development study

AIM

To develop and psychometrically test the triage decision-making instrument, a tool to measure Emergency Department Registered Nurses decision-making.

DESIGN

Five phases: (1) defining the concept, (2) item generation, (3) face validity, (4) content validity and (5) pilot testing.

METHODS

Concept definition informed by a grounded theory study from which four domains emerged. Items relevant to the four domains were generated and revised. Face validity was established using three focus groups. The target population upon which the reliability and validity of the triage decision-making instrument was explored were triage registered nurses in emergency departments. Three expert judges assessed 89 items for content and domain designation using a 4-point scale. Psychometric properties were assessed by exploratory factor analysis, following which the names of the four domains were modified.

RESULTS

The triage decision-making instrument is a 22-item tool with four factors: clinical judgement, managing acuity, professional collaboration and creating space. Focus group data indicated support for the domains. Expert review resulted in 46 items with 100% agreement and 13 with 66% agreement. Fifty-nine items were distributed to a convenience sample of 204 triage nurses from six hospitals in 2019. The Kaiser-Meyer-Olkin measures indicated that the data were sufficient for exploratory factor analysis. Bartlett's test indicated patterned relationships among the items (X2 (231) = 1156.69). An eigenvalue of >1.0 was used and four factors explained 48.64% of the variance. All factor loadings were ≥0.40. Internal consistency was demonstrated by Cronbach's alphas of .596 factor 1, .690 factor 2, .749 factor 3 and .822 for factor 4.

CONCLUSION

The triage decision-making instrument meets the criteria for face validity, content validity and internal consistency. It is suitable for further testing and refinement.

IMPACT

The instrument is a first step in quantifying triage decision-making in real-world clinical environments. The triage decision-making instrument can be used for targeted triage interventions aimed at improving throughput and staff education.

STATISTICAL SUPPORT

Dr. Tak Fung who is a member of the research team is a statistician.

STATISTICAL METHODS

Development, validation and assessment of instruments/scales. Descriptive statistics.

REPORTING METHOD

STROBE cross-sectional checklist.

IMPLICATIONS FOR THE PROFESSION AND/OR PATIENT CARE

The TDI makes the complexity of triage decision-making visible. Identifying the influence of decision-making factors in addition to acuity that affect triage decisions will enable nurse managers and educators to develop targeted interventions and staff development initiatives. By extension, this will enhance patient care and safety.

Drinking from the Holy Grail-Does a Perfect Triage System Exist? And Where to Look for It?

The Emergency Department (ED) is a facility meant to treat patients in need of medical assistance. The choice of triage system hugely impactsed the organization of any given ED and it is important to analyze them for their effectiveness. The goal of this review is to briefly describe selected triage systems in an attempt to find the perfect one. Papers published in PubMed from 1990 to 2022 were reviewed. The following terms were used for comparison: "ED" and "triage system". The papers contained data on the design and function of the triage system, its validation, and its performance. After studies comparing the distinct means of patient selection were reviewed, they were meant to be classified as either flawed or non-ideal. The validity of all the comparable segregation systems was similar. A possible solution would be to search for a new, measurable parameter for a more accurate risk estimation, which could be a game changer in terms of triage assessment. The dynamic development of artificial intelligence (AI) technologies has recently been observed. The authors of this study believe that the future segregation system should be a combination of the experience and intuition of trained healthcare professionals and modern technology (artificial intelligence).

Usability evaluation of a glove-type wearable device for efficient biometric collection during triage

To efficiently allocate medical resources at disaster sites, medical workers perform triage to prioritize medical treatments based on the severity of the wounded or sick. In such instances, evaluators often assess the severity status of the wounded or sick quickly, but their measurements are qualitative and rely on experience. Therefore, we developed a wearable device called Medic Hand in this study to extend the functionality of a medical worker's hand so as to measure multiple biometric indicators simultaneously without increasing the number of medical devices to be carried. Medic Hand was developed to quantitatively and efficiently evaluate "perfusion" during triage. Speed is essential during triage at disaster sites, where time and effort are often spared to attach medical devices to patients, so the use of Medic Hand as a biometric measurement device is more efficient for collecting biometric information. For Medic Hand to be handy during disasters, it is essential to understand and improve upon factors that facilitate its public acceptance. To this end, this paper reports on the usability evaluation of Medic Hand through a questionnaire survey of nonmedical workers.

Evaluating the Reliability of a Remote Acuity Prediction Tool in a Canadian Academic Emergency Department

STUDY OBJECTIVE

There is increasing interest in harnessing artificial intelligence to virtually triage patients seeking care. The objective was to examine the reliability of a virtual machine learning algorithm to remotely predict acuity scores for patients seeking emergency department (ED) care by applying the algorithm to retrospective ED data.

METHODS

This was a retrospective review of adult patients conducted at an academic tertiary care ED (annual census 65,000) from January 2021 to August 2022. Data including ED visit date and time, patient age, sex, reason for visit, presenting complaint and patient-reported pain score were used by the machine learning algorithm to predict acuity scores. The algorithm was designed to up-triage high-risk complaints to promote safety for remote use. The predicted scores were then compared to nurse-led triage scores previously derived in real time using the electronic Canadian Triage and Acuity Scale (eCTAS), an electronic triage decision-support tool used in the ED. Interrater reliability was estimated using kappa statistics with 95% confidence intervals (CIs).

RESULTS

In total, 21,469 unique ED patient encounters were included. Exact modal agreement was achieved for 10,396 (48.4%) patient encounters. Interrater reliability ranged from poor to fair, as estimated using unweighted kappa (0.18, 95% CI 0.17 to 0.19), linear-weighted kappa (0.25, 95% CI 0.24 to 0.26), and quadratic-weighted kappa (0.36, 95% CI 0.35 to 0.37) statistics. Using the nurse-led eCTAS score as the reference, the machine learning algorithm overtriaged 9,897 (46.1%) and undertriaged 1,176 (5.5%) cases. Some of the presenting complaints under-triaged were conditions generally requiring further probing to delineate their nature, including abnormal lab/imaging results, visual disturbance, and fever.

CONCLUSION

This machine learning algorithm needs further refinement before being safely implemented for patient use.

Repeatability, reproducibility, and diagnostic accuracy of a commercial large language model (ChatGPT) to perform emergency department triage using the Canadian triage and acuity scale

PURPOSE

The release of the ChatGPT prototype to the public in November 2022 drastically reduced the barrier to using artificial intelligence by allowing easy access to a large language model with only a simple web interface. One situation where ChatGPT could be useful is in triaging patients arriving to the emergency department. This study aimed to address the research problem: "can emergency physicians use ChatGPT to accurately triage patients using the Canadian Triage and Acuity Scale (CTAS)?".

METHODS

Six unique prompts were developed independently by five emergency physicians. An automated script was used to query ChatGPT with each of the 6 prompts combined with 61 validated and previously published patient vignettes. Thirty repetitions of each combination were performed for a total of 10,980 simulated triages.

RESULTS

In 99.6% of 10,980 queries, a CTAS score was returned. However, there was considerable variations in results. Repeatability (use of the same prompt repeatedly) was responsible for 21.0% of overall variation. Reproducibility (use of different prompts) was responsible for 4.0% of overall variation. Overall accuracy of ChatGPT to triage simulated patients was 47.5% with a 13.7% under-triage rate and a 38.7% over-triage rate. More extensively detailed text given as a prompt was associated with greater reproducibility, but minimal increase in accuracy.

CONCLUSIONS

This study suggests that the current ChatGPT large language model is not sufficient for emergency physicians to triage simulated patients using the Canadian Triage and Acuity Scale due to poor repeatability and accuracy. Medical practitioners should be aware that while ChatGPT can be a valuable tool, it may lack consistency and may frequently provide false information.

Development of a machine learning-based acuity score prediction model for virtual care settings

OBJECTIVE

Healthcare is increasingly digitized, yet remote and automated machine learning (ML) triage prediction systems for virtual urgent care use remain limited. The Canadian Triage and Acuity Scale (CTAS) is the gold standard triage tool for in-person care in Canada. The current work describes the development of a ML-based acuity score modelled after the CTAS system.

METHODS

The ML-based acuity score model was developed using 2,460,109 de-identified patient-level encounter records from three large healthcare organizations (Ontario, Canada). Data included presenting complaint, clinical modifiers, age, sex, and self-reported pain. 2,041,987 records were high acuity (CTAS 1-3) and 416,870 records were low acuity (CTAS 4-5). Five models were trained: decision tree, k-nearest neighbors, random forest, gradient boosting regressor, and neural net. The outcome variable of interest was the acuity score predicted by the ML system compared to the CTAS score assigned by the triage nurse.

RESULTS

Gradient boosting regressor demonstrated the greatest prediction accuracy. This final model was tuned toward up triaging to minimize patient risk if adopted into the clinical context. The algorithm predicted the same score in 47.4% of cases, and the same or more acute score in 95.0% of cases.

CONCLUSIONS

The ML algorithm shows reasonable predictive accuracy and high predictive safety and was developed using the largest dataset of its kind to date. Future work will involve conducting a pilot study to validate and prospectively assess reliability of the ML algorithm to assign acuity scores remotely.

The effect of computer-based decision support system on emergency department triage: Non-randomised controlled trial

BACKGROUND

Deciding on triage in emergency departments is difficult and requires comprehensive knowledge and experience.

PURPOSE

This study was conducted to evaluate the effect of a "computer-based emergency department triage decision support system (DSS)," which was designed and integrated into the hospital information management system, on triage decision accuracy and triage duration by using real patient data.

METHODS

Single-group, pretest-posttest non-randomised clinical trial. The study was conducted with the real data of patients who had been triaged in the adult emergency department of a university hospital. The pretest was applied between July 16 and September 16, 2019, and the post-test on September 1 and October 31, 2020. In the pre-test and post-test phases of the study, triage decision accuracy rates, and triage duration were evaluated. In the post-test phase, Emergency Triage Decision Support System (ETDSS) was prepared with a rule-based decision trees method using the Emergency Severity Index Version 4 and The Australasian Triage Scale and was integrated into the hospital information management system. The effect of the developed ETDSS was evaluated. The mean, standard deviation, frequency and percentage values were calculated for the descriptive characteristics. Independent samples t-test, analysis of variance, Sidak paired comparison, and Bonferroni tests were applied.

RESULTS

The effect of the computer-based emergency triage DSS on triage management was tested based on the data of 16,409 patients in the pretest phase and 7,765 patients in the posttest phase. While the accuracy rate of nurses' triage decisions was 57.8% in the pretest, it was found to increase to 64.9% in the posttest. The mean duration of triage was 1.47 ± 0.72 in the pretest and 1.79 ± 0.85 min in the posttest.

CONCLUSIONS

The DSS increased triage decision accuracy independently of professional and triage experience and brought the triage duration closer to the time recommended in the literature. Clinically, this is associated with patient safety, quality improvement processes, and professional accountability.

The Effect of Human Supervision on an Electronic Implementation of the Canadian Triage Acuity Scale (CTAS)

BACKGROUND

Most electronic emergency department (ED) triage systems allow nurses to modify computer-generated triage scores. It is currently unclear how this affects triage validity.

OBJECTIVE

Are nurse-generated triage scores more strongly associated with rates of admission, intensive care unit (ICU) consultation, and mortality than computer-generated scores?

METHODS

Retrospective observational cohort study of all adult visits to a tertiary ED. An electronic implementation of the Canadian Triage Acuity Scale (CTAS) generated a CTAS score for each visit. In some cases, the triage nurse overwrote the computer-generated CTAS score with a score they felt was more appropriate. Among visits with nurse-modified triage scores, we compared the rate of acuity-related outcomes (mortality, ICU consultation, hospital admission) in each CTAS level as categorized by nurse-generated vs. computer-generated scores.

RESULTS

In a cohort of 229,744 patients, 19,566 (8.51%) had nurse-modified triage scores. Most modifications consisted of assigning a higher acuity triage score than recommended by the computer. Visits with triage scores 1-2 according to the nurse-generated scores had the same or higher rates of the acuity outcomes than visits that were CTAS 1-2 according to the computer-generated CTAS scores. Conversely, visits with triage scores 4-5 according to the nurse-generated scores had lower rates of the outcomes than visits that were CTAS 4-5 according to the computer-generated CTAS scores.

CONCLUSIONS

Nursing supervision of the computer-automated CTAS triage system was associated with fewer hospital admissions, ICU consultations, and deaths in the triage score 4-5 categories, suggesting a safer triage process than the automated CTAS algorithm alone.

Development and Validation of Simple Age-Adjusted Objectified Korean Triage and Acuity Scale for Adult Patients Visiting the Emergency Department

PURPOSE

The study aimed to develop an objectified Korean Triage and Acuity Scale (OTAS) that can objectively and quickly classify severity, as well as a simple age-adjusted OTAS (S-OTAS) that reflects age and evaluate its usefulness.

MATERIALS AND METHODS

A retrospective analysis was performed of all adult patients who had visited the emergency department at three teaching hospitals. Sex, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, O₂ saturation, and consciousness level were collected from medical records. The OTAS was developed with objective criterion and minimal OTAS level, and S-OTAS was developed by adding the age variable. For usefulness evaluation, the 30-day mortality, the rates of computed tomography scan and emergency procedures were compared between Korean Triage and Acuity Scale (KTAS) and OTAS.

RESULTS

A total of 44402 patients were analyzed. For 30-day mortality, S-OTAS showed a higher area under the curve (AUC) compared to KTAS (0.751 vs. 0.812 for KTAS and S-OTAS, respectively, p<0.001). Regarding the rates of emergency procedures, AUC was significantly higher in S-OTAS, compared to KTAS (0.807 vs. 0.830, for KTAS and S-OTAS, respectively, p=0.013).

CONCLUSION

S-OTAS showed comparative usefulness for adult patients visiting the emergency department as a triage tool compared to KTAS.

METASTART: A Systematic Review and Meta-Analysis of the Diagnostic Accuracy of the Simple Triage and Rapid Treatment (START) Algorithm for Disaster Triage

INTRODUCTION

The goal of disaster triage at both the prehospital and in-hospital level is to maximize resources and optimize patient outcomes. Of the disaster-specific triage methods developed to guide health care providers, the Simple Triage and Rapid Treatment (START) algorithm has become the most popular system world-wide. Despite its appeal and global application, the accuracy and effectiveness of the START protocol is not well-known.

OBJECTIVES

The purpose of this meta-analysis was two-fold: (1) to estimate overall accuracy, under-triage, and over-triage of the START method when used by providers across a variety of backgrounds; and (2) to obtain specific accuracy for each of the four START categories: red, yellow, green, and black.

METHODS

A systematic review and meta-analysis was conducted that searched Medline (OVID), Embase (OVID), Global Health (OVID), CINAHL (EBSCO), Compendex (Engineering Village), SCOPUS, ProQuest Dissertations and Theses Global, Cochrane Library, and PROSPERO. The results were expanded by hand searching of journals, reference lists, and the grey literature. The search was executed in March 2020. The review considered the participants, interventions, context, and outcome (PICO) framework and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Accuracy outcomes are presented as means with 95% confidence intervals (CI) as calculated using the binomial method. Pooled meta-analyses of accuracy outcomes using fixed and random effects models were calculated and the heterogeneity was assessed using the Q statistic.

RESULTS

Thirty-two studies were included in the review, most of which utilized a non-randomized study design (84%). Proportion of victims correctly triaged using START ranged from 0.27 to 0.99 with an overall triage accuracy of 0.73 (95% CI, 0.67 to 0.78). Proportion of over-triage was 0.14 (95% CI, 0.11 to 0.17) while the proportion of under-triage was 0.10 (95% CI, 0.072 to 0.14). There was significant heterogeneity of the studies for all outcomes (P < .0001).

CONCLUSION

This meta-analysis suggests that START is not accurate enough to serve as a reliable disaster triage tool. Although the accuracy of START may be similar to other models of disaster triage, development of a more accurate triage method should be urgently pursued.

Automation in nursing decision support systems: A systematic review of effects on decision making, care delivery, and patient outcomes

OBJECTIVE

The study sought to summarize research literature on nursing decision support systems (DSSs ); understand which steps of the nursing care process (NCP) are supported by DSSs, and analyze effects of automated information processing on decision making, care delivery, and patient outcomes.

MATERIALS AND METHODS

We conducted a systematic review in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement. PubMed, CINAHL, Cochrane, Embase, Scopus, and Web of Science were searched from January 2014 to April 2020 for studies focusing on DSSs used exclusively by nurses and their effects. Information about the stages of automation (information acquisition, information analysis, decision and action selection, and action implementation), NCP, and effects was assessed.

RESULTS

Of 1019 articles retrieved, 28 met the inclusion criteria, each studying a unique DSS. Most DSSs were concerned with two NCP steps: assessment (82%) and intervention (86%). In terms of automation, all included DSSs automated information analysis and decision selection. Five DSSs automated information acquisition and only one automated action implementation. Effects on decision making, care delivery, and patient outcome were mixed. DSSs improved compliance with recommendations and reduced decision time, but impacts were not always sustainable. There were improvements in evidence-based practice, but impact on patient outcomes was mixed.

CONCLUSIONS

Current nursing DSSs do not adequately support the NCP and have limited automation. There remain many opportunities to enhance automation, especially at the stage of information acquisition. Further research is needed to understand how automation within the NCP can improve nurses' decision making, care delivery, and patient outcomes.

Real-Time Monitoring Electronic Triage Tag System for Improving Survival Rate in Disaster-Induced Mass Casualty Incidents

This study was conducted to contribute to active disaster response by developing internet of things (IoT)-based vital sign monitoring e-triage tag system to improve the survival rate at disaster mass casualty incidents fields. The model used in this study for developing the e-triage tag system is the rapid prototyping model (RAD). The process comprised six steps: analysis, design, development, evaluation, implementation, and simulation. As a result of detailed assessment of the system design and development by an expert group, areas with the highest score in the triage sensor evaluation were rated "very good", with 5 points for continuous vital sign data delivery, portability, and robustness. In addition, ease of use, wearability, and electricity consumption were rated 4.8, 4.7, and 4.6 points, respectively. In the triage application evaluation, the speed and utility scored a perfect 5 points, and the reliability and expressiveness were rated 4.9 points and 4.8 points, respectively. This study will contribute significantly to increasing the survival rate via the development of a conceptual prehospital triage for field applications and e-triage tag system implementation.

Influence of electronic triage decision-support on hospital admission, left without being seen and time to physician initial assessment in the emergency department

OBJECTIVE

To explore the impact of the implementation of eCTAS, a real-time electronic decision-support tool, on hospital admission, rate of left without being seen, and time from triage to physician initial assessment.

METHODS

We conducted a cohort study using population-based administrative data from all Ontario emergency departments (EDs) that had implemented eCTAS for 9 months. We compared 6 months post-eCTAS data to the same 6 months the previous year (pre-eCTAS). We included triage encounters of adult (≥ 18 years) patients if they had one of 16 pre-specified, high-volume presenting complaints. Multivariable logistic regression and quantile regression models informed the effect of eCTAS on outcomes.

RESULTS

We included data from 354,176 triage encounters from 31 EDs. There was a change in the distribution of triage scores post-eCTAS, with fewer patients classified as CTAS 2 and CTAS 3, and more patients classified as CTAS 1 and CTAS 4. Overall, hospital admission decreased post-eCTAS (adjusted OR: 0.98; 95% CI: 0.97 to 1.00), with fewer CTAS 2 and more CTAS 3 and CTAS 4 patients admitted post-eCTAS. The rate of left without being seen increased (2.8% vs. 3.0%; adjusted OR: 1.07; 95% CI: 1.03 to 1.11) post-eCTAS, while time to physician initial assessment proved similar pre and post-eCTAS.

CONCLUSIONS

eCTAS implementation had little impact on admission, rate of left without being seen and time to physician initial assessment. eCTAS appears to reclassify patients from higher to lower acuity scores, resulting in higher admission rates for CTAS 3 and CTAS 4 patients. It remains unknown if this reclassification is appropriate.

Starting, building and sustaining a program of research in emergency medicine in Canada

OBJECTIVE

To develop pragmatic recommendations for starting, building and sustaining a program of research in emergency medicine (EM) in Canada at sites with limited infrastructure and/or prior research experience.

METHODS

At the direction of the Canadian Association of Emergency Physicians (CAEP) academic section, we assembled an expert panel of 10 EM researchers with experience building programs of research. Using a modified Delphi approach, our panel developed initial recommendations for (1) starting, (2) building, and (3) sustaining a program of research in EM. These recommendations were peer-reviewed by emergency physicians and researchers from each of the panelist's home institutions and tested for face and construct validity, as well as ease of comprehension. The recommendations were then iteratively revised based on feedback and suggestions from peer review and amended again after being presented at the 2020 CAEP academic symposium.

RESULTS

Our panel created 15 pragmatic recommendations for those intending to start (formal research training, find mentors, local support, develop a niche, start small), build (funding, build a team, collaborate, publish, expect failure) and sustain (become a mentor, obtain leadership roles, lead national studies, gain influence, prioritize wellness) a program of EM research in centers without an established research culture. Additionally, we suggest four recommendations for department leads aiming to foster a program of research within their departments.

CONCLUSION

These recommendations serve as guidance for centres wanting to establish a program of research in EM.

Consistency of triage scores by presenting complaint pre- and post-implementation of a real-time electronic triage decision support tool

OBJECTIVE

eCTAS is a real-time electronic decision-support tool designed to standardize the application of the Canadian Triage and Acuity Scale (CTAS). This study addresses the variability of CTAS score distributions across institutions pre- and post-eCTAS implementation.

METHODS

We used population-based administrative data from 2016-2018 from all emergency departments (EDs) that had implemented eCTAS for 9 months. Following a 3-month stabilization period, we compared 6 months post-eCTAS data to the same 6 months the previous year (pre-eCTAS). We included triage encounters of adult (≥17 years) patients who presented with 1 of 16 pre-specified, high-volume complaints. For each ED, consistency was calculated as the absolute difference in CTAS distribution compared to the average of all included EDs for each presenting complaint. Pre-eCTAS and post-eCTAS change scores were compared using a paired-samples t-test. We also assessed if eCTAS modifiers were associated with triage consistency.

RESULTS

There were 363,214 (183,231 pre-eCTAS, 179,983 post-eCTAS) triage encounters included from 35 EDs. Triage scores were more consistent (P < 0.05) post-eCTAS for 6 (37.5%) presenting complaints: chest pain (cardiac features), extremity weakness/symptoms of cerebrovascular accident, fever, shortness of breath, syncope, and hyperglycemia. Triage consistency was similar pre- and post-eCTAS for altered level of consciousness, anxiety/situational crisis, confusion, depression/suicidal/deliberate self-harm, general weakness, head injury, palpitations, seizure, substance misuse/intoxication, and vertigo. Use of eCTAS modifiers was associated with increased triage consistency.

CONCLUSIONS

eCTAS increased triage consistency across many, but not all, high-volume presenting complaints. Modifier use was associated with increased triage consistency, particularly for non-specific complaints such as fever and general weakness.

Performance of a three-level triage scale in live triage encounters in an emergency department in Hong Kong

Background

Despite its continued use in many low-volume emergency departments (EDs), 3-level triage systems have not been extensively studied, especially on live triage cases. We have modified from the Australasian Triage Scale and developed a 3-level triage scale, and sought to evaluate its validity, reliability, and over- and under-triage rates in real patient encounters in our setting.

Method

This was a cross-sectional study in a single ED with 24,000 attendances per year. At triage, each patient was simultaneously assessed by a triage nurse, an adjudicator (the “criterion standard”), and a study nurse independently. Predictive validity was determined by comparing clinical outcomes, such as hospitalization, across triage levels. The discriminating performance of the triage tool in identifying patients requiring earlier medical attention was determined. Inter-observer reliability between the triage nurse and criterion standard, and across providers were determined using kappa statistics.

Results

In total, 453 triage ratings of 151 triage cases, involving 17 ED triage nurses and 57 nurse pairs, were analysed. The proportion of hospital admission significantly increased with a higher triage rating. The performance of the scale in identifying patients requiring earlier medical attention was as follows: sensitivity, 68.2% (95% CI 45.1–86.1%); specificity, 99.2% (95% CI 95.8–100%); positive predictive value, 93.8% (95% CI 67.6–99.1%); and negative predictive value, 94.8% (95% CI 90.8–97.1%). The over-triage and under-triage rates were 0.7% and 4.6%, respectively. Agreement between the triage nurse and criterion standard was substantial (quadratic-weighted kappa = 0.76, 95% CI, 0.60–0.92, p < 0.001), so was the agreement across nurses (quadratic-weighted kappa = 0.81, 95% CI 0.65–0.97, p < 0.001).

Conclusions

The 3-level triage system appears to have good validity and reasonable reliability in a low-volume ED setting. Further studies comparing 3-level and prevailing 5-level triage scales in live triage encounters and different ED settings are warranted.

Reliability and validity of three international triage systems within a private health-care group in the Middle East

AIM

To measure and compare the reliability and validity of three international triage systems (CTAS, MTS and ESI) when applied to patient presentations in the emergency centres of a private health-care group in the United Arab Emirates.

BACKGROUND

The ability of triage systems to sort patients into categories based on the urgency of their need and time to physician is a key indicator. Three international triage systems are being used for this purpose in private emergency centre settings.

METHOD

Bespoke reference scenarios, 50 vignettes (10 per severity level) were created and validated by a local expert panel. Nurses performing triage at four emergency centres in the Emirate of Dubai completed online surveys to categorise the vignettes based on the triage system they used in their emergency centre.

RESULTS

Overall inter-rater reliability per triage category was substantial for allocations in category one, moderate for those in categories two and five, and fair for those in categories three and four. Agreement between raters and the reference standard was consistent throughout all four emergency centres. The accuracy of triaging allocations into categories one, two and five were good, whereas allocations in categories three and four were less accurate.

CONCLUSION

International triage systems focus on the identification of more urgent cases and perform poorly in discriminating between those that are less serious, which is less ideal in a setting where less-serious cases are more prevalent.