Assessment of inter-observer reliability of two five-level triage and acuity scales: a randomized controlled trial

Modern Paediatric Emergency Department: Potential Improvements in Light of New Evidence

The increasing attendance of paediatric emergency departments has become a serious health issue. To reduce an elevated burden of medical errors, inevitably caused by a high level of stress exerted on emergency physicians, we propose potential areas for improvement in regular paediatric emergency departments. In an effort to guarantee the demanded quality of care to all incoming patients, the workflow in paediatric emergency departments should be sufficiently optimised. The key component remains to implement one of the validated paediatric triage systems upon the patient's arrival at the emergency department and fast-tracking patients with a low level of risk according to the triage system. To ensure the patient's safety, emergency physicians should follow issued guidelines. Cognitive aids, such as well-designed checklists, posters or flow charts, generally improve physicians' adherence to guidelines and should be available in every paediatric emergency department. To sharpen diagnostic accuracy, the use of ultrasound in a paediatric emergency department, according to ultrasound protocols, should be targeted to answer specific clinical questions. Combining all mentioned improvements might reduce the number of errors linked to overcrowding. The review serves not only as a blueprint for modernising paediatric emergency departments but also as a bin of useful literature which can be suitable in the paediatric emergency field.

Utility of the Emergency Severity Index by Accuracy of Interrater Agreement by Expert Triage Nurses in a Simulated Scenario in Japan: A Randomized Controlled Trial

OBJECTIVE

The Emergency Severity Index (ESI) is a highly reliable and valid triage scale that is widely used in emergency departments in not only English language regions but also other countries. The Japan Triage and Acuity Scale (JTAS) is frequently used for emergency patients, and the ESI has not been evaluated against the JTAS in Japan. This study aimed to examine the decision accuracy of the ESI for simulated clinical scenarios among nursing specialists in Japan compared with the JTAS.

METHOD

A parallel group randomized trial was conducted. In total, 23 JTAS-trained triage nurses from 10 Japanese emergency departments were randomly assigned to the ESI or the JTAS group. Nurses independently assigned triage categories to 80 emergency cases for the assessment of interrater agreement.

RESULTS

Interrater agreement between the expert and triage nurses was κ = 0.82 (excellent) in the ESI group and κ = 0.74 (substantial) in the JTAS group. In addition, interrater agreement by acuity was level 2 = 0.42 (moderate) in the ESI group and level 2 = 0.31 (fair) in the JTAS group. Interrater agreement for triage decisions was classified in a higher category in the ESI group than in the JTAS Scale group at level 2. Triage decisions based on the ESI in Japan maintained the same level of interrater agreement and sensitivity as those in other countries.

CONCLUSION

These findings suggest that the ESI can be introduced in Japan, despite its different emergency medical background compared with other countries.

Implementation of a novel three-tier triage tool in Papua New Guinea: A model for resource-limited emergency departments

In emergency departments (EDs), demand for care often exceeds the available resources. Triage addresses this problem by sorting patients into categories of urgency.

The Interagency Integrated Triage Tool (IITT) is a novel triage system designed for resource-limited emergency care (EC) settings. The system was piloted by two EDs in Papua New Guinea as part of an EC capacity development program.

Implementation involved a five-hour teaching program for all ED staff, complemented by training resources including flowcharts and reference guides. Clinical redesign helped optimise flow and infrastructure, and development of simple electronic registries enabled data collection. Local champions were identified, and experienced EC clinicians from Australia acted as mentors during system roll-out.

Evaluation data suggests the IITT, and the associated change management process, have high levels of acceptance amongst staff. Subject to validation, the IITT may be relevant to other resource-limited EC settings.

An epidemiological study of a patient population, triage category allocations and principal diagnosis within the emergency centres of a private healthcare group in the Emirate of Dubai, United Arab Emirates

Aim

To describe, compare and correlate the number of patients seen, their demographics, triage category allocations and principal diagnosis in four emergency centres; to better understand the patient population and triage practices in this setting.

Design

An observational, cross-sectional, epidemiological study.

Methods

Electronic medical records were retrospectively evaluated from patients triaged in each of the four emergency centres over six months. Descriptive statistics were used to describe the patient demographics and variance between triage category allocations.

Results

A total of 56,984 patient records were captured, with an equal gender split and the workforce being the largest patient population (20-50 years). Acute upper respiratory infection was the most prolific diagnosis, and lower acuity triage categories were allocated the most. There were inconsistencies in the application of triage systems between the emergency centres, the most obvious being the variance in triage system selection and application.

A Method for Grouping Emergency Department Visits by Severity and Complexity

INTRODUCTION

Triage functions to quickly prioritize care and sort patients by anticipated resource needs. Despite widespread use of the Emergency Severity Index (ESI), there is still no universal standard for emergency department (ED) triage. Thus, it can be difficult to objectively assess national trends in ED acuity and resource requirements. We sought to derive an ESI from National Hospital Ambulatory Medical Care Survey (NHAMCS) survey items (NHAMCS-ESI) and to assess the performance of this index with respect to stratifying outcomes, including hospital admission, waiting times, and ED length of stay (LOS).

METHODS

We used data from the 2010-2015 NHAMCS, to create a measure of ED visit complexity based on variables within NHAMCS. We used NHAMCS data on chief complaint, vitals, resources used, interventions, and pain level to group ED visits into five levels of acuity using a stepwise algorithm that mirrored ESI. In addition, we examined associations of NHAMCS-ESI with typical indicators of acuity such as waiting time, LOS, and disposition. The NHAMCS-ESI categorization was also compared against the "immediacy" variable across all of these outcomes. Visit counts used weighted scores to estimate national levels of ED visits.

RESULTS

The NHAMCS ED visits represent an estimated 805,726,000 ED visits over this time period. NHAMCS-ESI categorized visits somewhat evenly, with most visits (42.5%) categorized as a level 3. The categorization pattern is distinct from that of the "immediacy" variable within NHAMCS. Of admitted patients, 89% were categorized as NHAMCS-ESI level 2-3. Median ED waiting times increased as NHAMCS-ESI levels decreased in acuity (from approximately 14 minutes to 25 minutes). Median LOS decreased as NHAMCS-ESI decreased from almost 200 minutes for level 1 patients to nearly 80 minutes for level 5 patients.

CONCLUSION

We derived an objective tool to measure an ED visit's complexity and resource use. This tool can be validated and used to compare complexity of ED visits across hospitals and regions, and over time.

Interrater Reliability, Accuracy, and Triage Time Pre- and Post-implementation of a Real-Time Electronic Triage Decision-Support Tool

STUDY OBJECTIVE

The electronic Canadian Triage and Acuity Scale (eCTAS) is a real-time electronic triage decision-support tool designed to improve patient safety and quality of care by standardizing the application of the Canadian Triage and Acuity Scale (CTAS). The objective of this study is to determine interrater agreement of triage scores pre- and post-implementation of eCTAS.

METHODS

This was a prospective, observational study conducted in 7 emergency departments (EDs), selected to represent a mix of triage documentation practices, hospital types, and patient volumes. A provincial CTAS auditor observed triage nurses in the ED pre- and post-implementation of eCTAS and assigned an independent CTAS score in real time. Research assistants independently recorded triage time. Interrater agreement was estimated with κ statistics with 95% confidence intervals (CIs).

RESULTS

A total of 1,491 individual triage assessments (752 pre-eCTAS, 739 post-implementation) were audited during 42 7-hour triage shifts (21 pre-eCTAS, 21 post-implementation). Exact modal agreement was achieved for 567 patients (75.4%) pre-eCTAS compared with 685 patients (92.7%) triaged with eCTAS. With the auditor's CTAS score as the reference, eCTAS significantly reduced the number of patients over-triaged (12.0% versus 5.1%; Δ 6.9; 95% CI 4.0 to 9.7) and under-triaged (12.6% versus 2.2%; Δ 10.4; 95% CI 7.9 to 13.2). Interrater agreement was higher with eCTAS (unweighted κ 0.89 versus 0.63; quadratic-weighted κ 0.93 versus 0.79). Median triage time was 312 seconds (n=3,808 patients) pre-eCTAS and 347 seconds (n=3,489 patients) with eCTAS (Δ 35 seconds; 95% CI 29 to 40 seconds).

CONCLUSION

A standardized, electronic approach to performing triage assessments improves both interrater agreement and data accuracy without substantially increasing triage time.

Emergency Medical Services Utilization Is Associated With Community Deprivation in Children

BACKGROUND

Pediatric emergency medical services (EMS) utilization is costly and resource intensive; significant variation exists across large-scale geographies. Less is known about variation at smaller geographic levels where factors including lack of transportation, low health literacy, and decreased access to medical homes may be more relevant. Our objective was to determine whether pediatric EMS utilization varied across Hamilton County, Ohio, census tracts and whether such utilization was associated with socioeconomic deprivation.

METHODS

This was a retrospective analysis of children living in Hamilton County, Ohio, transported by EMS to the Cincinnati Children's emergency department between July 1, 2014, and July 31, 2016. Participants' addresses were assigned to census tracts and an EMS utilization rate and deprivation index were calculated for each. Pearson's correlation coefficients evaluated relationships between tract-level EMS utilization and deprivation. Tract-level deprivation was used as a predictor in patient-level evaluations of acuity.

RESULTS

During the study period, there were 4,877 pediatric EMS transports from 219 of the 222 county census tracts. The county EMS utilization rate during the study period was 2.4 transports per 100 children (range 0.2-11). EMS utilization rates were positively correlated with increasing deprivation (r = 0.72, 95% confidence interval [CI], 0.65-0.77). Deprivation was associated with lower illness severity at triage, fewer transports resulting in resuscitation suite use, and fewer transports resulting in hospitalizations (all p < 0.05).

CONCLUSIONS

EMS utilization varied substantially across census tracts in Hamilton County, Ohio. A deeper understanding into why certain socioeconomically deprived areas contribute to disproportionately high rates of EMS utilization could support development of targeted interventions to improve use.

Blood pressure documentation in the emergency department

Objective

To analyze the frequency of blood pressure documentation performed by nursing professionals in an emergency department.

Methods

This is a cross-sectional, observational, descriptive, and analytical study, which included medical records of adult patients admitted to the observation ward of an emergency department, between March and May 2014. Data were obtained through a collection instrument divided into three parts: patient identification, triage data, and blood pressure documentation. For statistical analysis, Pearson’s correlation coefficient was used, with a significance level of α<0.05.

Results

One hundred fifty-seven records and 430 blood pressure measurements were analyzed with an average of three measurements per patient. Of these measures, 46.5% were abnormal. The mean time from admission to documentation of the first blood pressure measurement was 2.5 minutes, with 42 minutes between subsequent measures. There is no correlation between the systolic blood pressure values and the mean time interval between blood pressure documentations: 0.173 (p=0.031).

Conclusion

The present study found no correlation between frequency of blood pressure documentation and blood pressure values. The frequency of blood pressure documentation increased according to the severity of the patient and decreased during the length of stay in the emergency department.

Importance of respiratory rate for the prediction of clinical deterioration after emergency department discharge: a single-center, case-control study

Aim

The purpose of the present study was to investigate the predictors of clinical deterioration soon after emergency department (ED) discharge.

Methods

We undertook a case–control study using the ED database of the Nagano Municipal Hospital (Nagano, Japan) from January 2012 to December 2013. We selected adult patients with medical conditions who revisited the ED with deterioration within 2 days of ED discharge (deterioration group). The deterioration group was compared with a control group.

Results

During the study period, 15,724 adult medical patients were discharged from the ED. Of these, 170 patients revisited the ED because of clinical deterioration within 2 days. Among the initial vital signs, respiratory rate was less frequently recorded than other vital signs (P < 0.001 versus all other vital signs in each group). The frequency of recording each vital sign did not differ significantly between the groups. Overall, patients in the deterioration group had significantly higher respiratory rates than those in the control group (21 ± 5/min versus 18 ± 5/min, respectively; P = 0.002). A binary logistic regression analysis revealed that respiratory rate was an independent risk factor for clinical deterioration (unadjusted odds ratio, 1.15; 95% confidence interval, 1.04−1.26; adjusted odds ratio, 1.15; 95% confidence interval, 1.01−1.29).

Conclusions

An increased respiratory rate is a predictor of early clinical deterioration after ED discharge. Vital signs, especially respiratory rate, should be carefully evaluated when making decisions about patient disposition in the ED.

Prehospital Application of the Canadian Triage and Acuity Scale by Emergency Medical Services

OBJECTIVES

Triage is fundamental to emergency patient assessment. Effective triage systems accurately prioritize patients and help predict resource utilization. CTAS is a validated five-level triage score utilized in Emergency Departments (EDs) across Canada and internationally. Historically CTAS has been applied by triage nurses in EDs. Observational evidence suggests that the CTAS might be implemented reliably by paramedics in the prehospital setting. This is the first system-wide assessment of CTAS interrater reliability between paramedics and triage nurses during clinical practice.

METHODS

Variables were extracted from hospital and EMS databases. EMS providers determined CTAS on-scene, CTAS pre-transport, and CTAS on-arrival at hospital for each patient (N=14,378). The hospital arrival EMS CTAS (CTAS arrival ) score was compared to the initial nursing CTAS score (CTAS initial ) and the final nursing CTAS score (CTAS final ) incuding nursing overrides. Interrater reliability between ED CTAS initial and EMS CTAS arrival scores was assessed. Interrater reliability between ED CTAS final and EMS CTAS arrival scores, as well as proportion of patient encounters with perfect or near-perfect agreement, were evaluated.

RESULTS

Our primary outcome, interrater reliability [kappa=0.437 (p<0.001, 95% CI 0.421-0.452)], indicated moderate agreement. EMS CTAS arrival and ED CTAS initial scores had an exact or within one point match 84.3% of the time. The secondary interrater reliability outcome between hospital arrival EMS CTAS (CTAS arrival ) score and the final ED triage CTAS score (CTAS final ) showed moderate agreement with kappa =0.452 (p<0.001, 95% CI 0.437-0.466).

CONCLUSIONS

Interrater reliability of CTAS scoring between triage nurses and paramedics was moderate in this system-wide implementation study.

Acuity Assessment in Obstetrical Triage

OBJECTIVE

A five-category Obstetrical Triage Acuity Scale (OTAS) was developed with a comprehensive set of obstetrical determinants. The purposes of this study were: (1) to compare the inter-rater reliability (IRR) in tertiary and community hospital settings and measure the intra-rater reliability (ITR) of OTAS; (2) to establish the validity of OTAS; and (3) to present the first revision of OTAS from the National Obstetrical Triage Working Group.

METHODS

To assess IRR, obstetrical triage nurses were randomly selected from London Health Sciences Centre (LHSC) (n = 8), Stratford General Hospital (n = 11), and Chatham General Hospital (n= 7) to assign acuity levels to clinical scenarios based on actual patient visits. At LHSC, a group of nurses were retested at nine months to measure ITR. To assess validity, OTAS acuity level was correlated with measures of resource utilization.

RESULTS

OTAS has significant and comparable IRR in a tertiary care hospital and in two community hospitals. Repeat assessment in a cohort of nurses demonstrated significant ITR. Acuity level correlated significantly with performance of routine and second order laboratory investigations, point of care ultrasound, nursing work load, and health care provider attendance. A National Obstetrical Triage Working Group was formed and guided the first revision. Four acuity modifiers were added based on hemodynamics, respiratory distress, cervical dilatation, and fetal well-being.

CONCLUSION

OTAS is the first obstetrical triage scale with established reliability and validity. OTAS enables standardized assessments of acuity within and across institutions. Further, it facilitates assessment of patient care and flow based on acuity.

Reliability of the Canadian Triage and Acuity Scale: interrater and intrarater agreement from a community and an academic emergency department

OBJECTIVES

The Canadian Triage and Acuity Scale (CTAS) is a five-level triage tool that is used to help prioritize the order in which emergency department (ED) patients should be seen. The objectives of this study were to determine the interrater and intrarater agreement of the 2008 CTAS guideline revisions by triage nurses and to compare agreement between triage nurses working in a small community ED and an academic ED.

METHODS

Seventy-eight triage nurses assigned CTAS scores and free-text presenting complaints for 10 paper-based case scenarios. For five scenarios, the CTAS score should have remained unchanged from previous guidelines, whereas the other five scenarios should have been triaged differently based on the 2008 CTAS first-order modifiers. Thirty-three participants repeated the questionnaire 90 days later, and intrarater agreement was measured.

RESULTS

There was a higher level of agreement (κ  =  0.73; 95% CI 0.68-0.79) for the five case scenarios, which relied on the older 2004 guidelines compared to the scenarios where the 2008 guidelines would have suggested a different triage level (κ  =  0.50; 95% CI 0.42-0.59). For the 10 case scenarios analyzed, the free-text presenting complaints matched the Canadian Emergency Department Information System (CEDIS) list 90.1% of the time (κ  =  0.80; 95% CI 0.76-0.84).

CONCLUSION

The reliability of CTAS scoring by academic and community ED nurses was relatively good; however, the application of the 2008 CTAS revisions appears less reliable than the 2004 CTAS guidelines. These results may be useful to develop educational materials to strengthen reliability and validity for triage scoring using the 2008 CTAS guideline revisions.

The Reliability of the Canadian Triage and Acuity Scale: Meta-analysis

Background:

Although the Canadian Triage and Acuity Scale (CTAS) have been developed since two decades ago, the reliability of the CTAS has not been questioned comparing to moderating variable.

Aims:

The study was to provide a meta-analytic review of the reliability of the CTAS in order to reveal to what extent the CTAS is reliable.

Materials and Methods:

Electronic databases were searched to March 2014. Only studies were included that had reported samples size, reliability coefficients, adequate description of the CTAS reliability assessment. The guidelines for reporting reliability and agreement studies (GRRAS) were used. Two reviewers independently examined abstracts and extracted data. The effect size was obtained by the z-transformation of reliability coefficients. Data were pooled with random-effects models and meta-regression was done based on method of moments estimator.

Results:

Fourteen studies were included. Pooled coefficient for the CTAS was substantial 0.672 (CI 95%: 0.599-0.735). Mistriage is less than 50%. Agreement upon the adult version, among nurse-physician and near countries is higher than pediatrics version, other raters and farther countries, respectively.

Conclusion:

The CTAS showed acceptable level of overall reliability in the emergency department but need more development to reach almost perfect agreement.

Common Diagnoses and Outcomes in Elderly Patients Who Present to the Emergency Department with Non-Specific Complaints

OBJECTIVE

Elderly patients often present to the emergency department (ED) with non-specific complaints. Previous studies indicate that such patients are at greater risk for life-threatening illnesses than similarly aged patients with specific complaints. We evaluated the diagnoses and outcomes of elderly patients presenting with non-specific complaints.

METHODS

Two trained data abstractors independently reviewed all records of patients over 70 years old presenting (to two academic EDs) with non-specific complaints, as defined by the Canadian Emergency Department Information System (CEDIS). Outcomes of interest were ED discharge diagnosis, hospital admission, length of stay, and ED revisit within 30 days.

RESULTS

Of the 743 patients screened for the study, 265 were excluded because they had dizziness, vertigo, or a specific complaint recorded in the triage notes. 419 patients (87.7%) presented with weakness and 59 patients (12.3%) presented with general fatigue or unwellness. The most common diagnoses were urinary tract infection (UTI) (11.3%), transient ischemic attack (TIA) (10.0%), and dehydration (5.6%). There were 11 hospital admissions with median length of stay of five days. Eighty-one (16.9%) patients revisited the ED within 30 days of discharge. Regression analysis indicated that arrival to the ED by ambulance was independently associated with hospital admission.

CONCLUSIONS

Our results suggest that elderly patients presenting to the ED with non-specific complaints are not at high risk for life-threatening illnesses. The most common diagnoses are UTI, TIA, and dehydration. Most patients can be discharged safely, although a relatively high proportion revisit the ED within 30 days.

Triage-based resource allocation and clinical treatment protocol on outcome and length of stay in the emergency department

OBJECTIVE

The present study aimed to determine the relationship between the triage-based resource allocation and clinical treatment (TRACT) protocol and mortality and length of stay (LOS) in ED.

METHODS

This before-and-after study was conducted in an adult, tertiary, teaching hospital ED from August 2008 to July 2012. Patients who were younger than 18 years of age, who were dead on arrival and whose triage information was not available were excluded. TRACT was implemented in August 2010, and the Emergency Severity Index (ESI) was used for triage. Primary and secondary outcomes were ED mortality and ED LOS. Multivariate logistic regression models for ED mortality and multivariable general linear models on the ED LOS were used to compare the before- and after-intervention periods.

RESULTS

For the 155 563 visits over study period, the ED mortality rate was 0.2%, and the ED LOS was 4.6 h (median). The adjusted odds ratios (95% confidence intervals [CIs]) of the TRACT protocol on ED mortality were 0.69 (0.54-0.88) for total patients, 0.42 (0.30-0.59) for ESI 1, 1.04 (0.66-1.65) for ESI 2 and 1.45 (0.76-2.75) for ESI 3 group. The adjusted coefficients (95% CIs) of the TRACT on the ED LOS were -88.1 (-96.9 ∼ -79.2) min for all patients, -44.9 (-72.0 ∼ -17.9) min for ESI level 2 and -104.3 (-114.7 ∼ -94.0) min for ESI level 3.

CONCLUSIONS

The TRACT protocol decreased the ED mortality in ESI 1 group and reduced the ED LOS in ESI levels 2 and 3 groups.

The effect of training on nurse agreement using an electronic triage system

Objectives:

Emergency department (ED) triage prioritizes patients based on urgency of care, and the Canadian Triage and Acuity Scale (CTAS) is the national standard. We describe the inter-rater agreement and manual overrides of nurses using a CTAS-compliant web-based triage tool (eTRIAGE) for 2 different intensities of staff training.

Methods:

This prospective study was conducted in an urban tertiary care ED. In phase 1, eTRIAGE was deployed after a 3-hour training course for 24 triage nurses who were asked to share this knowledge during regular triage shifts with colleagues who had not received training (n = 77). In phase 2, a targeted group of 8 triage nurses underwent further training with eTRIAGE. In each phase, patients were assessed first by the duty triage nurse and then by a blinded independent study nurse, both using eTRIAGE. Inter-rater agreement was calculated using kappa (weighted κ) statistics.

Results:

In phase 1, 569 patients were enrolled with 513 (90.2%) complete records; 577 patients were enrolled in phase 2 with 555 (96.2%) complete records. Inter-rater agreement during phase 1 was moderate (weighted κ = 0.55; 95% confidence interval [CI] 0.49–0.62); agreement improved in phase 2 (weighted κ = 0.65; 95% CI 0.60–0.70). Manual overrides of eTRIAGE scores were infrequent (approximately 10%) during both periods.

Conclusions:

Agreement between study nurses and duty triage nurses, both using eTRIAGE, was moderate to good, with a trend toward improvement with additional training. Triage overrides were infrequent. Continued attempts to refine the triage process and training appear warranted.

Revisions to the Canadian Emergency Department Triage and Acuity Scale Implementation Guidelines

There has been widespread implementation of the Canadian Emergency Department Triage and Acuity Scale (CTAS) across Canada since it was introduced in 1999. This consensus document, developed by the CTAS National Working Group (NWG) of nurse and physician leaders in emergency department (ED) triage, continues to be viewed as a dynamic document that requires modification over time as experience is gained in its application. This article presents the first major modification to the CTAS.

Interrater agreement of Canadian Emergency Department Triage and Acuity Scale scores assigned by base hospital and emergency department nurses

Objective:

We sought to assess the applicability of the Canadian Emergency Department Triage and Acuity Scale (CTAS) in the prehospital setting by comparing CTAS scores assigned during ambulance transportation by base hospital (BH) nurses with CTAS scores given by emergency department (ED) nurses on patients' arrival.

Methods:

We recruited a prospective sample of consecutive patients who were transported to the ED by ambulance between December 2006 and March 2007 for whom a contact was made with the BH. Patients were triaged by the BH nurse with online communication and vital signs transmission. On arrival, patients were blindly triaged again by the ED nurse. We used the quadratic weighted κ statistic to measure the agreement between the 2 CTAS scores.

Results:

Ninety-four patients were triaged twice by 2 nursing teams (9 nurses at the BH and 39 nurses in the ED). The agreement obtained on prehospital and ED CTAS scores was moderate (κ = 0.50; 95% confidence interval 0.37–0.63).

Conclusion:

The moderate interrater agreement we obtained may be a result of the changing conditions of patients during transport or may indicate that CTAS scoring requires direct contact to produce reliable triage scores. Our study casts a serious doubt on the appropriateness of BH nurses performing triage with CTAS in the prehospital setting.

Outcomes for emergency severity index triage implementation in the emergency department

INTRODUCTION

Hospital triage scale in emergency departments needs to be valid and reliable. Lack of sufficient data exists on triage scale rigor in emergency departments of Iran. This study aimed to determine the impact of the emergency severity index (ESI) triage scale in the emergency department.

MATERIALS AND METHODS

A single-center study was conducted. Proportion of triage categories allocated to high-risk patients admitted to high-acuity departments was examined in observational period in June 2012 and May 2013. True triage score was reported based on patients` paper- based scenario questionnaire. Interrater reliability was assessed using unweighted kappa. Concordance among experts, nurses and physicians was examined. The Chi-square test and Kappa statistics was used for statistical analysis.

RESULTS

Triage decisions regarding high-risk patients before and after implementation period are independent from each other (χ2= 22.254; df=1; p<0.05) and more high-risk patients were recognized after implementation of the ESI. Overall agreement and concordance were (79%) and (κ=0.54) among nurses; (71%) and (κ=0.45) among physicians, (85%) and (κ=0.81) among experts, respectively. Correct triage decisions among clinicians were increased after implementation of the ESI.

CONCLUSION

The ESI as valid and reliable tool improving desirable outcomes` in the emergency department has been recommended but it may not reveal optimal outcomes in developing countries comparing to what have been achieved in the developed countries. In addition, patient influx in ESI level II could create considerable controversy with clinicians.

Effect of a triage course on quality of rating triage codes in a group of university nursing students:a before-after observational study

BACKGROUND

Most current triage tools have been tested among hospital nurses groups but there are not similar studies in university setting. In this study we analyzed if a course on a new four-level triage model, triage emergency method (TEM), could improve the quality of rating in a group of nursing students.

METHODS

This observational study was conducted with paper scenarios at the University of Parma, Italy. Fifty students were assigned a triage level to 105 paper scenarios before and after a course on triage and TEM. We used weighted kappa statistics to measure the inter-rater reliability of TEM and assessed its validity by comparing the students' predictions with the triage code rating of a reference standard (a panel of five experts in the new triage method).

RESULTS

Inter-rater reliability was K=0.42 (95%CI: 0.37-0.46) before the course on TEM, and K=0.61 (95%CI: 0.56-0.67) after. The accuracy of students' triage rating for the reference standard triage code was good: 81% (95%CI: 71-90). After the TEM course, the proportion of cases assigned to each acuity triage level was similar for the student group and the panel of experts.

CONCLUSION

Among the group of nursing students, a brief course on triage and on a new in-hospital triage method seems to improve the quality of rating codes. The new triage method shows good inter-rater reliability for rating triage acuity and good accuracy in predicting the triage code rating of the reference standard.

Evaluating implementation of the emergency severity index in a Belgian hospital

INTRODUCTION

Triage aims to categorize patients based on their clinical need and the available departmental resources. To accomplish this goal, one needs to ensure that the implemented triage system is reliable and that staff use it correctly. Therefore this study assessed the ability of Belgium nurses to apply the Emergency Severity Index (ESI), version 4, to hypothetical case scenarios after an educational intervention.

METHODS

An ESI educational intervention was implemented in accordance with the ESI manual. Using paper case scenarios, nurses' interrater agreement was assessed by comparing triage nurse ESI levels with the reference answers noted in the implementation manual. Interrater agreement was measured by the percentage of agreement and Cohen's κ coefficient using different weighting schemes.

RESULTS

Overall, 77.5% of the scenario cases were coded according the ESI guidelines, resulting in a good interrater agreement (κ = 0.72, linear weighted κ = 0.84, quadratic weighted κ = 0.92, and triage-weighted scheme = 0.79). Interrater agreement varied when evaluating each ESI level separately. Undertriage was more common than overtriage. The highest misclassification range (37.8%) occurred in ESI level 2 scenarios, with 99.2% of the misclassifications being undertriaged.

DISCUSSION

Implementation of the ESI into a novel setting guided by a locally developed training program resulted in suboptimal interrater agreement. Existing weighted κ schemes overestimated the interrater agreement between the triage nurse-assigned ESI level and the reference standard. By providing an aggregated measure of agreement, which allows partial agreement, clinically significant misclassification was masked by a misleading "good" interrater agreement.

Implementing an obstetric triage acuity scale: interrater reliability and patient flow analysis

A 5-category Obstetric Triage Acuity Scale (OTAS) was developed with a comprehensive set of obstetrical determinants. The objectives of this study were as follows: (1) to test the interrater reliability of OTAS and (2) to determine the distribution of patient acuity and flow by OTAS level. To test the interrater reliability, 110 triage charts were used to generate vignettes and the consistency of the OTAS level assigned by 8 triage nurses was measured. OTAS performed with substantial (Kappa, 0.61 - 0.77, OTAS 1-4) and near perfect correlation (0.87, OTAS 5). To assess patient flow, the times to primary and secondary health care provider assessments and lengths of stay stratified by acuity were abstracted from the patient management system. Two-thirds of triage visits were low acuity (OTAS 4, 5). There was a decrease in length of stay (median [interquartile range], minutes) as acuity decreased from OTAS 1 (120.0 [156.0] minutes) to OTAS 3 (75.0 [120.8]). The major contributor to length of stay was time to secondary health care provider assessment and this did not change with acuity. The percentage of patients admitted to the antenatal or birthing unit decreased from 80% (OTAS 1) to 12% (OTAS 5). OTAS provides a reliable assessment of acuity and its implementation has allowed for triaging of obstetric patients based on acuity, and a more in-depth assessment of the patient flow. By standardizing assessment, OTAS allows for opportunities to improve performance and make comparisons of patient care and flow across organizations.

Unscheduled return visits to the emergency department: consequences for triage

OBJECTIVES

The objective was to conduct a survey of unscheduled revisits (URs) to the emergency department (ED) within 8 days of a prior visit, to test the hypothesis that patients making these URs are disproportionately likely to suffer short-term mortality or manifest a need for any admission to the hospital (adverse events [AEs]) at the time of the UR, compared to patients triaged at the same level who did not have an unscheduled ED revisit within 8 days.

METHODS

This was a 1-year retrospective study of patients with an UR to the ED of an urban, 1,600-bed tertiary care center and teaching hospital. The criteria for inclusion as an UR were: 1) making an emergency visit to our adult ED during 2008, without being admitted to our hospital nor being transferred to another hospital; and 2) subsequently making an UR to the same ED within 8 days following the first one. Patients who were contacted by members of our staff and specifically asked to make return visits to our ED (such as those who returned for wound care follow-up visits), and those who made more than five visits to our ED during 2008, were excluded. AEs were defined as death or hospitalization within 8 days of the second visit.

RESULTS

During 2008, there were 946 patients with URs (2% of patients treated and released after the first ED visit), and 931 were analyzed (n = 15 missing values). Associated with the second visit, an AE was noted for 276 (30%) patients. Eight variables were significantly associated with AE: age ≥ 65 years, previously diagnosed cancer, previously diagnosed cardiac disease, previously diagnosed psychiatric disease, presence of a relative at the time of the UR, arrival with a letter from a general practitioner at the time of the UR, a higher level of severity assigned at triage for the UR than for the first ED visit, and having had blood sample analysis performed during the first visit. The median triage score for the UR was not significantly different from that group's median triage score for the first ED visit, whereas the proportion of admissions to the hospital (29%) or to the intensive care unit (ICU; 2%) was greater overall in the UR group than in the patients making their first ED visit.

CONCLUSIONS

The authors observed that 2% of patients had an UR. This UR population was at greater risk of AE at the time of their URs compared to their initial visits, but the median triage nurse score was not significantly different between the first visit and the UR. This suggests that the triage score should be systematically upgraded for UR patients.

Emergency department triage: do experienced nurses agree on triage scores?

BACKGROUND

The reproducibility of the Canadian Triage & Acuity Scale (CTAS), designed and introduced in the late 1990s in all Canadian emergency departments (EDs), has been studied mostly using measures of interrater agreement. However, each of these studies shares a common limitation: the nurses had received fresh CTAS training, which is likely to have led to an overestimation of the reproducibility of CTAS.

OBJECTIVES

This study aims to assess the interrater reliability of the CTAS in current clinical practice, that is, as used by experienced ED nurses without recent certification or recertification.

METHODS

A prospective sample of 100 patients arriving by ambulance was identified and yielded a set of 100 written scenarios. Five experienced ED nurses reviewed and blindly assigned a CTAS score to each scenario. The agreement among nurses was measured using the Kappa statistic calculated with quadratic weights. Kappa values were generated for each pair of nurses and a global Kappa coefficient was calculated to measure overall agreement.

RESULTS

Overall interrater agreement was moderate, with a global Kappa of 0.44 (95% confidence interval 0.40-0.48). However, pairwise, Kappa values were heterogeneous (0.30 to 0.61, p=0.0013).

CONCLUSIONS

The moderate interrater agreement observed in this study is disappointingly low and suggests that CTAS reliability may be lower than expected, and this warrants further research. Intra-observer reliability of CTAS should be ascertained more extensively among experienced nurses, and a future evaluation should involve several institutions.

The South African Triage Scale (adult version) provides reliable acuity ratings

OBJECTIVE

To estimate the inter- and intra-rater reliability of triage ratings on Emergency Centre patients by South African nurses and doctors using the South African Triage Scale (SATS).

METHODS

A cross-sectional reliability study was performed. Five emergency physicians and ten enrolled nursing assistants independently assigned triage categories to 100 written vignettes unaware of the ratings given by others. Four different quantitative reliability measures were calculated and compared. Graphical displays portrayed rating distributions for vignettes with mean ratings at different acuity categories.

RESULTS

The estimated quadratically weighted kappa for the group of emergency physicians was 0.76 (95% CI: 0.67-0.84) and for the group of nurses 0.66 (95% CI: 0.58-0.74). These values were close to the estimated intra-class correlation coefficients. For intra-rater reliability, the average exact agreement was 84%. The graphical displays showed that the least variability was evident in the vignettes that had a mean rating of 'emergency', 'very urgent' or 'routine'.

CONCLUSION

This study indicates good inter- and intra-rater reliability among nurses and doctors using the SATS. It suggests that the SATS is reliably applied, and supports the feasibility of further implementation of the SATS in similar settings.

Adherence to the guideline 'Triage in emergency departments': a survey of Dutch emergency departments

AIMS AND OBJECTIVES

The aim of this study was to evaluate the adherence to the 2004 guideline Triage in emergency departments three years after dissemination in Dutch emergency departments.

BACKGROUND

In 2004, a Dutch guideline Triage in emergency departments was developed. Triage is the first step performed by nurses when a patient arrives at an emergency department. It includes the prioritisation of patients to ensure that doctors see patients with the highest medical needs first. Although the national guideline was developed and disseminated in 2004, three years on there was no insight into the level of implementation of the guideline in practice.

DESIGN

A cross-sectional descriptive design.

METHODS

In February 2007, data were collected from ward managers and triage nurses at all emergency departments in the Netherlands (n = 108), using a questionnaire that was based on the recommendations and performance indicators of the guideline.

RESULTS

In total, 79% of all 108 Dutch emergency departments responded. The main findings showed that over 31% of the emergency departments did not use a triage system. Emergency departments using the Manchester Triage System had a mean adherence rate of 61% of the guideline's recommendations and emergency departments using the Emergency System Index adhered to a mean of 65%.

CONCLUSION

The guideline Triage in emergency departments was disseminated in 2004, but results from this study indicate that an improvement in adherence to this guideline is required.

RELEVANCE TO CLINICAL PRACTICE

Adherence to guidelines is important to standardise practice to ensure that patients receive the appropriate treatment and to improve quality of care.

Emergency department abnormal vital sign "triggers" program improves time to therapy

BACKGROUND

Implementation of rapid response systems to identify deteriorating patients in the inpatient setting has demonstrated improved patient outcomes. A "trigger" system using vital sign abnormalities to initiate evaluation by physician was recently described as an effective rapid response method.

OBJECTIVES

The objective was to evaluate the effect of a triage-based trigger system on the primary outcome of time to physician evaluation and the secondary outcomes of therapeutic intervention, antibiotics, and disposition in emergency department (ED) patients.

METHODS

A separate-samples pre- and postintervention study was conducted using retrospective chart review of outcomes in ED patients for three arbitrarily selected 5-day periods in 2007 (pretriggers) and 2008 (posttriggers). There were 2,165 and 2,212 charts in the pre- and posttriggers chart review, with 71 and 79 patients meeting trigger criteria. Trigger criteria used to identify patients at triage were: heart rate of <40 or >130 beats/min, respiratory rate of <8 or >30 breaths/min, systolic blood pressure of <90 mm Hg, and oxygen saturation of <90% on room air. Median times (in minutes) were compared between pre- and posttrigger groups with interquartile ranges (IQRs 25-75), with the Wilcoxon rank sum test used to determine statistical significance.

RESULTS

Overall median times were decreased among the posttriggers group. Median times to physician evaluation (21 minutes [IQR = 13-41 minutes] vs. 11 minutes [IQR = 5-21 minutes]; p < 0.001), first intervention (58 minutes [IQR = 20-139 minutes] vs. 26 minutes [IQR = 11-71 minutes]; p < 0.01), and antibiotics (110 minutes [IQR = 74-171 minutes] vs. 69 minutes [IQR = 23-130 minutes]; p < 0.01) were significant. Median times to disposition (177 minutes [IQR = 121-303 minutes] vs. 162 minutes [IQR = 114-230 minutes]; p = 0.18) were not significant.

CONCLUSIONS

Implementation of an ED triggers program allows for more rapid time to physician evaluation, therapeutic intervention, and antibiotics.

The influence of emergency medical services transport on Emergency Severity Index triage level for patients with abdominal pain

OBJECTIVES

The Emergency Severity Index (ESI) is a prospectively validated, five-level emergency department (ED) triage system designed to match triage acuity to both patient acuity and appropriate resource allocation. The study hypothesis was that, in practice, there exists an inappropriate bias toward triaging patients with abdominal pain to a higher ESI level based solely upon their mode of arrival to the ED.

METHODS

The authors performed a retrospective case-control study of patients presenting with abdominal pain. Patients were matched on sex, age (± 5 years), and date of arrival. Cases were those patients triaged to a Level 2, and controls were those triaged as Level 3. Conditional multiple variable logistic regression was used to evaluate the effect of the following variables on the odds of being triaged as Level 2: mode of arrival, systolic blood pressure (<90 mm Hg; normal, >140 mm Hg), heart rate, severe pain score (≥ 8 of 10), fever, race, history of cancer, and previous abdominal surgery. Age was also included in the regression modeling to confirm that matching was adequate. One-hundred cases and 100 controls were necessary to provide adequate sample size. A backward modeling technique was used, requiring a p < 0.05 for retention.

RESULTS

Of the 200 subjects, 52 arrived by emergency medical services (EMS) and 148 walked in. After matching for sex, age, and date of arrival, and after adjusting for heart rate, cancer diagnosis, and severe pain, the odds ratio (OR) for being triaged ESI Level 2 was 7.19 (95% confidence interval [CI] = 2.75 to 18.8, p < 0.0001) for EMS patients compared to walk-in patients. The admission rate for Level 2 patients was not different from that of Level 3 patients (49% vs. 35% of Level 3 patients, p = 0.06), but EMS patients were more likely to be admitted, regardless of ESI level assignment (65% vs. 34%, p < 0.001).

CONCLUSIONS

After adjusting for covariates, EMS patients with abdominal pain were more likely to be triaged to a higher acuity level. Triage level was not associated with admission, but patients arriving by EMS were more likely to be admitted. This may indicate that the effect of EMS arrival on triage level assignment is actually appropriate. Further research is necessary to validate whether mode of arrival should be incorporated in the initial ESI triage acuity assignment.

Modern triage in the emergency department

BACKGROUND

Because the volume of patient admissions to an emergency department (ED) cannot be precisely planned, the available resources may become overwhelmed at times ("crowding"), with resulting risks for patient safety. The aim of this study is to identify modern triage instruments and assess their validity and reliability.

METHODS

Review of selected literature retrieved by a search on the terms "emergency department" and "triage."

RESULTS

Emergency departments around the world use different triage systems to assess the severity of incoming patients' conditions and assign treatment priorities. Our study identified four such instruments: the Australasian Triage Scale (ATS), the Canadian Triage and Acuity Scale (CTAS), the Manchester Triage System (MTS), and the Emergency Severity Index (ESI). Triage instruments with 5 levels are superior to those with 3 levels in both validity and reliability (p<0.01). Good to very good reliability has been shown for the best-studied instruments, CTAS and ESI (κ-statistics: 0.7 to 0.95), while ATS and MTS have been found to be only moderately reliable (κ-statistics: 0.3 to 0.6). MTS and ESI are both available in German; of these two, only the ESI has been validated in German-speaking countries.

CONCLUSION

Five-level triage systems are valid and reliable methods for assessment of the severity of incoming patients' conditions by nursing staff in the emergency department. They should be used in German emergency departments to assign treatment priorities in a structured and dependable fashion.

An observational study of patients triaged in category 5 of the Emergency Severity Index

OBJECTIVE

Patients triaged in category 5 of the Emergency Severity Index (ESI) do not need any resources before discharge from the emergency department (ED). We studied the characteristics of these patients and focused on those who were admitted or sent to the outpatient department after their ED visit.

METHODS

A retrospective observational study was conducted on 117 740 patient presentations. Patients were included in the study when they were triaged with the ESI and presented to one of the two EDs under study between 1 September 2004 and 1 June 2006.

RESULTS

Overall, 22.2% of the patients were triaged in ESI 5. Patients aged less than 40 years, women, and self-referred patients were most likely triaged in ESI 5, as well as patients presenting with complaints such as 'checkup appointments at the ED' and 'complaints of the skin'. Patients triaged in ESI 5 who were admitted or sent to the outpatient department were most likely elderly (aged above 65 years) and referred patients. They were also more likely to present with complaints such as 'postoperative complications, wound care problems, and plaster problems' and 'complaints of the genitourinary system'.

CONCLUSION

Although younger patients and women were more likely triaged in ESI 5, patients within this category who were admitted or sent to the outpatient department were more likely elderly and referred patients. Being admitted or sent to the outpatient department and triaged in ESI 5 indicates undertriage. Revision of the system is required to properly account for these patient groups.

Calculating kappas from adjusted data improved the comparability of the reliability of triage systems: a comparative study

OBJECTIVES

It is difficult to compare the reliability of triage systems with the kappa statistic. In this article, a method for comparing triage systems was developed and applied to previously conducted triage reliability studies.

STUDY DESIGN AND SETTING

From simulations with theoretical distributions, the minimum, normal, and maximum weighted kappa for 3- to 5-level triage systems were computed. To compare the reliability of triage systems in previously conducted triage reliability studies, the normal kappa was calculated. Furthermore, the reported quadratically weighted kappas were compared with the minimum, normal, and maximum weighted kappa to characterize the degree and direction of skewness of the data.

RESULTS

The normal kappa was higher in 3-level triage systems (median: κ=0.84) compared with 4-level (median: κ=0.37) and 5-level (median: κ=0.57) systems. In 3-level triage systems, the percentages observed agreement were unequally distributed, which resulted in small quadratically weighted kappas. In 4- and 5-level systems, the percentages observed agreement were more equally distributed compared with 3-level systems, which resulted in higher quadratically weighted kappa values.

CONCLUSION

When comparing triage systems with different numbers of categories, one should report both the normal and quadratically weighted kappa. Calculating normal kappas from previously conducted triage reliability studies revealed substantial theoretical differences in interrater reliability of triage systems than previously reported.

Reliability and validity of two four-level emergency triage systems

OBJECTIVE

To measure and compare the reliability and predictive validity of a four-level triage system (I-4L) and the new four-level model triage emergency method (TEM).

METHODS

This observational study was conducted in an urban hospital. Ten nurses were randomly selected to assign a triage level to 189 paper scenarios, using either the I-4L model (5 nurses) or the TEM model (5 nurses). We used weighted kappa statistics to measure the interrater and intrarater reliability of each triage tool and assessed the validity of each models based on the accuracy in predicting admission.

RESULTS

Interrater reliability was kappa=0.73 [95% CI (confidence interval): 0.59-0.87] and kappa=0.79 (95% CI: 0.65-0.93) with I-4L and TEM, respectively. Intrarater reliability was kappa=0.82 (95% CI: 0.67-0.96) and kappa=0.78 (95% CI: 0.62-0.93), respectively. The accuracy of triage rating for admission prediction was similarly good with I-4L and TEM, namely, 79% (95% CI: 74-85) and 77% (95% CI: 74-85). The proportion of patients admitted per triage level was similar with the two models.

CONCLUSION

The interrater and intrarater reliability for rating triage acuity and for accuracy in patient admission prediction was good with both models. Performance with the new model was similar to that of I-4L despite the nurses' short experience. The new TEM model has the advantage of predicting utilization of emergency department resources.

The development of an independent rater system to assess residents' competence in invasive procedures

PURPOSE

To design an independent rater (IR) direct observation system to monitor invasive procedures performed by residents in the hospital setting.

METHOD

The authors recruited, trained, and tested nonphysicians to become IRs for an Agency for Healthcare Research and Quality-funded study evaluating the impact of partial task simulation training of ultrasound-guided central venous catheter (CVC) insertion on skills transfer at a major academic medical center. IR applicants completed four hours of training: a two-hour didactic session and a two-hour testing session, including observation of 5 of 10 choreographed CVC insertion videotapes and completion of a 50-data-point procedural checklist. Eligibility to be hired as an IR included timing the procedure accurately, detecting technical errors and complications, and completing the procedural checklist accurately.

RESULTS

Thirty-eight IR trainees completed the training module and videotape examinations. Twenty-seven (71%) trainees met criteria to be hired IRs by accurately assessing the duration of the procedure to within one minute, validating the checklist to within 95% accuracy, and detecting technical errors/complications to within a 3% margin of error. The authors found no association between educational level and hired status, and all 13 IRs assessed after the study had maintained their skills.

CONCLUSIONS

Recent innovations in procedural training with partial task simulation trainers necessitate developing methods to measure skills transfer from the simulator to the clinical setting. This description of a nonphysician IR direct observation system for CVC insertion offers a feasible tool that may be generalized to monitoring other invasive procedures.

Predictive validity comparison of two five-level triage acuity scales

INTRODUCTION

Each of the two most commonly used five-level triage tools in North America, the Emergency Severity Index and the Canadian Triage and Acuity Scale have been used as a measure of emergency department resource utilization in addition to acuity. In both cases, it is believed that patients triaged as having a higher level of acuity require a greater number of emergency department resources. We compared the ability of each tool to predict the emergency department resources for each emergency department visit and associated hospital admission and in-hospital mortality rates.

METHODS

This is an observational, cohort study of a population-based random sample of patients triaged at two emergency departments over a 4-month period. Correlational analyses were performed to examine the relationship between the triage assessment and: (i) resource utilization, (ii) hospital admission, and (iii) in-hospital mortality.

RESULTS

From 486 patients, analyses revealed the greatest correlation was between Emergency Severity Index and diagnostic resources [-0.54 (95% confidence intervals: -0.58, -0.50)] and the poorest correlation was between Canadian Triage and Acuity Scale and mortality [-0.16 (95% confidence intervals: -0.20, -0.12)]. No statistically significant differences (P<0.005) were observed between each tool 's ability to predict any of the outcomes measured.

CONCLUSION

No statistically significant difference was observed in the ability of Emergency Severity Index v. 3 and Canadian Triage and Acuity Scale to predict emergency department resource utilization or immediate patient outcomes. This ability is, at best, only moderate indicating that other, more accurate tools than measures of triage acuity are required for this purpose.

Reliability of a computerized version of the Pediatric Canadian Triage and Acuity Scale

BACKGROUND

The use of a standardized triage tool allows better comparison of the patients; a computerized version could theoretically improve its reliability.

OBJECTIVES

To compare the interrater agreement of the Pediatric Canadian Triage and Acuity Scale (PedCTAS) and a computerized version (Staturg).

METHODS

A two-phase experimental study was conducted to compare the interrater agreement between nurses assigning triage level to written case scenarios using either traditional PedCTAS or Staturg. Participants were nurses with at least one year of experience in pediatric emergency medicine and trained at triage. Each of the 54 scenarios was evaluated first by all nurses using either one of the strategies. Four weeks later, they evaluated the same scenarios using the other tool. The primary outcome was the interrater agreement measured using kappa score.

RESULTS

Eighteen of the 29 eligible nurses participated in the study. The computerized triage tool showed a better interrater agreement, with a Staturg kappa score of 0.55 (95% confidence interval = 0.53 to 0.57) versus a PedCTAS kappa score of 0.51 (95% confidence interval = 0.49 to 0.53). The computerized version was also associated with higher agreements for scenarios describing patients with the highest severity of triage (kappa score of 0.72 vs. 0.55 for level 1; kappa score of 0.70 vs. 0.51 for level 2).

CONCLUSIONS

A computerized version of the PedCTAS showed a statistically significant improvement in the interrater agreement for nurses evaluating the triage level of 54 clinical scenarios, but this difference has probably small clinical significance.