The effect of training on nurse agreement using an electronic triage system

How clinicians make decisions for patient management plans in telehealth

AIM

This systematic integrative literature review explores how clinicians make decisions for patient management plans in telehealth.

BACKGROUND

Telehealth is a modality of care that has gained popularity due to the development of digital technology and the COVID-19 pandemic. It is recognized that telehealth, compared to traditional clinical settings, carries a higher risk to patients due to its virtual characteristics. Even though the landscape of healthcare service is increasingly moving towards virtual systems, the decision-making process in telehealth remains not fully understood.

DESIGN

A systematic integrative review.

DATA SOURCES

Databases include CINAHL, APA PsycInfo, Academic Search Complete, PubMed, Web of Science and Google Scholar.

REVIEW METHODS

This systematic integrative review method was informed by Whittemore and Knafl (2005). The databases were initially searched with keywords in November 2022 and then repeated in October 2023. Thematic synthesis was conducted to analyse and synthesize the data.

RESULTS

The search identified 382 articles. After screening, only 10 articles met the eligibility criteria and were included. Five studies were qualitative, one quantitative and four were mixed methods. Five main themes relevant to decision-making processes in telehealth were identified: characteristics of decision-making in telehealth, patient factor, clinician factor, CDSS factor and external influencing factor.

CONCLUSIONS

The decision-making process in telehealth is a complicated cognitive process influenced by multi-faceted components, including patient factors, clinician factors, external influencing factors and technological factors.

IMPACT

Telehealth carries higher risk and uncertainty than face-to-face encounters. CDSS, rather than bringing unification and clarity, seems to bring more divergence and ambiguity. Some of the clinical reasoning processes in telehealth remain unknown and need to be verbalized and made transparent, to prepare junior clinicians with skills to minimize risks associated with telehealth.

PATIENT OR PUBLIC CONTRIBUTION

Not applicable.

Development of a machine learning-based acuity score prediction model for virtual care settings

OBJECTIVE

Healthcare is increasingly digitized, yet remote and automated machine learning (ML) triage prediction systems for virtual urgent care use remain limited. The Canadian Triage and Acuity Scale (CTAS) is the gold standard triage tool for in-person care in Canada. The current work describes the development of a ML-based acuity score modelled after the CTAS system.

METHODS

The ML-based acuity score model was developed using 2,460,109 de-identified patient-level encounter records from three large healthcare organizations (Ontario, Canada). Data included presenting complaint, clinical modifiers, age, sex, and self-reported pain. 2,041,987 records were high acuity (CTAS 1-3) and 416,870 records were low acuity (CTAS 4-5). Five models were trained: decision tree, k-nearest neighbors, random forest, gradient boosting regressor, and neural net. The outcome variable of interest was the acuity score predicted by the ML system compared to the CTAS score assigned by the triage nurse.

RESULTS

Gradient boosting regressor demonstrated the greatest prediction accuracy. This final model was tuned toward up triaging to minimize patient risk if adopted into the clinical context. The algorithm predicted the same score in 47.4% of cases, and the same or more acute score in 95.0% of cases.

CONCLUSIONS

The ML algorithm shows reasonable predictive accuracy and high predictive safety and was developed using the largest dataset of its kind to date. Future work will involve conducting a pilot study to validate and prospectively assess reliability of the ML algorithm to assign acuity scores remotely.

Daily triage audit can improve nurses' triage stratification: A pre-post study

AIMS

The objective was to evaluate whether the error rate in the application of the triage system decreased after the introduction of daily auditing, and it was also evaluated if the agreement rate between physician and nurse on triage priority levels increased after the introduction of daily auditing and if the error-related variables in the pre-intervention period changed in the post-intervention period.

DESIGN

A quasi-experimental study was performed with a pre-post design, between June 2019 and June 2021 in one emergency department.

METHODS

The accuracy and error rate of triage in the pre- and post-intervention period were compared. Univariate and multivariate logistic regression analyses were performed to explore the relationships between the variables related to the error. The comparison between the priority level assigned by the physician and the triage nurse was analysed using Cohen's K.

RESULTS

Nine hundred four patients were enrolled in the pre-intervention period and 869 in the post-intervention period. The error rate in the pre-intervention period was 23.3% and in the post-intervention period was 9.7%. The concordance between the degree of priority expressed by the physician and the nurse varied from a quadratically weighted Cohen's K of 0.447 in the pre-intervention period to 0.881 in the post-intervention period.

CONCLUSION

Daily auditing is a clinical procedure that improves the nurse's application of the triage system. Daily auditing has reduced errors by the nurse, improving performance and concordance with the physician.

IMPACT

Triage systems are a key point for the stratification of the priority level of patients and it is therefore evident that they maintain high-quality standards. Through the practice of daily auditing, not only a reduction in the error rate, which ensures patient safety, but also an improvement in triage performance has been demonstrated.

NO PATIENT OR PUBLIC CONTRIBUTION

The study did not involve any patients during its conduction.

Strategies to improve the quality of nurse triage in emergency departments: A realist review protocol

AIM

The purpose of this realist review was to assess what works, for whom and in what context, regarding strategies that influence nurses' behaviour to improve triage quality in emergency departments (ED).

DESIGN

Realist review protocol.

METHODS

This protocol follows the PRISMA-P statement and will include any type of study on strategies to improve the triage process in the ED (using recognized and validated triage scales). The included studies were examined for scientific quality using the Mixed Methods Appraisal Tool. The framework for this realist review is based on the Behaviour Change Wheel (BCW) and the context-mechanism-outcome (CMO) models.

DISCUSSION

Nurses and ED decision makers will be informed on the evidence regarding strategies to improve the quality of triage and the factors required to maximize their effectiveness. Research gaps may also be identified to guide future research projects on the adoption of best practices in ED nursing triage.

Educational intervention in triage with the Swedish triage scale RETTS©, with focus on specialist nurse students in ambulance and emergency care - A cross-sectional study

AIM

To determine the reliability of application of the RETTS© triage scale after an educational intervention using paper-based scenarios in emergency care education.

BACKGROUND

Knowledge about and education in triage are important factors in triagescale implementation. Presenting students with a large number of triage scenarios is a common part of triage education.

METHODS

In this prospective cross-sectional study at two universities students undergoing education in emergency care used RETTS© to assess triage level in 46 paper-based scenarios.

RESULTS

57 students in the study made 2590 final triage decisions. Fleiss Kappa for final triage was 0.411 which is in the lower range of moderate agreement. In 25 of 46 (53.4%) scenarios, final triage levels did not agree about whether the case was stable or unstable.

CONCLUSION/IMPLICATIONS

Application of the RETTS© triage scale after an educational intervention with paper-based simulation in emergency care education resulted in moderate agreement about the final levels of triage.

Safety assessment of a redirection program using an electronic application for low-acuity patients visiting an emergency department

Background

Emergency departments (EDs) are operating at or above capacity, which has negative consequences on patients in terms of quality of care and morbi-mortality. Redirection strategies for low-acuity ED patients to primary care practices are usually based on subjective eligibility criteria that sometimes necessitate formal medical assessment. Literature investigating the effect of those interventions is equivocal. The aim of the present study was to assess the safety of a redirection process using an electronic clinical support system used by the triage nurse without physician assessment.

Methods

A single cohort observational study was performed in the ED of a level 1 academic trauma center. All low-acuity patients redirected to nearby clinics through a clinical decision support system (February–August 2017) were included. This system uses different sets of medical prerequisites to identify patients eligible to redirection. Data on safety and patient experience were collected through phone questionnaires on day 2 and 10 after ED visit. The primary endpoint was the rate of redirected patients returning to any ED for an unexpected visit within 48 h. Secondary endpoints were the incidence of 7-day return visit and satisfaction rates.

Results

A total of 980 redirected low-acuity patients were included over the period: 18 patients (2.8%) returned unexpectedly to an ED within 48 h and 31 patients (4.8%) within 7 days. No hospital admission or death were reported within 7 days following the first ED visit. Among redirected patients, 81% were satisfied with care provided by the clinic staff.

Conclusion

The implementation of a specific electronic-guided decision support redirection protocol appeared to provide safe deferral to nearby clinics for redirected low-acuity patients. EDs are pivotal elements of the healthcare system pathway and redirection process could represent an interesting tool to improve the care to low-acuity patients.

The Effect of Human Supervision on an Electronic Implementation of the Canadian Triage Acuity Scale (CTAS)

BACKGROUND

Most electronic emergency department (ED) triage systems allow nurses to modify computer-generated triage scores. It is currently unclear how this affects triage validity.

OBJECTIVE

Are nurse-generated triage scores more strongly associated with rates of admission, intensive care unit (ICU) consultation, and mortality than computer-generated scores?

METHODS

Retrospective observational cohort study of all adult visits to a tertiary ED. An electronic implementation of the Canadian Triage Acuity Scale (CTAS) generated a CTAS score for each visit. In some cases, the triage nurse overwrote the computer-generated CTAS score with a score they felt was more appropriate. Among visits with nurse-modified triage scores, we compared the rate of acuity-related outcomes (mortality, ICU consultation, hospital admission) in each CTAS level as categorized by nurse-generated vs. computer-generated scores.

RESULTS

In a cohort of 229,744 patients, 19,566 (8.51%) had nurse-modified triage scores. Most modifications consisted of assigning a higher acuity triage score than recommended by the computer. Visits with triage scores 1-2 according to the nurse-generated scores had the same or higher rates of the acuity outcomes than visits that were CTAS 1-2 according to the computer-generated CTAS scores. Conversely, visits with triage scores 4-5 according to the nurse-generated scores had lower rates of the outcomes than visits that were CTAS 4-5 according to the computer-generated CTAS scores.

CONCLUSIONS

Nursing supervision of the computer-automated CTAS triage system was associated with fewer hospital admissions, ICU consultations, and deaths in the triage score 4-5 categories, suggesting a safer triage process than the automated CTAS algorithm alone.

Developing machine learning models to personalize care levels among emergency room patients for hospital admission

Objective

To develop prediction models for intensive care unit (ICU) vs non-ICU level-of-care need within 24 hours of inpatient admission for emergency department (ED) patients using electronic health record data.

Materials and Methods

Using records of 41 654 ED visits to a tertiary academic center from 2015 to 2019, we tested 4 algorithms—feed-forward neural networks, regularized regression, random forests, and gradient-boosted trees—to predict ICU vs non-ICU level-of-care within 24 hours and at the 24th hour following admission. Simple-feature models included patient demographics, Emergency Severity Index (ESI), and vital sign summary. Complex-feature models added all vital signs, lab results, and counts of diagnosis, imaging, procedures, medications, and lab orders.

Results

The best-performing model, a gradient-boosted tree using a full feature set, achieved an AUROC of 0.88 (95%CI: 0.87–0.89) and AUPRC of 0.65 (95%CI: 0.63–0.68) for predicting ICU care need within 24 hours of admission. The logistic regression model using ESI achieved an AUROC of 0.67 (95%CI: 0.65–0.70) and AUPRC of 0.37 (95%CI: 0.35–0.40). Using a discrimination threshold, such as 0.6, the positive predictive value, negative predictive value, sensitivity, and specificity were 85%, 89%, 30%, and 99%, respectively. Vital signs were the most important predictors.

Discussion and Conclusions

Undertriaging admitted ED patients who subsequently require ICU care is common and associated with poorer outcomes. Machine learning models using readily available electronic health record data predict subsequent need for ICU admission with good discrimination, substantially better than the benchmarking ESI system. The results could be used in a multitiered clinical decision-support system to improve ED triage.

Coupling Wearable Devices and Decision Theory in the United States Emergency Department Triage Process: A Narrative Review

This research was motivated by the nurses' decision-making process in the current emergency department (ED) triage process in the United States. It explores how continuous vital signs monitoring can be integrated into the ED. The article presents four shortcomings on current ED triage systems and proposes a new conceptual clinical decision support model that exploits the benefits of combining wireless wearable devices with Multi-Attribute Utility Theory to address those shortcomings. A literature review was conducted using various engineering and medical research databases, analyzing current practices and identifying potential improvement opportunities. The results from the literature review show that advancements in wireless wearable devices provide opportunities to enhance current ED processes by monitoring patients while they wait after triage and, therefore, reduce the risk of an adverse event. A dynamic mathematical decision support model to prioritize patients is presented, creating a feedback loop in the ED. The coupling of wearable devices (to collect data) with decision theory (to synthesize and organize the information) can assist in reducing sources of uncertainty inherent to ED systems. The authors also address the feasibility of the proposed conceptual model.

Performance of a three-level triage scale in live triage encounters in an emergency department in Hong Kong

Background

Despite its continued use in many low-volume emergency departments (EDs), 3-level triage systems have not been extensively studied, especially on live triage cases. We have modified from the Australasian Triage Scale and developed a 3-level triage scale, and sought to evaluate its validity, reliability, and over- and under-triage rates in real patient encounters in our setting.

Method

This was a cross-sectional study in a single ED with 24,000 attendances per year. At triage, each patient was simultaneously assessed by a triage nurse, an adjudicator (the “criterion standard”), and a study nurse independently. Predictive validity was determined by comparing clinical outcomes, such as hospitalization, across triage levels. The discriminating performance of the triage tool in identifying patients requiring earlier medical attention was determined. Inter-observer reliability between the triage nurse and criterion standard, and across providers were determined using kappa statistics.

Results

In total, 453 triage ratings of 151 triage cases, involving 17 ED triage nurses and 57 nurse pairs, were analysed. The proportion of hospital admission significantly increased with a higher triage rating. The performance of the scale in identifying patients requiring earlier medical attention was as follows: sensitivity, 68.2% (95% CI 45.1–86.1%); specificity, 99.2% (95% CI 95.8–100%); positive predictive value, 93.8% (95% CI 67.6–99.1%); and negative predictive value, 94.8% (95% CI 90.8–97.1%). The over-triage and under-triage rates were 0.7% and 4.6%, respectively. Agreement between the triage nurse and criterion standard was substantial (quadratic-weighted kappa = 0.76, 95% CI, 0.60–0.92, p < 0.001), so was the agreement across nurses (quadratic-weighted kappa = 0.81, 95% CI 0.65–0.97, p < 0.001).

Conclusions

The 3-level triage system appears to have good validity and reasonable reliability in a low-volume ED setting. Further studies comparing 3-level and prevailing 5-level triage scales in live triage encounters and different ED settings are warranted.

Utilization of an Electronic Triage System by Emergency Department Nurses

Introduction

Emergency departments use triage systems to prioritize patients according to the severity of their condition. The Electronic Canadian Triage and Acuity Scale (E-CTAS) is a popular system that categorizes patients into five levels to manage patient flow and prioritize patient access to health-care services.

Methods

We assessed the factors that influence E-CTAS usage in emergency departments in Eastern Saudi Arabia. Seventy-one nurses were included from two emergency departments that adopted E-CTAS. We used the technology acceptance model (TAM) to assess the influencing factors. The TAM was reliable in the study setting (Cronbach’s α = 0.87).

Results

All of the TAM domains were significantly related to the usage of E-CTAS: perceived ease of use, perceived usefulness, importance of training, social influence, behavior intention, and attitude. We also showed that E-CTAS use significantly increased with years of experience and training.

Discussion

Many factors influenced the use of this electronic triage system. Focusing on these factors in future electronic triage system implementations might increase the hospital staff’s compliance, thus improving accuracy and better organizing the patient flow in emergency departments.

Predicting Intensive Care Unit admission among patients presenting to the emergency department using machine learning and natural language processing

The risk stratification of patients in the emergency department begins at triage. It is vital to stratify patients early based on their severity, since undertriage can lead to increased morbidity, mortality and costs. Our aim was to present a new approach to assist healthcare professionals at triage in the stratification of patients and in identifying those with higher risk of ICU admission. Adult patients assigned Manchester Triage System (MTS) or Emergency Severity Index (ESI) 1 to 3 from a Portuguese and a United States Emergency Departments were analyzed. Variables routinely collected at triage were used and natural language processing was applied to the patient chief complaint. Stratified random sampling was applied to split the data in train (70%) and test (30%) sets and 10-fold cross validation was performed for model training. Logistic regression, random forests, and a random undersampling boosting algorithm were used. We compared the performance obtained with the reference model—using only triage priorities—with the models using additional variables. For both hospitals, a logistic regression model achieved higher overall performance, yielding areas under the receiver operating characteristic and precision-recall curves of 0.91 (95% CI 0.90-0.92) and 0.30 (95% CI 0.27-0.33) for the United States hospital and of 0.85 (95% CI 0.83-0.86) and 0.06 (95% CI 0.05-0.07) for the Portuguese hospital. Heart rate, pulse oximetry, respiratory rate and systolic blood pressure were the most important predictors of ICU admission. Compared to the reference models, the models using clinical variables and the chief complaint presented higher recall for patients assigned MTS/ESI 3 and can identify patients assigned MTS/ESI 3 who are at risk for ICU admission.

Do prehospital providers and emergency nurses agree on triage assignment?: an efficacy study

OBJECTIVES

The aim of this study was to investigate the agreement on triage level between prehospital providers and emergency department (ED) nurses in clinical practice when using the same triage system. The objectives were as follows: (a) What is the agreement of triage between prehospital providers and ED nurses, when using Danish Emergency Process Triage (DEPT) correctly? (b) Which part of the triage process yields the highest agreement regarding the final triage?

METHODS

The study was a prospective and observational efficacy study. Patients transported to the ED by ambulances were included. They were triaged by prehospital providers while being transported by ambulance to the ED, and by ED nurses upon arrival. Triage was done using the DEPT - a five-level triage system based on vital signs and a presenting complaint algorithm. An agreement analysis was performed.

RESULTS

DEPT was used correctly by both professions in 292 patients. In 182 (62%) patients the prehospital providers and the ED nurses agreed on the same triage level. This equals to κ=0.47 [95% confidence interval (CI): 0.41-0.56]. When considering the triage based on vital signs the agreement was 72% (κ=0.46; 95% CI: 0.41-0.47), and based on presenting complaint the agreement was 46% (κ=0.41; 95% CI: 0.37-0.44).

CONCLUSION

There was a moderate interrater agreement on triage assignment between ED nurses and prehospital providers. They agreed on final triage more often if they agreed on triage based on vital signs rather than presenting complaints.

Exploring the sorting of patients in community health centres across Gauteng Province, South Africa

BACKGROUND

Primary health care worldwide faces large numbers of patients daily. Poor waiting times, low patient satisfaction and staff burnout are some problems facing such facilities. Limited research has been done on sorting patients in non-emergency settings in Africa. This research looked at community health centres (CHCs) in Gauteng Province, South Africa where queues appear to be poorly managed and patients waiting for hours. This study explores the views of clinicians in CHCs across Gauteng on sorting systems in the non-emergency ambulatory setting.

METHODS

The qualitative study design used one-to-one, in-depth interviews of purposively selected doctors. Interviews were conducted in English, with open-ended exploratory questions. Interviews were recorded, transcribed, anonymised and checked by interviewees later. Data collection and analysis stopped with information saturation. The co-author supervised and cross-checked the process. A thematic framework was developed by both authors, before final thematic coding of all transcripts was undertaken by the principal author. This analysis was based on the thematic framework approach.

RESULTS

Twelve primary health care (PHC) doctors with experience in patient sorting, from health districts across Gauteng, were interviewed. Two themes were identified, two major themes, namely Systems Implemented and Innovative Suggestions, and Factors Affecting Triage. Systems Implemented included those using vital signs, sorting by specialties, and using the Integrated Management of Childhood Illnesses approach. Systems Implemented also included doctor - nurse triage, first come first serve, eyeball triage and sorting based on main complaint. Innovative Suggestions, such as triage room treatment and investigations, telephone triage, longer clinic hours and a booking system emerged. There were three Factors Affecting Triage: Management Factor, including general management issues, equipment, documentation, infrastructure, protocol, and uniformity; and Staff Factor, including general staffing issues education and teamwork; and Patient Factor.

CONCLUSION

Developing a functional triage protocol with innovative systems for Gauteng is important. Findings from this study can guide the development of a functional triage system in the primary health care non-emergency outpatient setting of Gauteng's CHCs. The Emergency Triage, Assessment and Treatment (ETAT) tool, modified for adult and non-clinician use, could help this. However, addressing management, staff and patient factors must be integral.

Triage Performance in Emergency Medicine: A Systematic Review

STUDY OBJECTIVE

Rapid growth in emergency department (ED) triage literature has been accompanied by diversity in study design, methodology, and outcome assessment. We aim to synthesize existing ED triage literature by using a framework that enables performance comparisons and benchmarking across triage systems, with respect to clinical outcomes and reliability.

METHODS

PubMed, EMBASE, Scopus, and Web of Science were systematically searched for studies of adult ED triage systems through 2016. Studies evaluating triage systems with evidence of widespread adoption (Australian Triage Scale, Canadian Triage and Acuity Scale, Emergency Severity Index, Manchester Triage Scale, and South African Triage Scale) were cataloged and compared for performance in identifying patients at risk for mortality, critical illness and hospitalization, and interrater reliability. This study was performed and reported in adherence to Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines.

RESULTS

A total of 6,160 publications were identified, with 182 meeting eligibility criteria and 50 with sufficient data for inclusion in comparative analysis. The Canadian Triage and Acuity Scale (32 studies), Emergency Severity Index (43), and Manchester Triage Scale (38) were the most frequently studied triage scales, and all demonstrated similar performance. Most studies (6 of 8) reported high sensitivity (>90%) of triage scales for identifying patients with ED mortality as high acuity at triage. However, sensitivity was low (<80%) for identification of patients who had critical illness outcomes and those who died within days of the ED visit or during the index hospitalization. Sensitivity varied by critical illness and was lower for severe sepsis (36% to 74%), pulmonary embolism (54%), and non-ST-segment elevation myocardial infarction (44% to 85%) compared with ST-segment elevation myocardial infarction (56% to 92%) and general outcomes of ICU admission (58% to 100%) and lifesaving intervention (77% to 98%). Some proportion of hospitalized patients (3% to 45%) were triaged to low acuity (level 4 to 5) in all studies. Reliability measures (κ) were variable across evaluations, with only a minority (11 of 42) reporting κ above 0.8.

CONCLUSION

We found that a substantial proportion of ED patients who die postencounter or are critically ill are not designated as high acuity at triage. Opportunity to improve interrater reliability and triage performance in identifying patients at risk of adverse outcome exists.

Prehospital Application of the Canadian Triage and Acuity Scale by Emergency Medical Services

OBJECTIVES

Triage is fundamental to emergency patient assessment. Effective triage systems accurately prioritize patients and help predict resource utilization. CTAS is a validated five-level triage score utilized in Emergency Departments (EDs) across Canada and internationally. Historically CTAS has been applied by triage nurses in EDs. Observational evidence suggests that the CTAS might be implemented reliably by paramedics in the prehospital setting. This is the first system-wide assessment of CTAS interrater reliability between paramedics and triage nurses during clinical practice.

METHODS

Variables were extracted from hospital and EMS databases. EMS providers determined CTAS on-scene, CTAS pre-transport, and CTAS on-arrival at hospital for each patient (N=14,378). The hospital arrival EMS CTAS (CTAS arrival ) score was compared to the initial nursing CTAS score (CTAS initial ) and the final nursing CTAS score (CTAS final ) incuding nursing overrides. Interrater reliability between ED CTAS initial and EMS CTAS arrival scores was assessed. Interrater reliability between ED CTAS final and EMS CTAS arrival scores, as well as proportion of patient encounters with perfect or near-perfect agreement, were evaluated.

RESULTS

Our primary outcome, interrater reliability [kappa=0.437 (p<0.001, 95% CI 0.421-0.452)], indicated moderate agreement. EMS CTAS arrival and ED CTAS initial scores had an exact or within one point match 84.3% of the time. The secondary interrater reliability outcome between hospital arrival EMS CTAS (CTAS arrival ) score and the final ED triage CTAS score (CTAS final ) showed moderate agreement with kappa =0.452 (p<0.001, 95% CI 0.437-0.466).

CONCLUSIONS

Interrater reliability of CTAS scoring between triage nurses and paramedics was moderate in this system-wide implementation study.

Reliability of the Canadian Triage and Acuity Scale: interrater and intrarater agreement from a community and an academic emergency department

OBJECTIVES

The Canadian Triage and Acuity Scale (CTAS) is a five-level triage tool that is used to help prioritize the order in which emergency department (ED) patients should be seen. The objectives of this study were to determine the interrater and intrarater agreement of the 2008 CTAS guideline revisions by triage nurses and to compare agreement between triage nurses working in a small community ED and an academic ED.

METHODS

Seventy-eight triage nurses assigned CTAS scores and free-text presenting complaints for 10 paper-based case scenarios. For five scenarios, the CTAS score should have remained unchanged from previous guidelines, whereas the other five scenarios should have been triaged differently based on the 2008 CTAS first-order modifiers. Thirty-three participants repeated the questionnaire 90 days later, and intrarater agreement was measured.

RESULTS

There was a higher level of agreement (κ  =  0.73; 95% CI 0.68-0.79) for the five case scenarios, which relied on the older 2004 guidelines compared to the scenarios where the 2008 guidelines would have suggested a different triage level (κ  =  0.50; 95% CI 0.42-0.59). For the 10 case scenarios analyzed, the free-text presenting complaints matched the Canadian Emergency Department Information System (CEDIS) list 90.1% of the time (κ  =  0.80; 95% CI 0.76-0.84).

CONCLUSION

The reliability of CTAS scoring by academic and community ED nurses was relatively good; however, the application of the 2008 CTAS revisions appears less reliable than the 2004 CTAS guidelines. These results may be useful to develop educational materials to strengthen reliability and validity for triage scoring using the 2008 CTAS guideline revisions.

Canadian Triage and Acuity Scale: testing the mental health categories

PURPOSE

The study tested the inter-rater reliability and accuracy of triage nurses' assignment of urgency ratings for mental health patient scenarios based on the 2008 Canadian Triage and Acuity Scale (CTAS) guidelines, using a standardized triage tool. The influence of triage experience, educational preparation, and comfort level with mental health presentations on the accuracy of urgency ratings was also explored.

METHODS

Study participants assigned urgency ratings to 20 mental health patient scenarios in randomized order using the CTAS. The scenarios were developed using actual triage notes and were reviewed by an expert panel of emergency and mental health clinicians for face and content validity.

RESULTS

The overall Fleiss' kappa, the measure of inter-rater reliability for this sample of triage nurses (n=18), was 0.312, representing only fair albeit statistically significant (P<0.0001) agreement. Kendall's coefficient of concordance for the sample was calculated to be 0.680 (P<0.0001), which signifies moderate agreement. Although the sample reported high levels of education, comfort with mental health presentations, and experience, accuracy in urgency ratings measured by the percentage of correct responses ranged from 0.05% to 94% (mean: 54%). Greater accuracy in urgency ratings was recorded for triage nurses who used second-order modifiers and avoided the use of override.

CONCLUSION

Specific focus on the use of second-order modifiers in orientation and ongoing education of triage nurses may improve the reliability and validity of the CTAS when used to assign urgency ratings to mental health presentations.

The Reliability of the Canadian Triage and Acuity Scale: Meta-analysis

Background:

Although the Canadian Triage and Acuity Scale (CTAS) have been developed since two decades ago, the reliability of the CTAS has not been questioned comparing to moderating variable.

Aims:

The study was to provide a meta-analytic review of the reliability of the CTAS in order to reveal to what extent the CTAS is reliable.

Materials and Methods:

Electronic databases were searched to March 2014. Only studies were included that had reported samples size, reliability coefficients, adequate description of the CTAS reliability assessment. The guidelines for reporting reliability and agreement studies (GRRAS) were used. Two reviewers independently examined abstracts and extracted data. The effect size was obtained by the z-transformation of reliability coefficients. Data were pooled with random-effects models and meta-regression was done based on method of moments estimator.

Results:

Fourteen studies were included. Pooled coefficient for the CTAS was substantial 0.672 (CI 95%: 0.599-0.735). Mistriage is less than 50%. Agreement upon the adult version, among nurse-physician and near countries is higher than pediatrics version, other raters and farther countries, respectively.

Conclusion:

The CTAS showed acceptable level of overall reliability in the emergency department but need more development to reach almost perfect agreement.

Interrater agreement of Canadian Emergency Department Triage and Acuity Scale scores assigned by base hospital and emergency department nurses

Objective:

We sought to assess the applicability of the Canadian Emergency Department Triage and Acuity Scale (CTAS) in the prehospital setting by comparing CTAS scores assigned during ambulance transportation by base hospital (BH) nurses with CTAS scores given by emergency department (ED) nurses on patients' arrival.

Methods:

We recruited a prospective sample of consecutive patients who were transported to the ED by ambulance between December 2006 and March 2007 for whom a contact was made with the BH. Patients were triaged by the BH nurse with online communication and vital signs transmission. On arrival, patients were blindly triaged again by the ED nurse. We used the quadratic weighted κ statistic to measure the agreement between the 2 CTAS scores.

Results:

Ninety-four patients were triaged twice by 2 nursing teams (9 nurses at the BH and 39 nurses in the ED). The agreement obtained on prehospital and ED CTAS scores was moderate (κ = 0.50; 95% confidence interval 0.37–0.63).

Conclusion:

The moderate interrater agreement we obtained may be a result of the changing conditions of patients during transport or may indicate that CTAS scoring requires direct contact to produce reliable triage scores. Our study casts a serious doubt on the appropriateness of BH nurses performing triage with CTAS in the prehospital setting.

Techniques to aid the implementation of novel clinical information systems: a systematic review

BACKGROUND

This systematic review identifies and evaluates techniques that aid the implementation of novel clinical information systems (CIS) within healthcare.

METHODS

We searched electronic databases (MEDLINE, EMBASE, PsycINFO and HMIC Health Management Information Consortium). Desktop reviews for all potentially eligible studies were also conducted via reference lists and forward citation searches. 14,198 abstracts were identified through the initial electronic search. 63 articles were retained following title and abstract reviews, and submitted for full text evaluation. Of these, 18 papers met eligibility criteria.

RESULTS

The 5 techniques that emerged from the review and that can assist CIS implementation were: system piloting, eliciting acceptance, use of simulation, training and education, and provision of incentives. These techniques were evaluated with a range of study endpoints (including system utilisation, clinical effectiveness, user satisfaction, attitudes towards system training, and attitudes towards implementation). Consideration of the clinical context in which the CIS was implemented was a consistent theme in the evidence-base.

CONCLUSIONS

Although some evidence is available for the effectiveness of the 5 implementation techniques found in this review, the variable endpoints and the non-comparable study designs mean that the evidence-base needs further developing. We discuss the potential role of simulation and clinical leadership, particularly in relation to surgeons, in CIS implementation and we propose practical advice for CIS implementation and evaluation within hospital settings.

Reducing uncertainty in triaging mental health presentations: examining triage decision-making

Little is known about how emergency department (ED) nurses make decisions and even less is known about triage nurses' decision-making. There is compelling motivation to better understand the processes by which triage nurses make decisions, particularly with complex patient populations such as those with frequently emotive mental health and illness issues. While accuracy and reliability of triage decisions generally have been improved through the introduction of standardised triage scales and instruments, other factors such as lack of knowledge or confidence related to mental health issues, past experiences that may elicit transference and countertransference, judgments about individuals based on their behavioural presentations may impact on decisions made at triage. In this paper, we review the current research regarding the effectiveness of triage tools particularly with mental health presentations, present a theoretical framework that may guide research in understanding how triage nurses approach decision-making, and apply that framework to thinking about research in mental health-related triage. Developing a better understanding of how triage nurses make decisions, particularly in situations where issues related to mental health and illness may raise the levels of uncertainty, is crucial to ensure that they have the skills and tools they need to provide the most effective, sensitive, and compassionate care possible.

Atividades do enfermeiro nos sistemas de triagem/classificação de risco nos serviços de urgência: revisão integrativa

Objetivou-se identificar e avaliar as evidências disponíveis na literatura sobre as atividades do enfermeiro na classificação de risco nos serviços de urgência. Realizou-se uma revisão integrativa, com busca nas bases de dados Science Direct, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Medical Literature Analysis and Retrievel System Online (MEDLINE), Literatura Latino-Americana e do Caribe em Ciências da Saúde (LILACS) e Scientific Electronic Library Online (SCIELO). Foram selecionados 22 artigos que atenderam aos critérios de inclusão. Os resultados evidenciaram que as principais atribuições deste profissional são a avaliação do estado de saúde do usuário e a tomada de decisão, processo que necessita de conhecimento clínico e de tempo de experiência. O enfermeiro tem a capacidade de organizar o fluxo dos usuários conforme a prioridade do atendimento e a demanda dos serviços, sendo um profissional de excelência na execução da triagem/classificação de risco nos serviços de urgência.

A training programme did not increase agreement between allied health clinicians prioritizing patients for community rehabilitation

Objective: To evaluate the effect of formal training on agreement between clinicians making decisions on client priority.

Setting: A centralized intake service receiving referrals for a community rehabilitation programme.

Design: Agreement was measured between the priority categories allocated to consecutive referrals by one of five clinicians in the referral office compared with a second rating made by an independent occupational therapist, blinded to the initial priority rating. Data collection followed the implementation of four 1-hour workshops involving all raters, designed to increase consistency of triage decisions. Results were compared to a previous study conducted prior to the training.

Participants: Two hundred and one consecutive referrals received for community rehabilitation services, triaged by experienced clinicians with allied health or nursing qualifications.

Outcome measure: Agreement using weighted kappa (&kappa;w).

Results: There was no change in agreement between clinicians after training, compared with a previous study in the same setting. Agreement remained moderate (&kappa;w = 0.50), with clinicians disagreeing on approximately 30% of referrals.

Conclusions: Three out of 10 clients will receive a different priority rating and waiting time for rehabilitation services depending on which clinician in the referral office made the rating. This result was not improved by conducting a training programme.

Modern triage in the emergency department

BACKGROUND

Because the volume of patient admissions to an emergency department (ED) cannot be precisely planned, the available resources may become overwhelmed at times ("crowding"), with resulting risks for patient safety. The aim of this study is to identify modern triage instruments and assess their validity and reliability.

METHODS

Review of selected literature retrieved by a search on the terms "emergency department" and "triage."

RESULTS

Emergency departments around the world use different triage systems to assess the severity of incoming patients' conditions and assign treatment priorities. Our study identified four such instruments: the Australasian Triage Scale (ATS), the Canadian Triage and Acuity Scale (CTAS), the Manchester Triage System (MTS), and the Emergency Severity Index (ESI). Triage instruments with 5 levels are superior to those with 3 levels in both validity and reliability (p<0.01). Good to very good reliability has been shown for the best-studied instruments, CTAS and ESI (κ-statistics: 0.7 to 0.95), while ATS and MTS have been found to be only moderately reliable (κ-statistics: 0.3 to 0.6). MTS and ESI are both available in German; of these two, only the ESI has been validated in German-speaking countries.

CONCLUSION

Five-level triage systems are valid and reliable methods for assessment of the severity of incoming patients' conditions by nursing staff in the emergency department. They should be used in German emergency departments to assign treatment priorities in a structured and dependable fashion.

Interrater agreement between nurses for the Pediatric Canadian Triage and Acuity Scale in a tertiary care center

OBJECTIVES

The objective was to measure the interrater agreement between nurses assigning triage levels to children visiting a pediatric emergency departments (EDs) assisted by a computerized version of the Pediatric Canadian Triage and Acuity Scale (PedCTAS).

METHODS

This was a prospective cohort study evaluating children triaged from Level 2 (emergent) to Level 5 (nonurgent). A convenience sample of patients triaged during 38 shifts from April to September 2007 in a tertiary care pediatric ED was evaluated. All patients were initially triaged by regular triage nurses using a computerized version of the PedCTAS. Research nurses performed a second evaluation blinded to the first evaluation using the same triage tool. These research nurses were regular ED nurses performing extra hours for research purposes exclusively. The primary outcome measure was the interrater agreement between the two nurses as measured by the linear weighted kappa score. Secondary outcomes included the proportion of patient for which nurses did not apply the triage level suggested by Staturg (override) and agreement for these overrides.

RESULTS

A total of 499 patients were recruited. The overall interrater agreement was moderate (linear weighted kappa score of 0.55 [95% confidence interval {CI} = 0.48 to 0.61] and quadratic weighted kappa score of 0.61 [95% CI = 0.42 to 0.80]). There was a discrepancy of more than one level in only 10 patients (2% of the study population). Overrides occurred in 23.2 and 21.8% for regular and research triage nurses, respectively. These overrides were equally distributed between increase and decrease in triage level.

CONCLUSIONS

Nurses using Staturg, which is a computerized version of the PedCTAS, demonstrated moderate interrater agreement for assignment of triage level to children presenting to a pediatric ED.