Targets and moving goal posts: changes in waiting times in a UK emergency department

Gaming New Zealand's Emergency Department Target: How and Why Did It Vary Over Time and Between Organisations?

BACKGROUND

Gaming is a potentially dysfunctional consequence of performance measurement and management systems in the health sector and more generally. In 2009, the New Zealand government initiated a Shorter Stays in Emergency Department (SSED) target in which 95% of patients would be admitted, discharged or transferred from an emergency department (ED) within 6 hours. The implementation of similar targets in England led to well-documented practices of gaming. Our research into ED target implementation sought to answer how and why gaming varies over time and between organisations.

METHODS

We developed a mixed-methods approach. Four organisation case study sites were selected. ED lengths of stay (ED LOS) were collected over a 6-year period (2007-2012) from all sites and indicators of target gaming were developed. Two rounds of surveys with managers and clinicians were conducted. Interviews (n=68) were conducted with clinicians and managers in EDs and the wider hospital in two phases across all sites. The interview data was used to develop explanations of the patterns of variation across time and across sites detected in the ED LOS data.

RESULTS

Our research established that gaming behaviour - in the form of 'clock-stopping' and decanting patients to short-stay units (SSUs) or observation beds to avoid target breaches - was common across all 4 case study sites. The opportunity to game was due to the absence of independent verification of ED LOS data. Gaming increased significantly over time (2009-2012) as the means to game became more available, usually through the addition or expansion of short-stay facilities attached to EDs. Gaming varied between sites, but those with the highest levels of gaming differed substantially in terms of organisational dynamics and motives. In each case, however, high levels of gaming could be attributed to the strategies of senior management more than to the individual motivations of frontline staff.

CONCLUSION

Gaming of New Zealand's ED target increased after the real benefits (in terms of process improvement) of the target were achieved. Gaming of ED targets could be minimised by eliminating opportunities to game through independent verification, or by monitoring and limiting the means and motivations to game.

Validity of the Japan Acuity and Triage Scale in adults: a cohort study

OBJECTIVE

The Japan Acuity and Triage Scale (JTAS) was developed based on Canadian Triage and Acuity Scale in 2012 and has been implemented in many Japanese EDs. We assessed the validity of JTAS by examining the association between JTAS triage levels and throughput and clinical outcomes in adult patients.

METHODS

We conducted a retrospective analysis of prospectively collected clinical data in the ED of a Japanese tertiary-care hospital. We included self-presenting patients who were ≥16 years of age and triaged between June 2013 and May 2014. We assessed the association between the triage level and overall admission and admission to the intensive care units (ICUs) with multivariable logistic regression analysis adjusted with patients' age and the time of visit and ED length of stay using the Kruskal-Wallis rank-sum test. We examined the predictive ability of JTAS for determining overall and ICU admission using receiver operating characteristic curves.

RESULTS

We included a total of 27 120 adult patients in our study. The OR for overall admission was greater with a higher triage level compared with the lowest urgency levels. ED length of stay was significantly longer with a higher JTAS level (p<0.001). The OR for ICU admission was greater in JTAS 1 (117.93 (95% CI 69.07 to 201.38)) and JTAS 2 (9.43 (95% CI 13.74 to 29.30)) compared with the lowest urgency levels. The areas under the curve for the predictive ability of JTAS for overall and ICU admission were 0.726 and 0.792, respectively.

CONCLUSION

Our study suggests an association of JTAS acuity with overall admission, ICU admission and ED length of stay, thereby demonstrating the predictive validity of JTAS.

Patients' intention to use online postings of ED wait times: A modified UTAUT model

BACKGROUND

As health care becomes more reliant on technology, a better understanding of the factors that contribute to acceptance and use of technology is now critical. The Unified Theory of Acceptance and Use of Technology (UTAUT) has been applied to study a variety of technologies in different settings, and it is one of the most cited theories in Information Systems (IS) research. However, there has been limited application of UTAUT to health IT and, in particular, to patients' IT use.

OBJECTIVES

The aim of this study is to adapt UTAUT to the context of patient acceptance and use of an Emergency Department (ED) wait-times website, and to empirically test the modified model and compare the results to those of the original UTAUT model. Specifically, it is proposed that there will be a significant relationship between facilitating conditions and behavioral intention.

METHODS

A survey of patients in the ED of a Canadian hospital was conducted, yielding 118 completed surveys, and subsequently analyzed using Partial least squares (PLS).

RESULTS

This study found that the modified UTAUT produced a substantial improvement in variance explained in behavioral intention compared to the original UTAUT (66% versus 46%). The modified-UTAUT model showed significant effects in performance expectancy (r = 0.302, p < 0.01) and facilitating conditions (r = 0.539, p < 0.001) on behavioral intention to use the website, while the effort expectancy impact was not significant.

CONCLUSIONS

This study provides empirical support for the modified-UTAUT in the context of patients' intention to use an ED wait times website. Some results of this study support prior research, while some differ, such as the non-significant relationship between effort expectancy and behavioral intention and the finding that performance expectancy is not the main driver of intention to use. As proposed, facilitating conditions - having the resources necessary to view the website and having the ability to find the website - were the most important factors influencing behavioral intention. UTAUT is a key theoretical advance in IS research and by modifying it to the context of patient use, we contribute to both IS and health research.

New Zealand's emergency department target - did it reduce ED length of stay, and if so, how and when?

Background

In 2009, the New Zealand government introduced a hospital emergency department (ED) target – 95% of patients seen, treated or discharged within 6 h - in order to alleviate crowding in public hospital EDs. While these targets were largely met by 2012, research suggests that such targets can be met without corresponding overall reductions in ED length-of-stay (LOS). Our research explores whether the NZ ED time target actually reduced ED LOS, and if so, how and when.

Methods

We adopted a mixed-methods approach with integration of data sources. After selecting four hospitals as case study sites, we collected all ED utilisation data for the period 2006 to 2012. ED LOS data was derived in two forms-reported ED LOS, and total ED LOS - which included time spent in short-stay units. This data was used to identify changes in the length of ED stay, and describe the timing of these changes to these indicators. Sixty-eight semi-structured interviews and two surveys of hospital clinicians and managers were conducted between 2011 and 2013. This data was then explored to identify factors that could account for ED LOS changes and their timing.

Results

Reported ED LOS reduced in all sites after the introduction of the target, and continued to reduce in 2011 and 2012. However, total ED LOS only decreased from 2008 to 2010, and did not reduce further in any hospital. Increased use of short-stay units largely accounted for these differences. Interview and survey data showed changes to improve patient flow were introduced in the early implementation period, whereas increased ED resources, better information systems to monitor target performance, and leadership and social marketing strategies mainly took throughout 2011 and 2012 when total ED LOS was not reducing.

Conclusions

While the ED target clearly stimulated improvements in patient flow, our analysis also questions the value of ED targets as a long term approach. Increased use of short-stay units suggests that the target became less effective in ‘standing for’ improved timeliness of hospital care in response to increasing acute demand. As such, the overall challenges in managing demand for acute and urgent care in New Zealand hospitals remain.

Electronic supplementary material

The online version of this article (10.1186/s12913-017-2617-1) contains supplementary material, which is available to authorized users.

The impact of adding clinical assistants on patient waiting time in a crowded emergency department

BACKGROUND

Emergency department (ED) crowding causes prolonged waiting times.

OBJECTIVE

To evaluate the potential benefit of introducing clinical assistants to a busy and crowded ED.

METHODS

This was a retrospective cohort study at an urban, academic tertiary medical centre. We introduced one clinical assistant to each ED shift. The main task of clinical assistants was managing the flow of incoming ED patients. The case group consisted of all adult non-trauma emergency patients during the case period from 1 September to 30 November 2008. The first control group consisted of all adult non-trauma emergency patients between 1 June and 31 August 2008 and the second control group consisted of all patients treated between 1 September and 30 November 2007. The primary outcome was the 'waiting time', defined as the time from triage to the time of the first medical order entered into the computer system. The secondary outcome was the number of adult non-trauma emergency patients who left the ED without being seen.

RESULTS

There were 12 257 cases and 25 950 controls. The mean and median waiting times were significantly shorter in the case group. The mean waiting time of the case group was 20.86 min, which was 4.51 min (17.8%) shorter than that of the first control group and 7.41 min (26.2%) shorter than that of the second control group. The median waiting time of the case group was also significantly shorter than those of the control groups. The number of the patients who left without being seen was significantly smaller in the case period.

CONCLUSIONS

In a busy and crowded ED, the introduction of clinical assistants to an existing emergency health service effectively reduces patient waiting times and decreases the number of patients leaving without being seen.

Emergency department crowding: time for interventions and policy evaluations

This paper summarises the consequences of emergency department crowding. It provides a comparison of the scales used to measure emergency department crowding. We discuss the multiple causes of crowding and present an up-to-date literature review of the interventions that reduce the adverse consequences of crowding. We consider interventions at the level of an individual hospital and a policy level.

Prioritizing performance measurement for emergency department care: consensus on evidence-based quality of care indicators

BACKGROUND

The evaluation of emergency department (ED) quality of care is hampered by the absence of consensus on appropriate measures. We sought to develop a consensus on a prioritized and parsimonious set of evidence-based quality of care indicators for EDs.

METHODS

The process was led by a nationally representative steering committee and expert panel (representatives from hospital administration, emergency medicine, health information, government, and provincial quality councils). A comprehensive review of the scientific literature was conducted to identify candidate indicators. The expert panel reviewed candidate indicators in a modified Delphi panel process using electronic surveys; final decisions on inclusion of indicators were made by the steering committee in a guided nominal group process with facilitated discussion. Indicators in the final set were ranked based on their priority for measurement. A gap analysis identified areas where future indicator development is needed. A feasibility study of measuring the final set of indicators using current Canadian administrative databases was conducted.

RESULTS

A total of 170 candidate indicators were generated from the literature; these were assessed based on scientific soundness and their relevance or importance. Using predefined scoring criteria in two rounds of surveys, indicators were coded as "retained" (53), "discarded" (78), or "borderline" (39). A final set of 48 retained indicators was selected and grouped in nine categories (patient satisfaction, ED operations, patient safety, pain management, pediatrics, cardiac conditions, respiratory conditions, stroke, and sepsis or infection). Gap analysis suggested the need for new indicators in patient satisfaction, a healthy workplace, mental health and addiction, elder care, and community-hospital integration. Feasibility analysis found that 13 of 48 indicators (27%) can be measured using existing national administrative databases.

DISCUSSION

A broadly representative modified Delphi panel process resulted in a consensus on a set of 48 evidence-based quality of care indicators for EDs. Future work is required to generate technical definitions to enable the uptake of these indicators to support benchmarking, quality improvement, and accountability efforts.

Keynote address: United Kingdom experiences of evaluating performance and quality in emergency medicine

Demand for emergency care is rising throughout the western world and represents a major public health problem. Increased reliance on professionalized health care by the public means that strategies need to be developed to manage the demand safely and in a way that is achievable and acceptable to both consumers of emergency care, but also to service providers. In the United Kingdom, strategies have previously been aimed at managing demand better and included introducing new emergency services for patients to access, extending the skills within the existing workforce, and more recently, introducing time targets for emergency departments (EDs). This article will review the effect of these strategies on demand for care and discuss the successes and failures with reference to future plans for tackling this increasingly difficult problem in health care.

Association between waiting times and short term mortality and hospital admission after departure from emergency department: population based cohort study from Ontario, Canada

OBJECTIVE

To determine whether patients who are not admitted to hospital after attending an emergency department during shifts with long waiting times are at risk for adverse events.

DESIGN

Population based retrospective cohort study using health administrative databases. Setting High volume emergency departments in Ontario, Canada, fiscal years 2003-7.

PARTICIPANTS

All emergency department patients who were not admitted (seen and discharged; left without being seen).

OUTCOME MEASURES

Risk of adverse events (admission to hospital or death within seven days) adjusted for important characteristics of patients, shift, and hospital.

RESULTS

13,934,542 patients were seen and discharged and 617,011 left without being seen. The risk of adverse events increased with the mean length of stay of similar patients in the same shift in the emergency department. For mean length of stay ≥ 6 v <1 hour the adjusted odds ratio (95% confidence interval) was 1.79 (1.24 to 2.59) for death and 1.95 (1.79 to 2.13) for admission in high acuity patients and 1.71 (1.25 to 2.35) for death and 1.66 (1.56 to 1.76) for admission in low acuity patients). Leaving without being seen was not associated with an increase in adverse events at the level of the patient or by annual rates of the hospital.

CONCLUSIONS

Presenting to an emergency department during shifts with longer waiting times, reflected in longer mean length of stay, is associated with a greater risk in the short term of death and admission to hospital in patients who are well enough to leave the department. Patients who leave without being seen are not at higher risk of short term adverse events.

Consensus methods to identify a set of potential performance indicators for systems of emergency and urgent care

OBJECTIVES

To identify a comprehensive set of indicators to enable Primary Care Trust (PCT) commissioners in England and other NHS decision-makers to monitor the performance of systems of emergency and urgent care for which they are responsible.

METHODS

Using a combination of Delphi RAND methods in three successive rounds of consultation and nominal group review, we canvassed expert opinion on 70 potential indicators as good measures of system performance. The two Delphi panels consisted of senior clinicians and researchers, and urgent care leads and commissioners in PCTs and Strategic Health Authorities (SHAs). The indicators were formatted into a questionnaire according to whether they were outcome, process, structure, or equity-based measures. Participants scored each indicator on a Likert scale of 1-9 and had the opportunity to consider their scores informed by the group scores and feedback. The questionnaire was refined after each round. To ensure that the indicators rated most highly by the Delphi panels covered all dimensions of performance, the results of the Delphi were reviewed by a nominal group consisting of two researchers and three clinicians from the local health services research network (LHSR).

RESULTS

Overall, the process yielded 16 candidate indicators. It also produced a core set of serious, emergency and urgent care-sensitive conditions (defined as conditions whose exacerbations should be managed by a well-performing system without admission to an inpatient bed), for use with the indicators.

CONCLUSIONS

System-wide measures to monitor performance across multiple services should encourage providers to work for patient benefit in an integrated way. They will also assist commissioners to monitor and improve emergency and urgent care for their local populations. The indicators are now being calculated using routinely available data, and tested for their responsiveness to capture change over time.