Development of new methods to assess the outcomes of emergency care

The development of emergency medical services benefit score: a European Delphi study

Background

The helicopter emergency services (HEMS) Benefit Score (HBS) is a nine-level scoring system developed to evaluate the benefits of HEMS missions. The HBS has been in clinical use for two decades in its original form. Advances in prehospital care, however, have produced demand for a revision of the HBS. Therefore, we developed the emergency medical services (EMS) Benefit Score (EBS) based on the former HBS. As reflected by its name, the aim of the EBS is to measure the benefits produced by the whole EMS systems to patients.

Methods

This is a four-round, web-based, international Delphi consensus study with a consensus definition made by experts from seven countries. Participants reviewed items of the revised HBS on a 5-point Likert scale. A content validity index (CVI) was calculated, and agreement was defined as a 70% CVI. Study included experts from seven European countries. Of these, 18 were prehospital expert panellists and 11 were in-hospital commentary board members.

Results

The first Delphi round resulted in 1248 intervention examples divided into ten diagnostic categories. After removing overlapping examples, 413 interventions were included in the second Delphi round, which resulted in 38 examples divided into HBS categories 3–8. In the third Delphi round, these resulted in 37 prehospital interventions, examples of which were given revised version of the score. In the fourth and final Delphi round, the expert panel was given an opportunity to accept or comment on the revised scoring system.

Conclusions

The former HBS was revised by a Delphi methodology and EBS developed to represent its structural purpose better. The EBS includes 37 exemplar prehospital interventions to guide its clinical use.

Trial registration The study permission was requested and granted by Turku University Hospital (decision number TP2/010/18).

Supplementary Information

The online version contains supplementary material available at 10.1186/s13049-021-00966-3.

Self-Instruction versus Traditional Training on Cast Application Technique

Objective

To test the null hypothesis that for nurses working in emergency department, there is no difference between self-instruction and traditional method of training cast application technique, with respect to cost-effectiveness.

Method

This was a prospective, single-blind, randomised, controlled trial carried out at the Accident & Emergency Department of North District Hospital from 11 November 1998 to 16 May 2000. Thirty-three full-time emergency department nurses (9 nursing officers, 20 registered nurses and 4 enrolled nurses) were randomly assigned into two groups. Sixteen were given the self-instruction training program (intervention group) and 17 were instructed by traditional training program (control group). The primary outcome measure was cost-effectiveness analysis. The predetermined relative cost ratio was compared with the relative success ratio of the two methods. The secondary outcome measures were individual skill performance, knowledge scores and attitude ranks.

Results

There was no statistically significant difference between self-instruction and traditional method of training. The relative cost ratio of self-instruction to traditional method was 0.80. The relative success ratio of self-instruction to traditional method was 0.82. All nurses showed improvement in their knowledge (mean knowledge score, 38 versus 89). Trainees in the traditional method had more confidence performing cast application. Subgroup analysis showed that trainee characteristics and attitude did not predict the outcome.

Conclusion

Self-instruction method and traditional method may be equally cost-effective for training cast application technique. Both methods resulted in significant improvement in knowledge of cast application. Nurses trained with the traditional method had more confidence than those trained with self-instruction method.

Health-related quality of life in pediatric minor injury: reliability, validity, and responsiveness of the Pediatric Quality of Life Inventory in the emergency department

OBJECTIVE

To evaluate the feasibility, reliability, validity, and responsiveness of the Pediatric Quality of Life Inventory 4.0 Generic Core Scales (PedsQL) in the first 2 weeks after pediatric emergency department care of minor injury.

DESIGN

Prospective cohort study.

SETTING

Pediatric hospital emergency department.

PARTICIPANTS

Children and adolescents with minor injury (n = 334).

MAIN OUTCOME MEASURES

Child- and parent-reported clinical outcomes and PedsQL scale scores.

RESULTS

The PedsQL had good to excellent internal consistency reliability (α range, 0.73-0.93). For each day that the clinical symptoms persisted, there were consistent decreases in mean health-related quality of life (HRQOL) scores (validity testing). There were significantly greater negative changes in mean HRQOL scores for fractures vs soft-tissue injuries and for lower vs upper extremity injuries. Clinical outcomes categorized as poor had large negative changes in HRQOL not seen in good outcome groups. Distribution-based indicators of change supported good responsiveness (effect sizes for the physical summary score, 0.01-2.44; group differences at follow-up exceeded estimates of the minimal importance difference).

CONCLUSIONS

The PedsQL is feasible, reliable, and demonstrates good construct and discriminant validity and responsiveness in measuring short-term outcome after minor injury care in the pediatric emergency department. Assessing short-term outcome from the patient perspective with HRQOL measures may greatly enhance our ability to evaluate the effectiveness of emergency department care.

Outcomes considered most important by emergency physicians when determining disposition of patients with pulmonary embolism

Purpose

Clinical decision rules for the disposition of patients with pulmonary embolism (PE) are typically validated against an outcome of 30-day mortality or disease recurrence. There is little justification for this time frame, nor is it clear whether this outcome reflects emergency department (ED) decision making.

Aims

To determine which outcomes emergency physicians (EP) consider most relevant to disposition decisions.

Methods

Survey of attending EPs in geographically diverse US states using acute PE as the diagnostic framework. Responses required single-answer multiple choice, a numerical percentage, rank-ordered responses, or a five-point Likert scale. We distributed the survey via e-mail to 608 EPs.

Results

We received responses from 292 (48%) EPs: 88% board certified, 91% trained in emergency medicine, and 70% work in academics. Respondents reported discharging 1% of patients with PE from the ED, but 21% reported being asked to do so by an admitting service. EPs were more interested in knowing 5-day (in hospital) outcomes [192/265, 72% (95% exact CI = 66%–78%)] than 30-day outcomes [39/261, 15% (95% exact CI = 11%–20%)] or 90-day outcomes [29/263, 11% (95% exact CI = 8%–15%)]. On a Likert scale, 212/241 (88%, 95% exact CI = 83%–92%) agreed or strongly agreed that they considered 5-day (in hospital) clinical deterioration when making a decision to admit or discharge a patient from the ED compared to 184/242 (76%, 95% exact CI = 70%–81%) and 73/242 (30%, 95% exact CI = 24%–36%) for 30 and 90 days, respectively. A wide variety of clinical outcomes beyond death or recurrent PE were considered indicative of clinical deterioration.

Conclusions

Five-day (in hospital) outcomes that incorporate a variety of clinical deterioration events are of interest to EPs when determining the disposition of ED patients with PE. Researchers should consider this when developing and validating clinical decision rules.

Health-related quality of life for pediatric emergency department febrile illnesses: an evaluation of the Pediatric Quality of Life Inventory 4.0 Generic Core Scales

OBJECTIVE

We sought to assess the validity and short-term responsiveness of the Pediatric Quality of Life Inventory 4.0 Generic Core Scales (PedsQL) for febrile illnesses evaluated in the pediatric emergency department (ED).

DESIGN

Prospective cohort study of children 2-18 years discharged after ED evaluation for fever (>or= 38 degrees C). Self-administered, parent-report of health-related quality of life (HRQOL) was assessed using the PedsQL Acute Version, a validated HRQOL instrument. HRQOL was measured on ED presentation and at 7-10 day follow-up. At follow-up, duration of fever, child functional impairment, missed daycare/school, and disrupted family unit functioning, were assessed.

RESULTS

Of 160 subjects enrolled, 97 (61%) completed the study; mean follow-up was 8.7 days. Mean total HRQOL score on ED presentation was 76.4; mean follow-up score was 86.3. Compared to subjects that returned to baseline, statistically significant differences in HRQOL were noted for those with prolonged fever, child functional impairment, and relapse. Significant correlation was observed between HRQOL at follow-up and days of daycare/school missed (r = -0.35, p = .003) and days of family disruption (r = -0.43, p < .001). Mean change in HRQOL within subjects, from ED visit to follow-up, was +9.8 (95% CI: 5.6-14.6). Effect size was 0.53, indicating moderate responsiveness.

CONCLUSION

The PedsQL appears to be a valid and responsive indicator of HRQOL for short-term febrile illnesses evaluated in the ED.

The effect of emergency department crowding on patient satisfaction for admitted patients

OBJECTIVES

The objective was to study the association between factors related to emergency department (ED) crowding and patient satisfaction.

METHODS

The authors performed a retrospective cohort study of all patients admitted through the ED who completed Press-Ganey patient satisfaction surveys over a 2-year period at a single academic center. Ordinal and binary logistic regression was used to study the association between validated ED crowding factors (such as hallway placement, waiting times, and boarding times) and patient satisfaction with both ED care and assessment of satisfaction with the overall hospitalization.

RESULTS

A total of 1,501 hospitalizations for 1,469 patients were studied. ED hallway use was broadly predictive of a lower likelihood of recommending the ED to others, lower overall ED satisfaction, and lower overall satisfaction with the hospitalization (p < 0.05). Prolonged ED boarding times and prolonged treatment times were also predictive of lower ED satisfaction and lower satisfaction with the overall hospitalization (p < 0.05). Measures of ED crowding and ED waiting times predicted ED satisfaction (p < 0.05), but were not predictive of satisfaction with the overall hospitalization.

CONCLUSIONS

A poor ED service experience as measured by ED hallway use and prolonged boarding time after admission are adversely associated with ED satisfaction and predict lower satisfaction with the entire hospitalization. Efforts to decrease ED boarding and crowding might improve patient satisfaction.

Orofacial pain: patient satisfaction and delay of urgent care

OBJECTIVE

Accomplishing the Healthy People 2010 goal of eliminating disparities in oral disease will require a better understanding of the patterns of health care associated with orofacial pain. This study examined factors associated with pain-related acute oral health care.

METHODS

The authors used data on 698 participants in the Florida Dental Care Study, a study of oral health among dentate adults aged 45 years and older at baseline.

RESULTS

Fifteen percent of the respondents reported having had at least one dental visit as the result of orofacial pain. The majority of the respondents reportedly delayed contacting a dentist for at least one day; however, there was no difference between respondents reporting pain as the initiating symptom and those with other problems. Once respondents decided that dental services were needed, those with a painful symptom were nearly twice as likely as those without pain to want to be seen immediately. Rural adults were more likely than urban adults to report having received urgent dental care for a painful symptom. When orofacial pain occurred, those who identified as non-Hispanic African American were more likely than those who identified as non-Hispanic white to delay care rather than to seek treatment immediately, and women were more likely then men. Having a pain-related oral problem was associated with significantly less satisfaction with the services provided; non-Hispanic African American respondents were less likely than non-Hispanic white respondents to report being very satisfied, and rural residents were less likely than urban residents. Furthermore, men were more likely than women to suffer with orofacial pain without receiving either scheduled dental care or an urgent visit.

CONCLUSIONS

Barriers to care are complex and likely to be interactive, but must be understood before the goals of Healthy People 2010 can be accomplished.

Bench-to-bedside review: outcome predictions for critically ill patients in the emergency department

The escalating number of emergency department (ED) visits, length of stay, and hospital overcrowding have been associated with an increasing number of critically ill patients cared for in the ED. Existing physiologic scoring systems have traditionally been used for outcome prediction, clinical research, quality of care analysis, and benchmarking in the intensive care unit (ICU) environment. However, there is limited experience with scoring systems in the ED, while early and aggressive intervention in critically ill patients in the ED is becoming increasingly important. Development and implementation of physiologic scoring systems specific to this setting is potentially useful in the early recognition and prognostication of illness severity. A few existing ICU physiologic scoring systems have been applied in the ED, with some success. Other ED specific scoring systems have been developed for various applications: recognition of patients at risk for infection; prediction of mortality after critical care transport; prediction of in-hospital mortality after admission; assessment of prehospital therapeutic efficacy; screening for severe acute respiratory syndrome; and prediction of pediatric hospital admission. Further efforts at developing unique physiologic assessment methodologies for use in the ED will improve quality of patient care, aid in resource allocation, improve prognostic accuracy, and objectively measure the impact of early intervention in the ED.

Critical care and emergency medicine

The changing landscape of health care has resulted in an increase in the delivery of critical care in the emergency department. Although the emergency department duration is brief compared with the total length of hospitalization, physiologic determinants of outcome may be established before ICU admission. The care provided during the emergency department stay for critically ill patients has been shown to significantly impact the progression of organ failure and mortality. Despite these trends, formal critical care training for emergency physicians is limited. The need to maximize patient throughput in frequently overcrowded emergency departments hinders the provision of optimal care to the critically ill patient. Methodologies should be developed to examine the quality of patient care and objectively measure the impact of clinical interventions. The potential to improve outcome through educational initiatives and resource allocation should not be viewed as a burden of delivering a higher level of care in this setting but as a significant opportunity to effectively mitigate the socioeconomic consequences.

Outcomes research and emergency medical services for children: domains, challenges, and opportunities

Outcomes research focuses on the end results of medical care. This paper describes the specific domains of outcomes research, namely, clinical effectiveness, quality of care, quality of life, patient satisfaction, cost-effectiveness, and organization of care, which are the primary domains of outcomes research. Specific examples of pediatric emergency medicine applications are provided for each domain. Outcomes research in the clinical specialties of pediatrics, emergency medicine, and emergency medical services is in an early stage of development relative to that of adult medicine. Thus, the intersection of these clinical specialties, pediatric emergency medicine, presents many challenges and opportunities.

Triage nurses' clinical decision making. An observational study of urgency assessment

BACKGROUND

Researchers have described both the various decision tasks performed by triage nurses using self-report methods and identified time as a factor influencing the quality of triage decisions. However, little is known about the decision tasks performed by triage nurses when making acuity assessments, or the factors influencing triage duration in the real world.

AIMS

The aims of this study were to: describe the data triage nurses collect from patients in order to allocate a triage priority using the Australasian Triage Scale (ATS); describe the duration of nurses' decision making for ATS categories 2-5; and to explore the impact of patient and nurse variables on the duration of the triage nurses' decision making in the clinical setting.

DESIGN

A structured observational study was employed to address the research aims. Observational data was collected in one adult emergency department located in metropolitan Melbourne, Australia. A total of 26 triage nurses consented and were observed performing 404 occasions of triage. Data was collected by a single observer using a 20-item instrument that recorded the performance frequencies of a range of decision tasks and a number of observable patient, nurse and environmental variables. Additionally, the nurse-patient interaction was recorded as time in minutes.

RESULTS

It was found that there was limited use of objective physiological data collected by the nurses' in order to decide patient acuity, and large variability in the duration of triage decisions observed. In addition, analysis of variance indicated strong evidence of a true difference between triage duration and a range of nurse, patient and environmental variables.

CONCLUSION

These findings have implications for the development of practice standards and triage education. In particular, it is argued that practice standards should include routine measurement of physiological parameters in all but the collapsed or obviously unwell patient, where further delay may impede the delivery of time-critical intervention. Furthermore, the inclusion of arbitrary time frames for triage assessment in practice standards are not an appropriate method of evaluating triage decision making in the real world.

Critical care in the emergency department: A physiologic assessment and outcome evaluation

OBJECTIVES

The changing landscape of health care in this country has seen an increase in the delivery of care to critically ill patients in the emergency department (ED). However, methodologies to assess care and outcomes similar to those used in the intensive care unit (ICU) are currently lacking in this setting. This study examined the impact of ED intervention on morbidity and mortality using the Acute Physiology and Chronic Health Evaluation (APACHE II), the Simplified Acute Physiology Score (SAPS II), and the Multiple Organ Dysfunction Score (MODS).

METHODS

This was a prospective, observational cohort study over a three-month period. Critically ill adult patients presenting to a large urban ED and requiring ICU admission were enrolled. APACHE II, SAPS II, and MODS scores and predicted mortality were obtained at ED admission, ED discharge, and 24, 48, and 72 hours in the ICU. In-hospital mortality was recorded.

RESULTS

Eighty-one patients aged 64 +/- 18 years were enrolled during the study period, with a 30.9% in-hospital mortality. The ED length of stay was 5.9 +/- 2.7 hours and the hospital length of stay was 12.2 +/- 16.6 days. Nine (11.1%) patients initially accepted for ICU admission were later admitted to the general ward after ED intervention. Septic shock was the predominant admitting diagnosis. At ED admission, there was a significantly higher APACHE II score in nonsurvivors (23.0 +/- 6.0) vs survivors (19.8 +/- 6.5, p = 0.04), while there was no significant difference in SAPS II or MODS scores. The APACHE II, SAPS II, and MODS scores were significantly lower in survivors than nonsurvivors throughout the hospital stay (p </= 0.001). The hourly rates of change (decreases) in APACHE II, SAPS II, and MODS scores were significantly greater during the ED stay (-0.55 +/- 0.64, -1.02 +/- 1.10, and -0.16 +/- 0.43, respectively) than subsequent periods of hospitalization in survivors (p < 0.05). There was a significant decrease in APACHE II and SAPS II predicted mortality during the ED stay (-8.0 +/- 14.0% and -6.0 +/- 14.0%, respectively, p < 0.001) and equally at 24 hours in the ICU (-7.0 +/- 13.0% and -4.0 +/- 16.0%, respectively, p </= 0.02). The APACHE II and SAPS II predicted mortality approached actual in-hospital mortality at approximately 12 hours and 36 hours after ED admission (in the ICU), respectively.

CONCLUSIONS

The care provided during the ED stay for critically ill patients significantly impacts the progression of organ failure and mortality. Although this period is brief compared with the total length of hospitalization, physiologic determinants of outcome may be established before ICU admission. This study emphasizes the importance of ED intervention. It also suggests that unique physiologic assessment methodologies should be developed to examine the quality of patient care, improve the accuracy of prognostic decisions, and objectively measure the impact of clinical interventions and pathways in the ED setting.

A 5-year time study analysis of emergency department patient care efficiency

STUDY OBJECTIVES

We conducted a 5-year time study analysis of emergency department patient care efficiency. Our specific aims were (1) to calculate the main ED patient care time intervals to identify areas of inefficiency, (2) to measure the effect of ED and inpatient bed availability on patient flow, (3) to quantitatively assess the effects of administrative interventions aimed at improving efficiency, and (4) to evaluate the relationship between waiting times to see a physician and the number of patients who leave without being seen (LWBS) by a physician.

METHODS

Seven 1-week ED patient flow time studies were conducted from September 1993 to July 1998 using identical study design and methodology. Patients presenting with complaints of chest pain, abdominal pain, vaginal bleeding, and extremity injury were included to represent the level of severity of patient conditions seen in our Los Angeles County hospital ED. The calculated time intervals representing the main phases of evaluation and treatment were (1) triage presentation to completion of registration, (2) completion of registration to ED treatment area entry, (3) ED treatment area entry to initial medical assessment, (4) triage presentation to initial medical assessment, (5) initial medical assessment to disposition order, and (6) disposition order to patient discharge from the ED. Total ED lengths of stay (LOS) were also calculated as overall measures of efficiency. Time intervals were compared depending on the availability of ED and hospital inpatient beds. The effects of administrative interventions on the specific time intervals were assessed. The relationship between the median waiting time to see a physician and the number of LWBS patients was evaluated. Administrative interventions were implemented by a special interdepartmental continuous quality improvement committee. Interventions were aimed at specific sources of delay and inefficiency identified by the time studies.

RESULTS

Eight hundred twenty-six patients were included in the 7 time studies. The unavailability of ED and inpatient beds was associated with significant delays. There was a significant reduction of the median total ED LOS from 6.8 hours to 4.6 hours over the first 5 periods, presumably resulting from the administrative interventions. Median total ED LOS, however, increased from 4.6 hours to 6.0 hours during the last 2 periods, possibly as a result of an increase in our ED patient census and reductions in both nursing and physician staffing imposed by the recent Los Angeles County fiscal crisis. The number of LWBS patients was closely correlated to waiting time to see a physician ( r =0.79, beta=5.20, P =.033).

CONCLUSION

Time studies are an effective method of identifying areas of patient care delay. In our ED, targeted administrative interventions apparently reduced the total ED LOS and improved overall efficiency. Despite initial decreases in ED LOS, efficiency appeared to be adversely affected by reductions in nursing and physician staffing and increases in our patient census. The strength of the relationship between waiting times to see a physician and the number of LWBS patients suggests that decreasing waiting times may reduce the number of LWBS patients.