CORD-AEUS: consensus document for the emergency ultrasound milestone project

Ultrasound Integration in Undergraduate Medical Education: Comparison of Ultrasound Proficiency Between Trained and Untrained Medical Students

OBJECTIVES

The benefit of formal ultrasound implementation in undergraduate medical education remains unclear. The goal of this study was to evaluate the effectiveness of ultrasound curriculum implementation during year 1 medical student physical examination teaching on the ultrasound proficiency of medical students.

METHODS

An ultrasound curriculum was formally implemented at our institution in August 2012 and since then has successfully trained 2 classes of medical students (year 1 and year 2). Year 3 and year 4 medical students did not receive any formal ultrasound training, as the curriculum had not yet been implemented during their preclinical years. With the use of a 22-point ultrasound objective structured clinical examination (US-OSCE), trained medical students were compared to untrained medical students. The US-OSCE tested image acquisition and interpretation of the following systems: ocular, neck, vascular, pulmonary, cardiovascular, and abdominal. Emergency medicine (EM) residents formally trained in bedside ultrasound were also tested with the US-OSCE to provide a reference standard.

RESULTS

There were 174 year 1, 25 year 2, and 19 year 3/year 4 medical students and 30 EM residents tested on the US-OSCE. Ultrasound-trained medical students were compared to untrained medical students, and overall US-OSCE scores ± SD were 91.4% ± 14.0% versus 36.1% ± 21.4% (P < .001), respectively. The EM resident group had an overall score of 99.1% ± 1.8%. The absolute percentage differences were 7.7% between EM resident and ultrasound-trained medical student scores and 63.0% between EM resident and untrained medical student scores (P < .001).

CONCLUSIONS

Implementation of an ultrasound curriculum in undergraduate medical education showed a significant increase in trained versus untrained medical student ultrasound capabilities.

Dedicated time for deliberate practice: one emergency medicine program’s approach to point-of-care ultrasound (PoCUS) training

Point-of-care ultrasound (PoCUS) has become an essential skill in the practice of emergency medicine (EM). Various EM residency programs now require competency in basic PoCUS applications. The education literature suggests that deliberate practice is necessary for skill acquisition and mastery. We used an educational theory, Ericsson’s model of deliberate practice, to create a PoCUS curriculum for our Royal College of Physicians and Surgeons of Canada EM residency.

Although international recommendations around curriculum requirements exist, this will be one of the first papers to describe the implementation of a specific PoCUS training program. This paper details the features of the program and lessons learned during its initial 3 years. Sharing this experience may serve as a nidus for scholarly discussion around how to best approach medical education in this area.

Learning curves in emergency ultrasound education

OBJECTIVES

Proficiency in the use of bedside ultrasound (US) has become standard in emergency medicine residency training. While milestones have been established for this training, supporting data for minimum standard experience are lacking. The objective of this study was to characterize US learning curves to identify performance plateaus for both image acquisition and interpretation, as well as compare performance characteristics of learners to those of expert sonographers.

METHODS

A retrospective review of an US database was conducted at a single academic institution. Each examination was scored for agreement between the learner and expert reviewer interpretation and given a score for image quality. A locally weighted scatterplot smoothing method was used to generate a model of predicted performance for each individual examination type. Performance characteristics for expert sonographers at the site were also tracked and used in addition to performance plateaus as benchmarks for learning curve analysis.

RESULTS

There were 52,408 US examinations performed between May 2007 and January 2013 and included for analysis. Performance plateaus occurred at different points for different US protocols, from 18 examinations for soft tissue image quality to 90 examinations for right upper quadrant image interpretation. For the majority of examination types, a range of 50 to 75 examinations resulted in both excellent interpretation (sensitivity > 84% and specificity > 90%) and good image quality (90% the image quality benchmark of expert sonographers).

CONCLUSIONS

Educational performance benchmarks occur at variable points for image interpretation and image quality for different examination types. These data should be considered when developing training standards for US education as well as experience requirements for US credentialing.

Point-of-care ultrasonography by pediatric emergency medicine physicians

Emergency physicians have used point-of-care ultrasonography since the 1990 s. Pediatric emergency medicine physicians have more recently adopted this technology. Point-of-care ultrasonography is used for various scenarios, particularly the evaluation of soft tissue infections or blunt abdominal trauma and procedural guidance. To date, there are no published statements from national organizations specifically for pediatric emergency physicians describing the incorporation of point-of-care ultrasonography into their practice. This document outlines how pediatric emergency departments may establish a formal point-of-care ultrasonography program. This task includes appointing leaders with expertise in point-of-care ultrasonography, effectively training and credentialing physicians in the department, and providing ongoing quality assurance reviews.

Assessment of the numbers-based model for evaluation of resident competency in emergency ultrasound core applications

Background: The objective of this study was to assess the current numbers-based model of evaluation of emergency medicine residents in emergency ultrasound, specifically by focusing on the proficiency of residents to interpret right upper quadrant, focused abdominal sonogram for trauma, abdominal aorta, first trimester pregnancy, and cardiac scans. Methods: A prospective analysis of emergency medicine residents' competency in emergency ultrasound was conducted in 2009 and 2010. Residents completed a multiple choice style examination primarily assessing residents' diagnostic accuracy. The accuracy and confidence of diagnosis for each resident based on the examinations were compared to his/her training level and number of scans per application completed prior to testing, in order to assess the relationship of experience to competence. Results: Thirty-six emergency medicine residents, postgraduate year 1–3 (n = 15, 8, 13), were enrolled in the study and took the exam. There was a statistically significant relationship between the number of scans and mean exam score for right upper quadrant (p = 0.0013) and abdominal aorta (p = 0.0013) scans. Residents demonstrated a statistically significant improvement in scores at greater than 16 scans for right upper quadrant (p = 0.0018) and 13 scans for abdominal aorta (p = 0.0018). Conclusion: Adequate resident training for interpretation for certain applications such as right upper quadrant and abdominal aorta, may be achievable using smaller numbers than the generally accepted guideline of 25 scans per application. Additionally, the number of scans needed to reach competency may be variable dependent on the specific ultrasound application.

A review of lawsuits related to point-of-care emergency ultrasound applications

INTRODUCTION

New medical technology brings the potential of lawsuits related to the usage of that new technology. In recent years the use of point-of-care (POC) ultrasound has increased rapidly in the emergency department (ED). POC ultrasound creates potential legal risk to an emergency physician (EP) either using or not using this tool. The aim of this study was to quantify and characterize reported decisions in lawsuits related to EPs performing POC ultrasound.

METHODS

We conducted a retrospective review of all United States reported state and federal cases in the Westlaw database. We assessed the full text of reported cases between January 2008 and December 2012. EPs with emergency ultrasound fellowship training reviewed the full text of each case. Cases were included if an EP was named, the patient encounter was in the emergency department, the interpretation or failure to perform an ultrasound was a central issue and the application was within the American College of Emergency Physician (ACEP) ultrasound core applications. In order to assess deferred risk, cases that involved ultrasound examinations that could have been performed by an EP but were deferred to radiology were included.

RESULTS

We identified five cases. All reported decisions alleged a failure to perform an ultrasound study or a failure to perform it in a timely manner. All studies were within the scope of emergency medicine and were ACEP emergency ultrasound core applications. A majority of cases (n=4) resulted in a patient death. There were no reported cases of failure to interpret or misdiagnoses.

CONCLUSION

In a five-year period from January 2008 through December 2012, five malpractice cases involving EPs and ultrasound examinations that are ACEP core emergency ultrasound applications were documented in the Westlaw database. All cases were related to failure to perform an ultrasound study or failure to perform a study in a timely manner and none involved failure to interpret or misdiagnosis when using of POC ultrasound.

Emergency ultrasound fellowship training: a novel team-based approach

OBJECTIVES

To describe our experience with implementation of a novel team-based emergency ultrasound (EUS) fellowship training program.

METHODS

We conducted a retrospective review at an academic medical center. All emergency department (ED) faculty and residents were divided into 3 teams, and a fellow was assigned to each team. Each fellow was responsible for ultrasound (US) education of their team members and faculty credentialing. Additionally, each fellow was assigned to one of these specific responsibilities every month: (1) education, (2) quality assurance/billing, and (3) equipment. The fellows also received a checklist at the beginning of the fellowship training, which outlined research, reading, teaching, and scanning requirements.

RESULTS

With a team-based approach, all US examinations were reviewed for quality control within 1 week after the scans were performed. The number of US examinations billed by faculty increased by 180%. The US billing revenue increased by 60% during the study period. Fifteen additional faculty members were credentialed to use US in the ED. Compliance with bedside documentation of US findings increased from 30% to 90%. There was increased resident engagement, and the number of scans performed by residents increased by 130%. There was also a substantial decrease in the funds required to replace damaged transducers.

CONCLUSIONS

A team-based EUS fellowship training program had a substantial impact on US use in the ED, quality control review, faculty credentialing, US billing revenue, compliance with documentation, and resident US education.

SonoGames: an innovative approach to emergency medicine resident ultrasound education

SonoGames was created by the Academy of Emergency Ultrasound for the 2012 annual meeting of the Society for Academic Emergency Medicine. The assessment of resident knowledge and of the performance of point-of-care ultrasound examinations is an integral component of ultrasound education and is required in emergency medicine residency training. With that in mind, game organizers sought to assess and improve emergency medicine residents' point-of-care ultrasound knowledge, hands-on skills, and integration of knowledge into clinical decision making. SonoGames is an annual 4-hour competition consisting of 3 rounds. In this article, we provide a description of SonoGames and provide a blueprint for an effective and successful educational event.

The Milestones Passport: A Learner-Centered Application of the Milestone Framework to Prompt Real-Time Feedback in the Emergency Department

BACKGROUND

In July 2013, emergency medicine residency programs implemented the Milestone assessment as part of the Next Accreditation System.

OBJECTIVE

We hypothesized that applying the Milestone framework to real-time feedback in the emergency department (ED) could affect current feedback processes and culture. We describe the development and implementation of a Milestone-based, learner-centered intervention designed to prompt real-time feedback in the ED.

METHODS

We developed and implemented the Milestones Passport, a feedback intervention incorporating subcompetencies, in our residency program in July 2013. Our primary outcomes were feasibility, including faculty and staff time and costs, number of documented feedback encounters in the first 2 months of implementation, and user-reported time required to complete the intervention. We also assessed learner and faculty acceptability.

RESULTS

Development and implementation of the Milestones Passport required 10 hours of program coordinator time, 120 hours of software developer time, and 20 hours of faculty time. Twenty-eight residents and 34 faculty members generated 257 Milestones Passport feedback encounters. Most residents and faculty reported that the encounters required fewer than 5 minutes to complete, and 48% (12 of 25) of the residents and 68% (19 of 28) of faculty reported satisfaction with the Milestones Passport intervention. Faculty satisfaction with overall feedback in the ED improved after the intervention (93% versus 54%, P  =  .003), whereas resident satisfaction with feedback did not change significantly.

CONCLUSIONS

The Milestones Passport feedback intervention was feasible and acceptable to users; however, learner satisfaction with the Milestone assessment in the ED was modest.

Ultrasound competency assessment in emergency medicine residency programs

OBJECTIVES

In the Model of the Clinical Practice of Emergency Medicine (EM), bedside ultrasound (US) is listed as one of the essential procedural skills. EM milestones released by Accreditation Council for Graduate Medical Education and American Board of Emergency Medicine require residents to demonstrate competency in bedside US. The purpose of this study was to assess the current methods used by EM residency training programs to evaluate resident competency in bedside US.

METHODS

This was a cross-sectional survey study. A questionnaire on US education and competency assessment was electronically sent to all EM residency program directors and emergency US directors. The survey consisted of questions regarding the US rotation, structure of US curriculum, presence of US fellowship, image archiving, quality assurance methods, feedback, competency assessment tools, and frequency of assessment. The survey responses are reported as the percentages of total respondents along with 95% confidence intervals (CIs).

RESULTS

A total of 124 of 161 EM residency programs participated in this study, representing a 77% response rate. Twenty-six percent (95% CI = 18% to 34%) of programs assess competency only at the end of the US rotation. Eight percent (95% CI = 3% to 13%) assess competency only every 6 months, and 13% (95% CI = 7% to 19%) assess competency only annually. Eight percent (95% CI = 3% to 13%) assess competency only during the final year of training. Thirty percent (95% CI = 22% to 38%) of programs assess competency with a combination of the above intervals, and 16% (95% CI = 10% to 22%) do not assess US competency. Fourteen percent (95% CI = 8% to 20%) use objective structured clinical examinations (OSCEs), and 21% (95% CI = 14% to 28%) use standardized direct observation tools (SDOTs) to assess resident competency in US. Approximately one-third (33%, 95% CI = 24% to 41%) of standardized testing for US competency is conducted with multiple-choice questions. Thirty percent (95% CI = 21% to 38%) administer practical examinations to assess US skills.

CONCLUSIONS

Currently, a majority of EM residency programs assess resident competency in bedside US. However, there is significant variation in the methods of competency assessment.

The core content of clinical ultrasonography fellowship training

The purpose of developing a core content for subspecialty training in clinical ultrasonography (US) is to standardize the education and qualifications required to provide oversight of US training, clinical use, and administration to improve patient care. This core content would be mastered by a fellow as a separate and unique postgraduate training, beyond that obtained during an emergency medicine (EM) residency or during medical school. The core content defines the training parameters, resources, and knowledge of clinical US necessary to direct clinical US divisions within medical specialties. Additionally, it is intended to inform fellowship directors and candidates for certification of the full range of content that might appear in future examinations. This article describes the development of the core content and presents the core content in its entirety.

Assessment of a human cadaver model for training emergency medicine residents in the ultrasound diagnosis of pneumothorax

Objectives. To assess a human cadaver model for training emergency medicine residents in the ultrasound diagnosis of pneumothorax. Methods. Single-blinded observational study using a human cadaveric model at an academic medical center. Three lightly embalmed cadavers were used to create three “normal lungs” and three lungs modeling a “pneumothorax.” The residents were blinded to the side and number of pneumothoraces, as well as to each other's findings. Each resident performed an ultrasound examination on all six lung models during ventilation of cadavers. They were evaluated on their ability to identify the presence or absence of the sliding-lung sign and seashore sign. Results. A total of 84 ultrasound examinations (42-“normal lung,” 42-“pneumothorax”) were performed. A sliding-lung sign was accurately identified in 39 scans, and the seashore sign was accurately identified in 34 scans. The sensitivity and specificity for the sliding-lung sign were 93% (95% CI, 85–100%) and 90% (95% CI, 81–99%), respectively. The sensitivity and specificity for the seashore sign were 80% (95% CI, 68–92%) and 83% (95% CI, 72–94%), respectively. Conclusions. Lightly embalmed human cadavers may provide an excellent model for mimicking the sonographic appearance of pneumothorax.

Personalized peer-comparison feedback and its effect on emergency medicine resident ultrasound scan numbers

Background

Clinician-performed ultrasound has become a widely utilized tool in emergency medicine and is a mandatory component of the residency curricula. We aimed to assess the effect of personalized peer-comparison feedback on the number of ultrasound scans performed by emergency medicine residents.

Findings

A personalized peer-comparison feedback was performed by sending 44 emergency medicine residents a document including personally identified scan numbers and class averages. The number of ultrasound scans per clinical shift for a 3-month period before and after the feedback intervention was calculated. The average number of ultrasound exams per shift improved from 0.39 scans/shift before to 0.61 scans/shift after feedback (p = 0.04). Among the second year residents, the scans/shift ratio improved from 0.35 to 0.87 (p = 0.07); for third year residents, from 0.51 to 0.58 (p = 0.46); and from 0.33 to 0.41 (p = 0.21) for the fourth year residents before and after the intervention, respectively.

Conclusions

A personalized peer-comparison feedback provided to emergency medicine residents resulted in increased ultrasound scan numbers per clinical shift. Incorporating this method of feedback may help encourage residents to scan more frequently.

Ability of emergency physicians with advanced echocardiographic experience at a single center to identify complex echocardiographic abnormalities

OBJECTIVES

To determine the ability of emergency physicians to detect complex abnormalities on point-of-care (POC) echocardiograms.

METHODS

Single-blinded, nonrandomized, cross-sectional study. Twenty-five different emergency medicine clinical scenarios (video clips and digital images) covering a variety of echocardiographic abnormalities were presented to a group of emergency physician sonologists. The echocardiographic abnormalities included right ventricular dysfunction, left ventricular systolic dysfunction, diastolic dysfunction, regional wall motion abnormalities, Doppler abnormalities of pericardial tamponade physiology, left ventricular hypertrophy, hypertrophic cardiomyopathy, and aortic abnormalities. All emergency physician sonologists were blinded to the study hypothesis. They reviewed echocardiography video clips and images individually, and their interpretations were compared with the criterion standard (expert echocardiographer interpretations).

RESULTS

A total of 200 echocardiography studies (video clips and images) were independently reviewed by 8 emergency physician sonologists with varying POC echocardiography experiences. Emergency physicians accurately identified left ventricular systolic dysfunction 94% of the time, diastolic dysfunction (100%), and right ventricular dysfunction 80% of the time. Regional wall motion abnormalities were detected only 50% of the time. Doppler echocardiographic abnormalities of pericardial tamponade physiology were accurately identified 57% of the time. Emergency physicians who performed more than 250 POC echocardiograms were found to be more accurate in identifying complex echocardiographic abnormalities.

CONCLUSIONS

Our study results suggest that with increased experience, emergency physicians can accurately identify most of complex echocardiographic abnormalities.