Simulation in health care education

During the past 15 years there has been widespread adoption of simulation in health care education as a method to train and assess learners. Multiple factors have contributed to this movement, including reduced patient availability, limited faculty teaching time, technological advances in diagnosis and treatment that require a new skills set, greater attention to patient safety with the need to reduce medical errors, and a focus on outcomes-based education. In this discussion, simulation refers broadly to any device or set of conditions that attempts to present the patient authentically. While simulation offers many advantages over traditional methods of teaching, there are several challenges that must be addressed to ensure its effective use. This article presents the range of available simulation technologies, explores the challenges that health care educators face when using this method, provides an example of a successful program that has integrated simulation into the culture of learning at its institution, and discusses an approach to maximizing the effectiveness of simulation as a means to learning and practicing skills in a safe, interactive environment.

Assessing the relationship between cardiac physical examination technique and accurate bedside diagnosis during an objective structured clinical examination (OSCE)

BACKGROUND

Many standardized patient (SP) encounters employ SPs without physical findings and, thus, assess physical examination technique. The relationship between technique, accurate bedside diagnosis, and global competence in physical examination remains unclear.

METHOD

Twenty-eight internists undertook a cardiac physical examination objective structured clinical examination, using three modalities: real cardiac patients (RP), "normal" SPs combined with related cardiac audio-video simulations, and a cardiology patient simulator (CPS). Two examiners assessed physical examination technique and global bedside competence. Accuracy of cardiac diagnosis was scored separately.

RESULTS

The correlation coefficients between participants' physical examination technique and diagnostic accuracy were 0.39 for RP (P < .05), 0.29 for SP, and 0.30 for CPS. Patient modality impacted the relative weighting of technique and diagnostic accuracy in the determination of global competence.

CONCLUSIONS

Assessments of physical examination competence should evaluate both technique and diagnostic accuracy. Patient modality affects the relative contributions of each outcome towards a global rating.

Does physical examination competence correlate with bedside diagnostic acumen? An observational study

AIM

To examine the relationship between a physician's ability to examine a standardized patient (SP) and their ability to correctly identify related clinical findings created with simulation technology.

METHOD

The authors conducted an observational study of 347 candidates during a Canadian national specialty examination at the end of post-graduate internal medicine training. Stations were created that combined physical examination of an SP with evaluation of a related audio-video simulation of a patient abnormality, in the domains of cardiology and neurology. Examiners evaluated a candidate's competence at performing a physical examination of an SP and their accuracy in diagnosing a related audio-video simulation.

RESULTS

For the cardiology stations, the correlation between the physical examination scores and recognition of simulation abnormalities was 0.31 (p < 0.01). For the neurology stations, the correlation was 0.27 (p < 0.01). Addition of the simulations identified 18% of 197 passing candidates on the cardiology stations and 17% of 240 passing candidates on the neurology stations who were competent in their physical examination technique but did not achieve the passing score for diagnostic skills.

CONCLUSIONS

Assessments incorporating SPs without physical findings may need to include other methodologies to assess bedside diagnostic acumen.

Skill improvement during emergency response to terrorism training

OBJECTIVES

To assess the individual and team skills acquired from an interactive training program to prepare emergency personnel to respond to terrorist acts.

METHODS

We developed a 16-hour, two-day, multimedia- and simulation-enhanced course that places learners in realistic situations using the equipment required to respond to various chemical, biologic, radiologic, and explosive acts of terrorism. Small-group sessions and drills were conducted. Errors in skill performance were corrected immediately, and then skills were repeated to achieve mastery. Participants included emergency medical technicians, paramedics, nurses, and physicians. Team performance was assessed over four successive scenarios using a 100-mm visual analog scale. Individual learner skill acquisition was assessed with precourse and postcourse evaluation of selected skills in a randomized sampling of consenting learners.

RESULTS

Nearly all teams achieved mastery of the required skills by the second assessment rotation. Individual learners demonstrated significant gains in the ability to emergently don personal protective equipment and administer a nerve agent antidote kit.

CONCLUSIONS

An interactive, simulation-enhanced curriculum of terrorism response training for emergency responders can produce significant, quantifiable individual and team skill gain. Future studies should further address performance benchmarks for these newly acquired skills.

Learning objects in medical education

A learning object (LO) is a grouping of instructional materials structured to meet a specified educational objective. Digital LOs, which can be stored electronically, allow a new approach to instructional activity, making medical education more efficient, and potentially more cost-effective. They are reusable and can incorporate text, graphics, animations, audio, and video to support and enhance learning. A learning object can stand alone or be aggregated with additional objects to create larger forms of educational content meeting multiple educational objectives. Digital learning objects located in online repositories can be accessed by many computers and are easily handled by an array of learning management systems for delivery to learners at any time. Integrating digital learning objects with traditional educational methods in a blended learning approach assists medical educators in meeting the challenges of competing priorities. Multimedia LOs enable learners to tailor their experience to their preferred learning style. Through the use of learning objects, learners' reactions, their acquisition of knowledge, skills and attitudes, and their behavioral changes become readily measurable. Learning objects provide multiple research opportunities, such as their use in adaptive learning, their added value in preclinical versus clinical education, and their impact as part of a blended learning strategy.

Development, implementation and outcomes of a training program for responders to acts of terrorism

INTRODUCTION

Responding to acts of terrorism requires the effective use of public-safety and medical-response resources. The knowledge, skills and attitudes necessary to respond to future threats is unfamiliar to most emergency responders.

OBJECTIVES

The purpose of this report to describe the development, implementation and evaluation of a multidisciplinary, interactive and simulation-enhanced course to prepare responders to acts of terrorism.

METHODS

We used a 5-step systematic process to develop a blended-learning, simulation-enhanced training program. Learners completed a self-confidence questionnaire and written examination prior to the course and a self-confidence questionnaire, written examination and course evaluation when they finished the course.

RESULTS

From July 7, 2003 to March 8, 2005, 497 consenting learners completed the course. After course completion, learners demonstrated significant increases in their knowledge of terrorism response (t = -64.3, df = 496, p < 0.05) and their confidence in responding to terrorist events (t = -45.5, df = 496, p < 0.05). Learner feedback about the course was highly positive.

CONCLUSIONS

We successfully implemented a two-day course for professionals likely to respond to terrorist acts that included scenario-based performance training and assessment. Course participants increased their knowledge and were more confident in their ability to respond to acts of terrorism after participating in this course.

Effect of practice on standardised learning outcomes in simulation-based medical education

OBJECTIVES

This report synthesises a subset of 31 journal articles on high-fidelity simulation-based medical education containing 32 research studies drawn from a larger qualitative review published previously. These studies were selected because they present adequate data to allow for quantitative synthesis. We hypothesised an association between hours of practice in simulation-based medical education and standardised learning outcomes measured as weighted effect sizes.

METHODS

Journal articles were screened using 5 exclusion and inclusion criteria. Response data were extracted and 3 judges independently coded each study. Learning outcomes were standardised using a common metric, the average weighted effect size (AWES), due to the heterogeneity of response measures in individual studies. anova was used to evaluate AWES differences due to hours of practice on a high-fidelity medical simulator cast in 5 categories. The eta squared (eta2) statistic was used to assess the association between AWES and simulator practice hours.

RESULTS

There is a strong association (eta2=0.46) between hours of practice on high-fidelity medical simulators and standardised learning outcomes. The association approximates a dose-response relationship.

CONCLUSIONS

Hours of high-fidelity simulator practice have a positive, functional relationship with standardised learning outcomes in medical education. More rigorous research methods and more stringent journal editorial policies are needed to advance this field of medical education research.

A multicenter study to provide evidence of construct validity in a computer-based outcome measure of neurology clinical skills

BACKGROUND

Using computer-based simulation to assess clinical skill-a key competence for medical trainees-enables standardization and exposure to a broad sample of physical findings. The purpose of this study is to provide evidence of construct validity for a computer-based outcome measure of neurology clinical skills.

METHOD

A total of 128 medical students and neurology residents at four institutions volunteered to take a 34-question computer-based test designed to measure neurology clinical skills. Subjects were classified into three groups based on level of training: novice, intermediate, and experienced.

RESULTS

Overall performance increased with level of training. Question difficulty discriminated between groups as predicted. Twenty-six of 34 individual items discriminated between novices and more advanced learners. This test separated learners at different levels of training with a consistency of .92.

CONCLUSION

This study provided evidence of construct validity for a computer-based outcome measure of neurology clinical skills.

Incorporating simulation technology in a canadian internal medicine specialty examination: a descriptive report

High-stakes assessment of clinical performance through the use of standardized patients (SPs) is limited by the SP's lack of real physical abnormalities. The authors report on the development and implementation of physical examination stations that combine simulation technology in the form of digitized cardiac auscultation videos with an SP assessment for the 2003 Royal College of Physicians and Surgeons of Canada's Comprehensive Objective Examination in Internal Medicine. The authors assessed candidates on both the traditional stations and the stations that combined the traditional SP examination with the digitized cardiac auscultation video. For the combined stations, candidates first completed a physical examination of the SP, watched and listened to a computer simulation, and then described their auscultatory findings. The candidates' mean scores for both types of stations were similar, as were the mean discrimination indices for both types of stations, suggesting that the combined stations were of a testing standard similar to the traditional stations. Combining an SP with simulation technology may be one approach to the assessment of clinical competence in high-stakes testing situations.

Stroke training of prehospital providers: an example of simulation-enhanced blended learning and evaluation

Since appropriate treatment of patients in the first few hours of ischemic stroke may decrease the risk of long-term disability, prehospital providers should recognize, assess, manage and communicate about stroke patients in an effective and time-efficient manner. This requires the instruction and evaluation of a wide range of competencies including clinical skills, patient investigation and management and communication skills. The authors developed and assessed the effectiveness of a simulation-enhanced stroke course that incorporates several different learning strategies to evaluate competencies in the care of acute stroke patients. The one-day, interactive, emergency stroke course features a simulation-enhanced, blended-learning approach that includes didactic lectures, tabletop exercises, and focused-examination training and small-group sessions led by paramedic instructors as standardized patients portraying five key neurological syndromes. From January to October 2000, 345 learners were assessed using multiple-choice tests as were randomly selected group of 73 learners using skills' checklists during two pre- and two post-course simulated patient encounters. Among all learners there was a significant gain in knowledge (pre: 53.9%+/-13.9 and post: 85.4%+/-8.5; p<0.001), and for the 73 learners a significant improvement in their clinical and communication skills (p<0.0001 for all). By using a simulation-enhanced, blended-learning approach, pre-hospital paraprofessionals were successfully trained and evaluated in a wide range of competences that will lead to the more improved recognition and management of acute stroke patients.

Effective use of simulations for the teaching and acquisition of veterinary professional and clinical skills

Simulation technology will feature prominently in this exciting, yet challenging, time for veterinary medicine. The profession is evolving to keep pace with rapid changes in clinical practice, scientific discovery, and educational strategy, while ensuring that it follows the public mandate to produce competent veterinarians. Among the challenges to meeting this educational goal are limitations-due to important issues such as animal welfare-on the availability of real patients for training. Drawing chiefly on the experience in human medicine, this article explores the use of simulations in veterinary medical education to provide safe and ethical alternative opportunities for learners to practice essential clinical and professional skills.

Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review

REVIEW DATE

1969 to 2003, 34 years.

BACKGROUND AND CONTEXT

Simulations are now in widespread use in medical education and medical personnel evaluation. Outcomes research on the use and effectiveness of simulation technology in medical education is scattered, inconsistent and varies widely in methodological rigor and substantive focus.

OBJECTIVES

Review and synthesize existing evidence in educational science that addresses the question, 'What are the features and uses of high-fidelity medical simulations that lead to most effective learning?'.

SEARCH STRATEGY

The search covered five literature databases (ERIC, MEDLINE, PsycINFO, Web of Science and Timelit) and employed 91 single search terms and concepts and their Boolean combinations. Hand searching, Internet searches and attention to the 'grey literature' were also used. The aim was to perform the most thorough literature search possible of peer-reviewed publications and reports in the unpublished literature that have been judged for academic quality.

INCLUSION AND EXCLUSION CRITERIA

Four screening criteria were used to reduce the initial pool of 670 journal articles to a focused set of 109 studies: (a) elimination of review articles in favor of empirical studies; (b) use of a simulator as an educational assessment or intervention with learner outcomes measured quantitatively; (c) comparative research, either experimental or quasi-experimental; and (d) research that involves simulation as an educational intervention.

DATA EXTRACTION

Data were extracted systematically from the 109 eligible journal articles by independent coders. Each coder used a standardized data extraction protocol.

DATA SYNTHESIS

Qualitative data synthesis and tabular presentation of research methods and outcomes were used. Heterogeneity of research designs, educational interventions, outcome measures and timeframe precluded data synthesis using meta-analysis.

HEADLINE RESULTS

Coding accuracy for features of the journal articles is high. The extant quality of the published research is generally weak. The weight of the best available evidence suggests that high-fidelity medical simulations facilitate learning under the right conditions. These include the following: providing feedback--51 (47%) journal articles reported that educational feedback is the most important feature of simulation-based medical education; repetitive practice--43 (39%) journal articles identified repetitive practice as a key feature involving the use of high-fidelity simulations in medical education; curriculum integration--27 (25%) journal articles cited integration of simulation-based exercises into the standard medical school or postgraduate educational curriculum as an essential feature of their effective use; range of difficulty level--15 (14%) journal articles address the importance of the range of task difficulty level as an important variable in simulation-based medical education; multiple learning strategies--11 (10%) journal articles identified the adaptability of high-fidelity simulations to multiple learning strategies as an important factor in their educational effectiveness; capture clinical variation--11 (10%) journal articles cited simulators that capture a wide variety of clinical conditions as more useful than those with a narrow range; controlled environment--10 (9%) journal articles emphasized the importance of using high-fidelity simulations in a controlled environment where learners can make, detect and correct errors without adverse consequences; individualized learning--10 (9%) journal articles highlighted the importance of having reproducible, standardized educational experiences where learners are active participants, not passive bystanders; defined outcomes--seven (6%) journal articles cited the importance of having clearly stated goals with tangible outcome measures that will more likely lead to learners mastering skills; simulator validity--four (3%) journal articles provided evidence for the direct correlation of simulation validity with effective learning.

CONCLUSIONS

While research in this field needs improvement in terms of rigor and quality, high-fidelity medical simulations are educationally effective and simulation-based education complements medical education in patient care settings.

Bedside cardiology skills training for the osteopathic internist using simulation technology

Medical practice changes that limit patient availability, instructor time, and advances in technology have led to a greater use of simulators and multimedia computers in medical education. These systems address the problem of inadequate bedside skills training and poor proficiency among all health care providers. While studies have shown their effectiveness among medical students, residents, and practicing physicians, none has focused on the osteopathic internist population--one that is becoming more responsible for conducting initial and follow-up physical examinations. This report describes the use of "Harvey," the cardiology patient simulator, and the UMedic Multimedia Computer System at a workshop conducted at the American College of Osteopathic Internists' 61st Annual Convention and Scientific Sessions. Participants in this study significantly improved their ability to identify common cardiac auscultatory events, as indicated by pretest-to-posttest scores. Workshop participants were nearly unanimous in their belief that they would like to use these tools for learning and assessment.

Adoption and integration of simulation-based learning technologies into the curriculum of a UK Undergraduate Education Programme

CONTEXT

At a time of significant changes in medical education world-wide, the international dimensions and global issues relating to the application of new learning technologies have been recognised.

OBJECTIVE

The aim of this paper is to describe the adoption and integration within the curriculum in one United Kingdom (UK) medical school of 'Harvey', the Cardiology Patient Simulator, and the UMedic multimedia computer-based cardiology curriculum - resources developed in a medical school in the USA.

PARTICIPANTS

The integration of the resources into the curriculum is described by 3 teachers actively involved in the cardiology curriculum of the UK medical school and 3 teachers associated with the development of resources in the USA. ASPECTS CONSIDERED: The review considers the adoption of Harvey and UMedic in the UK in programmes in relation to: curricular issues, training needs, learning outcomes, curriculum content and sequences of content, educational strategies, teaching and learning methods, assessment, communication about the curriculum and management of the curriculum.

CONCLUSIONS

Learning resources, in the form of simulators and computer-based learning modules, developed in one country can be successfully adopted and implemented in another. Facets that facilitated the adoption included close liaison between the developers of the resources in the USA and the implementers in the UK, and careful and systematic planning including in-depth integration of the simulation-based resources into the required curriculum rather than their relegation to a peripheral ad hoc position. The successful use of simulators such as Harvey requires the presence of a 'champion', a clinician educator and a supporting administrative staff who ensure the simulator's appropriate use.

Development of a reliable multimedia, computer-based measure of clinical skills in bedside neurology

PURPOSE

This study reports the development of reliable multimedia, computer-based measures of bedside neurology skills.

METHOD

A consortium of neurologists and medical educators (1) identified bedside skills and (2) created a computer-based test. Test-item stems were multimedia clips of standardized patients. Options appeared as text. Sixty-one junior medical students responded to these items.

RESULTS

The 77-item test yielded a reliability coefficient of 0.85. Subgroup item analysis resulted in reliability coefficients of 0.73 for the 20-question "pretest" (Test A) and 0.75 for the 20-question "posttest" (Test B).

CONCLUSION

Two sets of test items were developed that can be used as outcome measures in studies that assess the effectiveness of educational interventions in bedside neurology.

Effectiveness of a cardiology review course for internal medicine residents using simulation technology and deliberate practice

BACKGROUND

Objective evaluations of residents' clinical skills reveal serious deficits.

PURPOSE

To develop, implement, and evaluate outcomes from a review course in cardiology bedside skills for internal medicine residents.

METHODS

We used a 1-group pretest-posttest design with historical comparisons. The study was conducted at the University of Miami School of Medicine as part of the internal medicine residency program from July 1999 to June 2000. A total of 67 2nd- and 3rd-year medicine residents received an educational intervention involving deliberate practice using simulation technology. A total of 155 4th-year medical students in one intervention and one comparison group (n = 53) served as historical comparisons. Outcome measures were a reliable computer-delivered pretest and posttest that evaluate cardiology bedside skills.

RESULTS

Residents who received the review course and medical students who received a comparable educational intervention showed large and statistically significant pretest-to-posttest improvement in bedside skills. These 2 groups are also significantly and substantially different at posttest from a comparison group of 4th-year medical students that did not receive a specific educational intervention.

CONCLUSION

Educational interventions using simulation technology that engage learners in deliberate practice of clinical skills produce large improvements in a relatively short time, with little faculty involvement.

Teamwork. University of Miami uses competition to sharpen EMS team performance

Many argue that experience is the best teacher. However, it's often dangerous for the patient and impractical for an EMS system to assess prehospital providers in their actual working environment. Simulated scenario competition fosters clearer thinking and translates into more effective action and enhanced patient outcomes during true emergencies.

Simulation and new learning technologies

Changes in medical practice that limit patient availability and instructors' time have resulted in poor physical diagnosis skills by learners at all levels. Advanced simulation technology, including the use of sophisticated multimedia computer systems, helps to address this problem. For many years 'Harvey', the Cardiology Patient Simulator, and the UMedic Multimedia Computer system have proven to be effective tools to teach and assess bedside cardiovascular skills when they are integrated into the required curriculum of medical school and postgraduate training. In the future, virtual reality technology, based initially on data from the Visible Human Data set, will provide the majority of simulation-based training. Models that provide a high level of visual fidelity and use sophisticated haptic devices that simulate the 'touch' and 'feel' of a procedure or examination are now being used in selected medical centers. The presence of these tools is not enough. Evidence-based outcomes must show these systems to be effective instruments for teaching and assessment, and medical educators must be willing to effect change in medical education to ensure the appropriate use of these systems in the next millennium.

Improvement in Paramedic Examination Skills Following a Stroke Course

P184

Introduction: The need to decrease time to treatment for acute stroke patients means that prehospital providers must play an increased role in their care. This is possible only if emergency medical services personnel are able to rapidly recognize and assess patients with neurologic syndromes. Purpose: To assess the performance of a focused neurologic exam by prehospital providers before and after participation in an interactive stroke course. Methods: We developed a 1-day emergency stroke course that consists of 2 hours of lectures and 6 hours of interactive instruction, including small-group sessions led by paramedic instructors as standardized patients (SPs) portraying 5 key neurologic syndromes: left hemisphere, right hemisphere, brainstem, cerebellum, and subarachnoid hemorrhage. We devised a 53-point skills checklist to evaluate paramedic performance of history, exam, management, and emergency department (ED) reporting during 2 pre- and 2 postcourse encounters with actors portraying one of 4 scenarios: left hemisphere stroke, right hemisphere stroke, right hemisphere seizure with postictal hemiparesis, and left hemisphere tumor with sudden worsening. Among the 53 total skills evaluated were 28 exam-related items, including traditional paramedic exam items such as pupil reaction, hand grasp, and foot strength and additional items from the Miami Emergency Neurologic Deficit (MEND) Exam. We randomly selected 46 of 281 learners to participate in the study. Results: Checklist scores for neurologic exam performance improved significantly. The precourse mean score for the 46 learners was 3.38 (12.1%) and the postcourse mean was 21.4 (76.4%) (p<.001). Conclusions: Paramedics significantly improved their performance of a focused neurologic exam after attending a stroke course utilizing paramedic instructors as SPs with key stroke syndromes. We conclude that prehospital providers can learn and perform a brief, focused neurologic exam after attending a 1-day stroke course that emphasizes hands-on instruction. We are continuing to evaluate the effect of the course on the history-taking, ED-reporting, and stroke-management skills of practicing paramedics.

Simulation technology for health care professional skills training and assessment

Changes in medical practice that limit instruction time and patient availability, the expanding options for diagnosis and management, and advances in technology are contributing to greater use of simulation technology in medical education. Four areas of high-technology simulations currently being used are laparoscopic techniques, which provide surgeons with an opportunity to enhance their motor skills without risk to patients; a cardiovascular disease simulator, which can be used to simulate cardiac conditions; multimedia computer systems, which includes patient-centered, case-based programs that constitute a generalist curriculum in cardiology; and anesthesia simulators, which have controlled responses that vary according to numerous possible scenarios. Some benefits of simulation technology include improvements in certain surgical technical skills, in cardiovascular examination skills, and in acquisition and retention of knowledge compared with traditional lectures. These systems help to address the problem of poor skills training and proficiency and may provide a method for physicians to become self-directed lifelong learners.

Implementation of a four-year multimedia computer curriculum in cardiology at six medical schools

The pressures of a changing health care system are making inroads on the commitment and effort that both basic science and clinical faculty can give to medical education. A tool that has the potential to compensate for decreased faculty time and thereby to improve medical education is multimedia computer instruction that is applicable at all levels of medical education, developed according to instructional design principles, and supported by evidence of effectiveness. The authors describe the experiences of six medical schools in implementing a comprehensive computer-based four-year curriculum in bedside cardiology developed by a consortium of university cardiologists and educational professionals. The curriculum consisted of ten interactive, patient-centered, case-based modules focused on the history, physical examination, laboratory data, diagnosis, and treatment. While an optimal implementation plan was recommended, each institution determined its own strategy. Major goals of the project, which took place from July 1996 to June 1997, were to identify and solve problems of implementation and to assess learners' and instructors' acceptance of the system and their views of its value. A total of 1,586 students used individual modules of the curriculum 6,131 times. Over 80% of students rated all aspects of the system highly, especially its clarity and educational value compared with traditional lectures. The authors discuss the aspects of the curriculum that worked, problems that occurred (such as difficulties in scheduling use of the modules in the third year), barriers to change and ways to overcome them (such as the type of team needed to win acceptance for and oversee implementation of this type of curriculum), and the need in succeeding years to formally assess the educational effectiveness of this and similar kinds of computer-based curricula.