Simulation technology for health care professional skills training and assessment

Changes in medical practice that limit instruction time and patient availability, the expanding options for diagnosis and management, and advances in technology are contributing to greater use of simulation technology in medical education. Four areas of high-technology simulations currently being used are laparoscopic techniques, which provide surgeons with an opportunity to enhance their motor skills without risk to patients; a cardiovascular disease simulator, which can be used to simulate cardiac conditions; multimedia computer systems, which includes patient-centered, case-based programs that constitute a generalist curriculum in cardiology; and anesthesia simulators, which have controlled responses that vary according to numerous possible scenarios. Some benefits of simulation technology include improvements in certain surgical technical skills, in cardiovascular examination skills, and in acquisition and retention of knowledge compared with traditional lectures. These systems help to address the problem of poor skills training and proficiency and may provide a method for physicians to become self-directed lifelong learners.

Practice Standards for Electrocardiographic Monitoring in Hospital Settings

The goals of electrocardiographic (ECG) monitoring in hospital settings have expanded from simple heart rate and basic rhythm determination to the diagnosis of complex arrhythmias, myocardial ischemia, and prolonged QT interval. Whereas computerized arrhythmia analysis is automatic in cardiac monitoring systems, computerized ST-segment ischemia analysis is available only in newer-generation monitors, and computerized QT-interval monitoring is currently unavailable. Even in hospitals with ST-monitoring capability, ischemia monitoring is vastly underutilized by healthcare professionals. Moreover, because no computerized analysis is available for QT monitoring, healthcare professionals must determine when it is appropriate to manually measure QT intervals (eg, when a patient is started on a potentially proarrhythmic drug). The purpose of the present review is to provide ‘best practices’ for hospital ECG monitoring. Randomized clinical trials in this area are almost nonexistent; therefore, expert opinions are based upon clinical experience and related research in the field of electrocardiography. This consensus document encompasses all areas of hospital cardiac monitoring in both children and adults. The emphasis is on information clinicians need to know to monitor patients safely and effectively. Recommendations are made with regard to indications, timeframes, and strategies to improve the diagnostic accuracy of cardiac arrhythmia, ischemia, and QT-interval monitoring. Currently available ECG lead systems are described, and recommendations related to staffing, training, and methods to improve quality are provided.

Resuscitation Education Science: Educational Strategies to Improve Outcomes From Cardiac Arrest: A Scientific Statement From the American Heart Association

The formula for survival in resuscitation describes educational efficiency and local implementation as key determinants in survival after cardiac arrest. Current educational offerings in the form of standardized online and face-to-face courses are falling short, with providers demonstrating a decay of skills over time. This translates to suboptimal clinical care and poor survival outcomes from cardiac arrest. In many institutions, guidelines taught in courses are not thoughtfully implemented in the clinical environment. A current synthesis of the evidence supporting best educational and knowledge translation strategies in resuscitation is lacking. In this American Heart Association scientific statement, we provide a review of the literature describing key elements of educational efficiency and local implementation, including mastery learning and deliberate practice, spaced practice, contextual learning, feedback and debriefing, assessment, innovative educational strategies, faculty development, and knowledge translation and implementation. For each topic, we provide suggestions for improving provider performance that may ultimately optimize patient outcomes from cardiac arrest.

Retention of CPR skills learned in a traditional AHA Heartsaver course versus 30-min video self-training: A controlled randomized study

BACKGROUND

Bystander CPR improves outcomes after out of hospital cardiac arrest. The length of current 4-h classes in cardiopulmonary resuscitation (CPR) is a barrier to more widespread dissemination of CPR training and older adults in particular are underrepresented in traditional classes. Training with a brief video self-instruction (VSI) program has shown that this type of training can produce short-term skill performance at least as good as that seen with traditional American Heart Association (AHA) Heartsaver training, although it is unclear whether there is comparable skill retention.

METHODS AND RESULTS

Two hundred and eight-five adults between the ages of 40 and 70 who had no CPR training within the past 5 years were assigned at random to a no-training control group, Heartsaver (HS) training, or one of three versions of brief VSI (i.e., self-trained-ST subjects). Post-training performance of CPR skills was assessed in a scenario format by human examiners and by sensored manikin at Time 1 (immediately post-training) and again at Time 2 (2 months post-training). Performance by controls was assessed only once. Significant (P<.001) decline was observed in the three measures recorded by examiners; assess responsiveness (from 72% to 60% for HS subjects and from 90% to 77% for ST subjects), call 911 (from 82% to 74% for HS subjects and from 71% to 53% for ST subjects), and overall performance (from 42% to 30% for HS subjects and from 60% to 44% for ST subjects). Significant (P<.001) decline was observed in two of three skills measured by a sensored manikin: ventilation volume (from 40% to 36% for HS subjects and from 61% to 41% for ST subjects, with a significant [P=.028] interaction) and correct hand placement (from 68% to 59% for HS subjects and from 80% to 64% for ST subjects). Heartsaver and self-trained subjects generally showed similar rates of decline. At Time 2, examiners rated trained subjects better than untrained controls in all skills except calling 911, where self-trained subjects did not differ from controls; manikin data revealed that trained subjects' performance was better than that of controls for ventilation volume, but had declined to the level of controls for both hand placement and compression depth.

CONCLUSIONS

Adults between 40 and 70 years of age who participated in a CPR VSI program experienced performance decline in their CPR skills after a post-training interval of 2 months. However, this decline was no greater than that seen in subjects who took Heartsaver training. The VSI program produced retention performance at least as good as that seen with traditional training. Additional effort is needed to improve both initial performance and retention of CPR skills.

CONDENSED ABSTRACT

Retention of CPR skills was compared 2 months post-training for adults between 40 and 70 years old who had taken either a traditional Heartsaver CPR course or a 22-min video self-directed training course. Although performance declines occurred in the 2-month interval, self-trained subjects generally demonstrated CPR skill retention equivalent to that of Heartsaver-trained subjects, although for both groups skill decline on some measures reached the level of untrained controls.

Competency in cardiac examination skills in medical students, trainees, physicians, and faculty: a multicenter study

BACKGROUND

Cardiac examination is an essential aspect of the physical examination. Previous studies have shown poor diagnostic accuracy, but most used audio recordings, precluding correlation with visible observations. The training spectrum from medical students (MSs) to faculty has not been tested, to our knowledge.

METHODS

A validated 50-question, computer-based test was used to assess 4 aspects of cardiac examination competency: (1) cardiac physiology knowledge, (2) auditory skills, (3) visual skills, and (4) integration of auditory and visual skills using computer graphic animations and virtual patient examinations (actual patients filmed at the bedside). We tested 860 participants: 318 MSs, 289 residents (225 internal medicine and 64 family medicine), 85 cardiology fellows, 131 physicians (50 full-time faculty, 12 volunteer clinical faculty, and 69 private practitioners), and 37 others.

RESULTS

Mean scores improved from MS1-2 to MS3-4 (P = .003) but did not improve or differ significantly among MS3, MS4, internal medicine residents, family medicine residents, full-time faculty, volunteer clinical faculty, and private practitioners. Only cardiology fellows tested significantly better (P<.001), and they were the best in all 4 subcategories of competency, whereas MS1-2 were the worst in the auditory and visual subcategories. Participants demonstrated low specificity for systolic murmurs (0.35) and low sensitivity for diastolic murmurs (0.49).

CONCLUSIONS

Cardiac examination skills do not improve after MS3 and may decline after years in practice, which has important implications for medical decision making, patient safety, cost-effective care, and continuing medical education. Improvement in cardiac examination competency will require training in simultaneous audio and visual examination in faculty and trainees.

Analysis of coronary angioplasty practice in the United States with an insurance-claims data base

BACKGROUND

Coronary angioplasty is frequently performed in the United States, with more than 300,000 procedures in 1990. Despite the high rate of use of the procedure, there have been few studies addressing practice patterns.

METHODS AND RESULTS

From a private insurance claims data base of 5.4 million individuals, a total of 2,101 patients who underwent coronary angioplasty during 1988-1989 were identified. Using their 4,578 hospital admission records and 87,578 outpatient claim records, with an average follow-up of 332 +/- 182 days, we compared patients' outcomes and charges according to whether they had an exercise stress test before the procedure, by sex, by region of the country, and by whether the angioplasty was performed in an institution with a training program. Only 29% of the study cohort had exercise testing before angioplasty; patients in the West (p = 0.001), those undergoing multivessel angioplasty (p = 0.00001), and those whose procedures were performed at sites with training programs (p = 0.04) were more likely to have a screening test, whereas women (p = 0.008) and those with a recent myocardial infarction (p = 0.00001) were less likely to have a screening test. The average length of stay for patients without myocardial infarction as a primary diagnosis was 5.6 days, with a total hospital charge of $15,027. In follow-up, 15.1% had coronary artery bypass surgery and 15% had at least one additional angioplasty procedure; the average follow-up charges were $4,879. Charges varied according to sex, region of the country, and academic status of the angioplasty institution. Certain outcomes showed variation by region of the country and academic status of the angioplasty institution.

CONCLUSIONS

The relative lack of an objective definition of myocardial ischemia and the marked variability of use of procedures according to geographic region suggest the need for further implementation of established guidelines.

The teaching and practice of cardiac auscultation during internal medicine and cardiology training. A nationwide survey

OBJECTIVES

To assess the time and importance given to cardiac auscultation during internal medicine and cardiology training and to evaluate the auscultatory proficiency of medical students and physicians-in-training.

STUDY DESIGN

A nationwide survey of internal medicine and cardiology program directors and a multicenter cross-sectional assessment of students' and housestaff's auscultatory proficiency.

SETTING

All accredited U.S. internal medicine and cardiology programs and nine university-affiliated internal medicine and cardiology programs.

PARTICIPANTS

Four hundred ninety-eight (75.6%) of all 659 directors surveyed; 203 physicians-in-training and 49 third-year medical students.

INTERVENTIONS

Directors completed a 23-item questionnaire, and students and trainees were tested on 12 prerecorded cardiac events.

MAIN OUTCOME MEASURES

The teaching and proficiency of cardiac auscultation at all levels of training.

RESULTS

Directors attributed great importance to cardiac auscultation and thought that more time should be spent teaching it. However, only 27.1% of internal medicine and 37.1% of cardiology programs offered any structured teaching of auscultation (P = 0.02). Programs without teaching were more likely to be large, university affiliated, and located in the northeast. The trainees' accuracy ranged from 0 to 56.2% for cardiology fellows (median, 21.9%) and from 2% to 36.8% for medical residents (median, 19.3%). Residents improved little with year of training and were never better than third-year medical students.

CONCLUSIONS

A low emphasis on cardiac auscultation appears to have affected the proficiency of medical trainees. Our study raises concern about the future of this time-honored art and, possibly, other bedside diagnostic skills.

Virtual reality training for the operating room and cardiac catheterisation laboratory

CONTEXT

High-profile cases of medical errors in the USA and UK, and major reports from organisations such as the US Institute of Medicine and UK Senate of Surgery, have sensitised the public and medical profession. Training is a key area that must be tackled to positively affect the problem of medical errors, especially in surgery and interventional cardiology. Despite the radically novel skills required for minimally invasive surgery or interventional cardiology, current training has gone largely unchanged. At the end of the 20th century, the public and the medical profession have concluded that training on patients is no longer acceptable.

STARTING POINT

Recently, Teodor Grantcharov and colleagues (Br J Surg 2004; 91: 146-50) did a randomised double-blind trial which showed that training by virtual reality (VR) significantly reduces objectively assessed intraoperative errors in laparoscopic cholecystectomy. They used a low-fidelity VR simulator. Much more sophisticated VR simulators exist for endoscopy, gynaecology, laparoscopy, orthopaedics, otolaryngology, robotics, and urology. There are few studies on the efficacy of these simulators in improving the safety of procedures on patients.

WHERE NEXT

There needs to be more large and multicentre studies. Technical skills training for procedural based medicine continues to be an ad-hoc mentor-based experience for the trainee, with experience gained by practising on patients. The skills required now are so difficult to learn that this type of training is no longer acceptable. VR-simulator-based training does work, but further empirical evidence is required to convince the more conservative members of the medical community.

Report of the National Heart, Lung, and Blood Institute Working Group on Research in Adult Congenital Heart Disease

The Working Group on research in adult congenital heart disease (ACHD) was convened in September 2004 under the sponsorship of National Heart, Lung, and Blood Institute (NHLBI) and the Office of Rare Diseases, National Institutes of Health, Department of Health and Human Services, to make recommendations on research needs. The purpose of the Working Group was to advise the NHLBI on the current state of the science in ACHD and barriers to optimal clinical care, and to make specific recommendations for overcoming those barriers. The members of the Working Group were chosen to provide expert input on a broad range of research issues from both scientific and lay perspectives. The Working Group reviewed data on the epidemiology of ACHD, long-term outcomes of complex cardiovascular malformations, issues in assessing morphology and function with current imaging techniques, surgical and catheter-based interventions, management of related conditions including pregnancy and arrhythmias, quality of life, and informatics. After research and training barriers were discussed, the Working Group recommended outreach and educational programs for adults with congenital heart disease, a network of specialized adult congenital heart disease regional centers, technology development to support advances in imaging and modeling of abnormal structure and function, and a consensus on appropriate training for physicians to provide care for adults with congenital heart disease.

Teaching from the clinical reasoning literature: combined reasoning strategies help novice diagnosticians overcome misleading information

OBJECTIVE

Previous research has revealed a pedagogical benefit of instructing novice diagnosticians to utilise a combined approach to clinical reasoning (familiarity-driven pattern recognition combined with a careful consideration of the presenting features) when diagnosing electrocardiograms (ECGs). This paper reports 2 studies demonstrating that the combined instructions are especially valuable in helping students overcome biasing influences.

METHODS

Undergraduate psychology students were trained to diagnose 10 cardiac conditions via ECG presentation. Half of all participants were instructed to reason in a combined manner and half were given no explicit instruction regarding the diagnostic task. In Study 1 (n = 60), half of each group was biased towards an incorrect diagnosis through presentation of counter-indicative features. In Study 2 (n = 48), a third of the test ECGs were presented with a correct diagnostic suggestion, a third with an incorrect suggestion, and a third without a suggestion.

RESULTS

Overall, the instruction to utilise a combined reasoning approach resulted in greater diagnostic accuracy relative to leaving students to their own intuitions regarding how best to approach new cases. The effect was particularly pronounced when cases were made challenging by biasing participants towards an incorrect diagnosis, either through mention of a specific feature or by making an inaccurate diagnostic suggestion.

DISCUSSION

These studies advance a growing body of evidence suggesting that various diagnostic strategies identified in the literature on clinical reasoning are not mutually exclusive and that trainees can benefit from explicit guidance regarding the value of both analytic and non-analytic reasoning tendencies.

Assessing housestaff diagnostic skills using a cardiology patient simulator

OBJECTIVE

To assess the cardiovascular physical examination skills of internal medicine housestaff.

DESIGN

Cross-sectional assessment of housestaff performance on three valvular abnormality simulations conducted on the cardiology patient simulator, "Harvey." Evaluations were done at the beginning (session I) and end (session II) of the academic year.

SETTING

Duke University Medical Center internal medicine training program.

SUBJECTS

Sixty-three (59%) of 107 eligible internal medicine housestaff (postgraduate years 1 through 3) agreed to participate and completed session I; 60 (95%) completed session II.

MEASUREMENTS

All volunteers were tested on three preprogrammed simulations (mitral regurgitation, mitral stenosis, and aortic regurgitation).

RESULTS

The overall correct response rates for all housestaff were 52% for mitral regurgitation, 37% for mitral stenosis, and 54% for aortic regurgitation. No difference was noted in correct response rates between sessions I and II. For mitral regurgitation, correct assessment of the contour of the holosystolic murmur predicted a correct diagnosis (P = 0.002). For mitral stenosis, identification of an opening snap and proper characterization of the mitral area diastolic murmur predicted a correct diagnosis (P < 0.0001). No individual observations were noted for the aortic regurgitation simulation, whose identification by the housestaff was associated with a correct diagnosis.

CONCLUSIONS

Housestaff had difficulty establishing a correct diagnosis for simulations of three common valvular heart diseases. Accurate recognition of a few "key" observations was associated with a correct diagnosis in two of the three diseases. Teaching housestaff to elicit and interpret a few critical signs accurately may improve their physical diagnosis abilities.

Simulation in health care education

During the past 15 years there has been widespread adoption of simulation in health care education as a method to train and assess learners. Multiple factors have contributed to this movement, including reduced patient availability, limited faculty teaching time, technological advances in diagnosis and treatment that require a new skills set, greater attention to patient safety with the need to reduce medical errors, and a focus on outcomes-based education. In this discussion, simulation refers broadly to any device or set of conditions that attempts to present the patient authentically. While simulation offers many advantages over traditional methods of teaching, there are several challenges that must be addressed to ensure its effective use. This article presents the range of available simulation technologies, explores the challenges that health care educators face when using this method, provides an example of a successful program that has integrated simulation into the culture of learning at its institution, and discusses an approach to maximizing the effectiveness of simulation as a means to learning and practicing skills in a safe, interactive environment.

Learning Curves and Reliability Measures for Virtual Reality Simulation in the Performance Assessment of Carotid Angiography

OBJECTIVES

Improvement in performance as measured by metric-based procedural errors must be demonstrated if virtual reality (VR) simulation is to be used as a valid means of proficiency assessment and improvement in procedural-based medical skills.

BACKGROUND

The Food and Drug Administration requires completion of VR simulation training for physicians learning to perform carotid stenting.

METHODS

Interventional cardiologists (n = 20) participating in the Emory NeuroAnatomy Carotid Training program underwent an instructional course on carotid angiography and then performed five serial simulated carotid angiograms on the Vascular Interventional System Trainer (VIST) VR simulator (Mentice AB, Gothenburg, Sweden). Of the subjects, 90% completed the full assessment. Procedure time (PT), fluoroscopy time (FT), contrast volume, and composite catheter handling errors (CE) were recorded by the simulator.

RESULTS

An improvement was noted in PT, contrast volume, FT, and CE when comparing the subjects' first and last simulations (all p < 0.05). The internal consistency of the VIST VR simulator as assessed with standardized coefficient alpha was high (range 0.81 to 0.93), except for FT (alpha = 0.36). Test-retest reliability was high for CE (r = 0.9, p = 0.0001).

CONCLUSIONS

A learning curve with improved performance was demonstrated on the VIST simulator. This study represents the largest collection of such data to date in carotid VR simulation and is the first report to establish the internal consistency of the VIST simulator and its test-retest reliability across several metrics. These metrics are fundamental benchmarks in the validation of any measurement device. Composite catheter handling errors represent measurable dynamic metrics with high test-retest reliability that are required for the high-stakes assessment of procedural skills.

Approaches to enhancing radiation safety in cardiovascular imaging: a scientific statement from the American Heart Association

Education, justification, and optimization are the cornerstones to enhancing the radiation safety of medical imaging. Education regarding the benefits and risks of imaging and the principles of radiation safety is required for all clinicians in order for them to be able to use imaging optimally. Empowering patients with knowledge of the benefits and risks of imaging will facilitate their meaningful participation in decisions related to their health care, which is necessary to achieve patient-centered care. Limiting the use of imaging to appropriate clinical indications can ensure that the benefits of imaging outweigh any potential risks. Finally, the continually expanding repertoire of techniques that allow high-quality imaging with lower radiation exposure should be used when available to achieve safer imaging. The implementation of these strategies in practice is necessary to achieve high-quality, patient-centered imaging and will require a shared effort and investment by all stakeholders, including physicians, patients, national scientific and educational organizations, politicians, and industry.

The rate at which residents learn to use hand-held echocardiography at the bedside

PURPOSE

Because there is little information about the training that general internists require to perform hand-carried cardiac ultrasonography (HCU), we studied the rate of learning of a group of medical residents performing HCU after minimal formal training.

METHODS

Medical residents on the inpatient services at Johns Hopkins Bayview Medical Center received formal training in HCU consisting of 15-30 minutes of didactic instruction about the principles of echocardiography, followed by ongoing one-on-one instruction in performing HCU and subsequent ongoing one-on-one training from a certified echocardiography technician as they were doing scans. The residents were shown how to position the patient to obtain 2-dimensional echo images from the parasternal short and long axes and apical 4-chamber views, and how to obtain color-flow Doppler images across the mitral and aortic valves. Residents were asked to determine whether pericardial effusion was present and to assess left ventricular size, left ventricular function, and the mitral and aortic valves. The residents performed cardiac physical examination and HCU independently on patients who had a conventional transthoracic echocardiogram (CTTE) performed within 24 hours of the HCU. The residents' HCU results were compared with the CTTE results by a cardiologist specializing in echocardiography. The rates at which residents gained technical proficiency and skills in interpreting their studies were measured by linear regression to fit various outcome variables against their experience at scanning as gauged by the number of scans performed.

RESULTS

Thirty medical residents performed a total of 231 HCU studies. Linear regression models showed that the residents' overall technical proficiency skills improved at the rate of 0.79 (95% confidence interval [CI] 0.53-1.04) points on an overall assessment index (0-3 scale) per 10 scans completed. Interpretation accuracy improved at a rate of 1.01 (95% CI 0.69-1.39) points per 10 scans as measured by an interpretation accuracy index (0-3 scale). Because scanning efforts and instruction in HCU occurred during residents' usual rotation duties, some residents gathered experience in HCU slowly and sporadically.

CONCLUSION

This study, the first prospective, experimental effort of its kind, shows that residents as a group learned important aspects of HCU scanning and interpretation at a reasonably rapid rate.

Designing a computer-based simulator for interventional cardiology training

Interventional cardiology training traditionally involves one-on-one experience following a master-apprentice model, much as other procedural disciplines. Development of a realistic computer-based training system that includes hand-eye coordination, catheter and guide wire choices, three-dimensional anatomic representations, and an integrated learning system is desirable, in order to permit learning to occur safely, without putting patients at risk. Here we present the first report of a PC-based simulator that incorporates synthetic fluoroscopy, real-time three-dimensional interactive anatomic display, and selective right- and left-sided coronary catheterization and angiography using actual catheters. Significant learning components also are integrated into the simulator.