Comparative effectiveness of instructional design features in simulation-based education: systematic review and meta-analysis

A Pilot Comparison of In-Room and Video Ratings of Team Behaviors of Students in Interprofesional Teams

Objective. To examine concordance between in-room and video faculty ratings of interprofessional behaviors in a standardized team objective structured clinical encounter (TOSCE). Methods. In-room and video-rated student performance scores in an interprofessional 2-station TOSCE were compared using a validated 3-point scale assessing six team competencies. Scores for each student were derived from two in-room faculty members and one faculty member who viewed video recordings of the same team encounter from equivalent visual vantage points. All faculty members received the same rigorous rater training. Paired sample t-tests were used to compare individual student scores. McNemar's test was used to compare student pass/fail rates to determine the impact of rating modality on performance scores. Results. In-room and video student scores were captured for 12 novice teams (47 students) with each team consisting of students from four professions (medicine, pharmacy, physician assistant, nursing). Video ratings were consistently lower for all competencies and significantly lower for competencies of roles and responsibilities, and conflict management. Using a criterion of an average score of 2 out of 3 for at least one station for passing, 56% of students passed when rated in-room compared with 20% when rated by video. Conclusion. In-room and video ratings are not equal. Educators should consider scoring discrepancies based on modality when assessing team behaviors.

Research Gaps in Wilderness Medicine

INTRODUCTION

Wilderness medicine involves the treatment of individuals in remote, austere environments. Given the high potential for injuries as well as the unique treatment modalities required in wilderness medicine, evidence-based clinical practice guidelines are necessary to provide optimal care. In this study, we identify evidence gaps from low-quality recommendations in wilderness medicine clinical practice guidelines and identify new/ongoing research addressing them.

METHODS

We included relevant clinical practice guidelines from the Wilderness Medical Society and obtained all 1C or 2C level recommendations. Patient/Problem/Population, intervention, comparison, outcome (PICO) questions were created to address each recommendation. Using 24 search strings, we extracted titles, clinical trial registry number, and recruitment status for 8899 articles. We categorized the articles by trial design to infer the effect they may have on future recommendations.

RESULTS

Twelve clinical practice guidelines met inclusion criteria. From these we located 275 low-quality recommendations and used them to create 275 PICO questions. Thirty-three articles were relevant to the PICO questions. Heat-related illness had the highest number of relevant articles (n=9), but acute pain and altitude sickness had the most randomized clinical trials (n=6).

CONCLUSION

Overall, few studies were being conducted to address research gaps in wilderness medicine. Heat-related illness had the most new or ongoing research, whereas no studies were being conducted to address gaps in eye injuries, basic wound management, or spine immobilization. Animals, cadavers, and mannequin research are useful in cases in which human evidence is difficult to obtain. Establishing research priorities is recommended for addressing research gaps identified by guideline panels.

A review of the curriculum development process of simulation-based educational intervention studies in Korea

BACKGROUND

Despite the increase in simulators at nursing schools and the high expectations regarding simulation for nursing education, the unique features of integrating simulation-based education into the curriculum are unclear.

OBJECTIVE

The purpose of this study was to assess the curriculum development process of simulation-based educational interventions in nursing in Korea.

DESIGN

Integrative review of literature used.

DATA SOURCES

Korean Studies Information Services System (KISS), Korean Medical Database (KMbase), KoreaMed, Research Information Sharing Service (RISS), and National Digital Library (NDL).

METHODS

Comprehensive databases were searched for records without a time limit (until December 2016), using terms such as "nursing," "simulation," and "education." A total of 1006 studies were screened. According to the model for simulation-based curriculum development (Khamis et al., 2016), the quality of reporting on the curriculum development was reviewed.

RESULTS

A total of 125 papers were included in this review. In three studies, simulation scenarios were made from easy to difficulty levels, and none of the studies presented the level of learners' proficiency. Only 17.6% of the studies reported faculty development or preparation. The inter-rater reliability was presented in performance test by 24 studies and two studies evaluated the long-term effects of simulation education although there was no statistically significant change in terms of publication years.

CONCLUSION

These findings suggest that educators and researchers should pay more attention to the educational strategies to integrate simulation into nursing education. It could contribute to guiding educators and researchers to develop a simulation-based curriculum and improve the quality of nursing education research.

Simulation for competency assessment in vascular and cardiac ultrasound

Healthcare providers who use peripheral vascular and cardiac ultrasound require specialized training to develop the technical and interpretive skills necessary to perform accurate diagnostic tests. Assessment of competence is a critical component of training that documents a learner's progress and is a requirement for competency-based medical education (CBME) as well as specialty certification or credentialing. The use of simulation for CBME in diagnostic ultrasound is particularly appealing since it incorporates both the psychomotor and cognitive domains while eliminating dependency on the availability of live patients with a range of pathology. However, successful application of simulation in this setting requires realistic, full-featured simulators and appropriate standardized metrics for competency testing. The principal diagnostic parameter in peripheral vascular ultrasound is measurement of peak systolic velocity (PSV) on Doppler spectral waveforms, and simulation of Doppler flow detection presents unique challenges. The computer-based duplex ultrasound simulator developed at the University of Washington uses computational fluid dynamics modeling and presents real-time color-flow Doppler images and Doppler spectral waveforms along with the corresponding B-mode images. This simulator provides a realistic scanning experience that includes measuring PSV in various arterial segments and applying actual diagnostic criteria. Simulators for echocardiography have been available since the 1990s and are currently more advanced than those for peripheral vascular ultrasound. Echocardiography simulators are now offered for both transesophageal echo and transthoracic echo. These computer-based simulators have 3D graphic displays that provide feedback to the learner and metrics for assessment of technical skill that are based on transducer tracking data. Such metrics provide a motion-based or kinematic analysis of skill in performing cardiac ultrasound. The use of simulation in peripheral vascular and cardiac ultrasound can provide a standardized and readily available method for training and competency assessment.

Assessment tool for the instructional design of simulation-based team training courses: the ID-SIM

Introduction

To achieve an expert performance of care teams, adequate simulation-based team training courses with an effective instructional design are essential. As the importance of the instructional design becomes ever more clear, an objective assessment tool would be valuable for educators and researchers. Therefore, we aimed to develop an evidence-based and objective assessment tool for the evaluation of the instructional design of simulation-based team training courses.

Methods

A validation study in which we developed an assessment tool containing an evidence-based questionnaire with Visual Analogue Scale (VAS) and a visual chart directly translating the results of the questionnaire. Psychometric properties of the assessment tool were tested using five descriptions of simulation-based team training courses. An expert-opinion-based ranking from poor to excellent was obtained. Ten independent raters assessed the five training courses twice, by using the developed questionnaire with an interval of 2 weeks. Validity and reliability analyses were performed by using the scores from the raters and comparing them with the expert’s ranking. Usability was assessed by an 11-item survey.

Results

A 42-item questionnaire, using VAS, and a propeller chart were developed. The correlation between the expert-opinion-based ranking and the evaluators’ scores (Spearman correlation) was 0.95, and the variance due to subjectivity of raters was 3.5% (VTraining*Rater). The G-coefficient was 0.96. The inter-rater reliability (intraclass correlation coefficient (ICC)) was 0.91 (95% CI 0.77 to 0.99), and intra-rater reliability for the overall score (ICC) was ranging from 0.91 to 0.99.

Conclusions

We developed an evidence-based and reliable assessment tool for the evaluation of the instructional design of a simulation-based team training: the ID-SIM. The ID-SIM is available as a free mobile application.

Randomized controlled trials of simulation-based interventions in Emergency Medicine: a methodological review

The number of trials assessing Simulation-Based Medical Education (SBME) interventions has rapidly expanded. Many studies show that potential flaws in design, conduct and reporting of randomized controlled trials (RCTs) can bias their results. We conducted a methodological review of RCTs assessing a SBME in Emergency Medicine (EM) and examined their methodological characteristics. We searched MEDLINE via PubMed for RCT that assessed a simulation intervention in EM, published in 6 general and internal medicine and in the top 10 EM journals. The Cochrane Collaboration risk of Bias tool was used to assess risk of bias, intervention reporting was evaluated based on the "template for intervention description and replication" checklist, and methodological quality was evaluated by the Medical Education Research Study Quality Instrument. Reports selection and data extraction was done by 2 independents researchers. From 1394 RCTs screened, 68 trials assessed a SBME intervention. They represent one quarter of our sample. Cardiopulmonary resuscitation (CPR) is the most frequent topic (81%). Random sequence generation and allocation concealment were performed correctly in 66 and 49% of trials. Blinding of participants and assessors was performed correctly in 19 and 68%. Risk of attrition bias was low in three-quarters of the studies (n = 51). Risk of selective reporting bias was unclear in nearly all studies. The mean MERQSI score was of 13.4/18.4% of the reports provided a description allowing the intervention replication. Trials assessing simulation represent one quarter of RCTs in EM. Their quality remains unclear, and reproducing the interventions appears challenging due to reporting issues.

Effectiveness of serious games and impact of design elements on engagement and educational outcomes in healthcare professionals and students: a systematic review and meta-analysis protocol

INTRODUCTION

Serious games (SGs) are interactive and entertaining digital software with an educational purpose. They engage the learner by proposing challenges and through various design elements (DEs; eg, points, difficulty adaptation, story). Recent reviews suggest the effectiveness of SGs in healthcare professionals' and students' education is mixed. This could be explained by the variability in their DEs, which has been shown to be highly variable across studies. The aim of this systematic review is to identify, appraise and synthesise the best available evidence regarding the effectiveness of SGs and the impact of DEs on engagement and educational outcomes of healthcare professionals and students.

METHODS AND ANALYSIS

A systematic search of the literature will be conducted using a combination of medical subject headings terms and keywords in Cumulative Index of Nursing and Allied Health, Embase, Education Resources Information Center, PsycInFO, PubMed and Web of Science. Studies assessing SGs on engagement and educational outcomes will be included. Two independent reviewers will conduct the screening as well as the data extraction process. The risk of bias of included studies will also be assessed by two reviewers using the Effective Practice and Organisation of Care criteria. Data regarding DEs in SGs will first be synthesised qualitatively. A meta-analysis will then be performed, if the data allow it. Finally, the quality of the evidence regarding the effectiveness of SGs on each outcome will be assessed using the Grading of Recommendations Assessment, Development and Evaluation approach.

ETHICS AND DISSEMINATION

As this systematic review only uses already collected data, no Institutional Review Board approval is required. Its results will be submitted in a peer-reviewed journal by the end of 2018.

PROSPERO REGISTRATION NUMBER

CRD42017077424.

Knowing How and Knowing Why: testing the effect of instruction designed for cognitive integration on procedural skills transfer

Transfer is a desired outcome of simulation-based training, yet evidence for how instructional design features promote transfer is lacking. In clinical reasoning, transfer is improved when trainees experience instruction integrating basic science explanations with clinical signs and symptoms. To test whether integrated instruction has similar effects in procedural skills (i.e., psychomotor skills) training, we studied the impact of instruction that integrates conceptual (why) and procedural (how) knowledge on the retention and transfer of simulation-based lumbar puncture (LP) skill. Medical students (N = 30) were randomized into two groups that accessed different instructional videos during a 60-min self-regulated training session. An unintegrated video provided procedural How instruction via step-by-step demonstrations of LP, and an integrated video provided the same How instruction with integrated conceptual Why explanations (e.g., anatomy) for key steps. Two blinded raters scored post-test, retention, and transfer performances using a global rating scale. Participants also completed written procedural and conceptual knowledge tests. We used simple mediation regression analyses to assess the total and indirect effects (mediated by conceptual knowledge) of integrated instruction on retention and transfer. Integrated instruction was associated with improved conceptual (p < .001) but not procedural knowledge test scores (p = .11). We found no total effect of group (p > .05). We did find a positive indirect group effect on skill retention (B _ab  = .93, p < .05) and transfer (B _ab  = .59, p < .05), mediated through participants improved conceptual knowledge. Integrated instruction may improve trainees' skill retention and transfer through gains in conceptual knowledge. Such integrated instruction may be an instructional design feature for simulation-based training aimed at improving transfer outcomes.

Rapid response systems for paediatrics: Suggestions for optimal organization and training

Resuscitation and cardiac arrest events in the paediatric population are rare occurrences. Improving outcomes from such events continues to be a difficult challenge. Rapid response systems and teams have been integrated into many hospitals in an effort to facilitate early identification and management of patients at risk for clinical deterioration. Optimizing education in the form of team training is a major component of successful team performance. Simulation-based team training, is a key educational supplement for existing standardized resuscitation courses. This position statement describes the evidence supporting rapid response systems and teams as well as simulation-based team training and provides recommendations for implementation in hospital care for paediatric patients.

The value proposition of simulation-based education

Simulation has become an integral part of physician education, and abundant evidence confirms that simulation-based education improves learners' skills and behaviors and is associated with improved patient outcomes. The resources required to implement simulation-based education, however, have led some stakeholders to question the overall value proposition of simulation-based education. This paper summarizes the information from a special panel on this topic and defines research priorities for the field. Future work should focus on both outcomes and costs, with robust measurement of resource investments, provider performance (in both simulation and real settings), patient outcomes, and impact on the health care organization. Increased attention to training practicing clinicians and health care teams is also essential. Clarifying the value proposition of simulation-based education will require a major national effort with funding from multiple sponsors and active engagement of a variety of stakeholders.

Self-perceived long-term transfer of learning after postpartum hemorrhage simulation training

OBJECTIVE

To explore long-term transfer (application of acquired knowledge and skills on the job) after postpartum hemorrhage simulation training based on either instructional design (ID) principles or conventional best practice.

METHODS

In this qualitative study, semi-structured interviews with obstetrics and gynecology healthcare practitioners were conducted between August 7 and September 26, 2015, in Recife, Brazil. The participants were randomly selected from each of two postpartum hemorrhage simulations attended 2 years earlier (one ID and one conventional best practice). Thematic analysis was used to explore (1) residents' perceptions of long-term transfer of learning, (2) ID elements influencing the perceived long-term transfer, and (3) differences in the participants' perceptions according to the type of simulation attended.

RESULTS

There were 12 interview participants. After either simulation format, residents perceived long-term transfer effects. Training design factors influencing transfer were, in their opinion, related to trainees' characteristics, simulation design, and workplace environment. Trainees who participated in the ID-based simulation perceived better communication skills and better overall situational awareness: "I didn't do that before."

CONCLUSION

All residents perceived long-term transfer after simulation training for postpartum hemorrhage. Those who attended the ID format additionally perceived improvements in communication skills and situational awareness, which are fundamental factors in the management of postpartum hemorrhage.

Fixed versus variable practice for teaching medical students the management of pediatric asthma exacerbations using simulation

Simulation-based trainings represent an interesting approach to teach medical students the management of pediatric asthma exacerbations (PAEs). In this study, we compared two pedagogical approaches, training students once on three different scenarios of PAEs versus training students three times on the same scenario of PAE. Eighty-five third-year medical students, novice learners for the management of PAEs, were randomized and trained. Students were assessed twice, 1 week and 4 months after the training, on a scenario of PAE new to both groups and on scenarios used during the training. The main outcome was the performance score on the new scenario of PAE at 1 week, assessed on a checklist custom-designed for the study. All students progressed rapidly and acquired excellent skills. One week after the training, there was no difference between the two groups on all the scenarios tested, including the new scenario of PAE (median performance score (IQR) of 8.3 (7.4-10.0) in the variation group versus 8.0 (6.0-10.0) in the repetition group (p = 0.16)). Four months later, the performance of the two groups remained similar.

CONCLUSION

Varying practice with different scenarios was equivalent to repetitive practice on the same scenario for novice learners, with both methods leading to transfer and long-term retention of the skills acquired during the training. What is known: • Simulation-based trainings represent an interesting approach to teach medical students the management of pediatric asthma exacerbations. • It is unclear whether students would benefit more from repetitive practice on the same scenario of asthma exacerbation or from practice on different scenarios in terms of transfer of skills. What is new: • An individual 30-min training on the management of pediatric asthma exacerbations using simulation allows transfer and long-term retention of the skills acquired. • Varying practice with different scenarios is equivalent to repetitive practice on the same scenario in terms of transfer of skills.

Centros de simulação e projeto pedagógico: dois lados da mesma moeda

OBJETIVOS: Discutir as principais questões relacionadas ao uso e estruturação de um centro de simulação: logística, aproveitamento dos recursos e alinhamento entre o projeto pedagógico e as atividades simuladas.MÉTODOS: Revisão narrativa da literatura, com busca nas bases de dados Web of Science, PubMed, SciELO e Google Scholar, incluiu artigos publicados até junho de 2017.RESULTADOS: A simulação médica tem sido inserida nos cursos da área de saúde tanto para treinamento quanto para avaliação. Com isso, houve um grande aumento de centros de simulação, os quais envolvem um alto custo tanto para sua infraestrutura geral e especificidades como para manutenção. Muitas vezes os centros de simulação têm uma baixa aceitação docente e discente, tornando-os subutilizados, principalmente quando a metodologia não é implementada adequadamente. O método será mais valorizado quando houver um alinhamento entre o projeto pedagógico e as atividades simuladas. O planejamento de um centro de simulação é uma tarefa árdua que exige estudo prévio, conhecimento sobre o currículo institucional, orçamento e visitas a outros centros já existentes para evitar grandes ajustes posteriores que potencialmente são difíceis e onerosos. Para um melhor aproveitamento logístico, é necessário inicialmente identificar o público alvo e a inserção curricular do método, para definir a quantidade de participantes, tipos de salas e tipos de simuladores. A capacitação docente e o envolvimento multiprofissional são necessários para o bom funcionamento do centro de simulação, sendo que um dos principais motivos para a sua subutilização é a falta de capacitação docente. Além de capacitar os docentes, outras formas para utilizar os centros de simulação estão relacionadas à pesquisa e à parceria entre escolas de medicina e serviços hospitalares. Nessa parceria, as instituições de ensino ganham oportunidades de financiamento e interação com a sociedade, enquanto os hospitais ganham qualificação e aumentam a segurança de seus pacientes. Todas essas questões devem ser consideradas quando o estudo orçamentário é realizado, pois apenas o ambiente físico e os simuladores não são suficientes para o aproveitamento do ambiente simulado.CONCLUSÕES: A integração entre educação, pesquisa e assistência, e o alinhamento com o projeto pedagógico, são de extrema importância para a utilização da simulação na área da saúde, e essenciais para o desenvolvimento de novos treinamentos e conhecimentos.

Transfer of Training: The Known and the Unknown

Transfer of training is one of the oldest topics of interest to industrial and organizational (I/O) psychologists. Drawing on several meta-analytic studies and recent empirical work, we first synthesize what is now reliably known with respect to the generalization and retention of learned knowledge and skills to work contexts. The second part of our review focuses on what is unknown—the significant gaps in our knowledge where we believe new directions in our research strategies are warranted. We offer three prescriptions: (a) going one step beyond most existing studies to offer greater precision in our specification and measurement of variables and interventions, (b) connecting the dots by focusing on transfer criteria and transfer trajectories, and (c) shifting the operative paradigm of research to examine contemporary learning from a problem-centered perspective. There is ample opportunity to increase the yield on enormous organizational investments in training if transfer scholars and practitioners are fully informed of what is known and prepared to systematically confront the unknown in new and innovative ways.

Comparison between Simulation-based Training and Lecture-based Education in Teaching Situation Awareness. A Randomized Controlled Study

RATIONALE

Situation awareness has been defined as the perception of the elements in the environment within volumes of time and space, the comprehension of their meaning, and the projection of their status in the near future. Intensivists often make time-sensitive critical decisions, and loss of situation awareness can lead to errors. It has been shown that simulation-based training is superior to lecture-based training for some critical scenarios. Because the methods of training to improve situation awareness have not been well studied in the medical field, we compared the impact of simulation vs. lecture training using the Situation Awareness Global Assessment Technique (SAGAT) score.

OBJECTIVES

To identify an effective method for teaching situation awareness.

METHODS

We randomly assigned 17 critical care fellows to simulation vs. lecture training. Training consisted of eight cases on airway management, including topics such as elevated intracranial pressure, difficult airway, arrhythmia, and shock. During the testing scenario, at random times between 4 and 6 minutes into the simulation, the scenario was frozen, and the screens were blanked. Respondents then completed the 28 questions on the SAGAT scale. Sample items were categorized as Perception, Projection, and Comprehension of the situation. Results were analyzed using SPSS Version 21.

RESULTS

Eight fellows from the simulation group and nine from the lecture group underwent simulation testing. Sixty-four SAGAT scores were recorded for the simulation group and 48 scores were recorded for the lecture group. The mean simulation vs. lecture group SAGAT score was 64.3 ± 10.1 (SD) vs. 59.7 ± 10.8 (SD) (P = 0.02). There was also a difference in the median Perception ability between the simulation vs. lecture groups (61.1 vs. 55.5, P = 0.01). There was no difference in the median Projection and Comprehension scores between the two groups (50.0 vs. 50.0, P = 0.92, and 83.3 vs. 83.3, P = 0.27).

CONCLUSIONS

We found a significant, albeit modest, difference between simulation training and lecture training on the total SAGAT score of situation awareness mainly because of the improvement in perception ability. Simulation may be a superior method of teaching situation awareness.

Basics of Infant Conventional Mechanical Ventilation: An Interactive Animated Teaching Module

INTRODUCTION

While conventional mechanical ventilation is a common therapy in the neonatal intensive care unit (NICU), pediatric residents receive insufficient instruction. This stand-alone computer module provides an interactive method of learning basic infant pulmonary physiology and principles of mechanical ventilation.

METHODS

This module runs offline and is compatible with a variety of operating systems. Participants complete a six-question, case-based pretest. The seven-section instructional module is self-paced, narrated, animated, and interactive. Learners can repeat each section as needed. At the conclusion of the module, participants complete the same six-question test and receive feedback. In total, the module requires 15-20 minutes to complete.

RESULTS

The curriculum has been implemented at the beginning of the NICU rotation over a 2-year period within our pediatric residency program. Participants preferred this interactive module and had higher posttest scores when compared to a PowerPoint presentation. After 4 months, there was evidence of knowledge decay.

DISCUSSION

The interactive module is enjoyable, effective, and convenient. It engages participants in active learning and allows them to control the time and pace of their instruction. We have implemented the curriculum within our residency program and believe it would be useful for a variety of NICU health care providers.

Basic Principles of Simulations: In situ and Online Simulation

Medical simulation is an artificial and a faithful representation of real clinical situations with the help of static and interactive doll simulators, standardized patients/actors, models for the exercise, simulation on a computer screen and a “serious” performance of a real clinical situation. That is a modern way of learning that helps students and health workers to achieve a higher level of expertise and provide a safer health care. Medical simulation is one of the greatest and most important innovations in medical education for the past 20 years. The goal of the simulation is to provide an optimal relationship between education and patient’s safety. It is used both in undergraduate and postgraduate studies, as well as in continuing medical education. Simulators used in medical education can be divided into six groups, ranging from the use of paper and pencils to the most modern computerized interactive simulator. Depending on the economic development level and the organization of health care, rigorous high medical simulation can be performed in three ways: in the center for medical simulation, in the form of “in situ” simulation, and as remote simulations.

Systematic Review of Patient-Specific Surgical Simulation: Toward Advancing Medical Education

OBJECTIVE

Simulation-based education has been shown to be an effective tool to teach foundational technical skills in various surgical specialties. However, most of the current simulations are limited to generic scenarios and do not allow continuation of the learning curve beyond basic technical skills to prepare for more advanced expertise, such as patient-specific surgical planning. The objective of this study was to evaluate the current medical literature with respect to the utilization and educational value of patient-specific simulations for surgical training.

METHODS

We performed a systematic review of the literature using Pubmed, Embase, and Scopus focusing on themes of simulation, patient-specific, surgical procedure, and education. The study included randomized controlled trials, cohort studies, and case-control studies published between 2005 and 2016. Two independent reviewers (W.H.R. and N.D) conducted the study appraisal, data abstraction, and quality assessment of the studies.

RESULTS

The search identified 13 studies that met the inclusion criteria; 7 studies employed computer simulations and 6 studies used 3-dimensional (3D) synthetic models. A number of surgical specialties evaluated patient-specific simulation, including neurosurgery, vascular surgery, orthopedic surgery, and interventional radiology. However, most studies were small in size and primarily aimed at feasibility assessments and early validation.

CONCLUSIONS

Early evidence has shown feasibility and utility of patient-specific simulation for surgical education. With further development of this technology, simulation-based education may be able to support training of higher-level competencies outside the clinical settingto aid learners in their development of surgical skills.

Core Competencies or a Competent Core? A Scoping Review and Realist Synthesis of Invasive Bedside Procedural Skills Training in Internal Medicine

PURPOSE

Invasive bedside procedures are core competencies for internal medicine, yet no formal training guidelines exist. The authors conducted a scoping review and realist synthesis to characterize current training for lumbar puncture, arthrocentesis, paracentesis, thoracentesis, and central venous catheterization. They aimed to collate how educators justify using specific interventions, establish which interventions have the best evidence, and offer directions for future research and training.

METHOD

The authors systematically searched Medline, Embase, the Cochrane Library, and ERIC through April 2015. Studies were screened in three phases; all reviews were performed independently and in duplicate. The authors extracted information on learner and patient demographics, study design and methodological quality, and details of training interventions and measured outcomes. A three-step realist synthesis was performed to synthesize findings on each study's context, mechanism, and outcome, and to identify a foundational training model.

RESULTS

From an initial 6,671 studies, 149 studies were further reduced to 67 (45%) reporting sufficient information for realist synthesis. Analysis yielded four types of procedural skills training interventions. There was relative consistency across contexts and significant differences in mechanisms and outcomes across the four intervention types. The medical procedural service was identified as an adaptable foundational training model.

CONCLUSIONS

The observed heterogeneity in procedural skills training implies that programs are not consistently developing residents who are competent in core procedures. The findings suggest that researchers in education and quality improvement will need to collaborate to design training that develops a "competent core" of proceduralists using simulation and clinical rotations.

Personalized Video Feedback and Repeated Task Practice Improve Laparoscopic Knot-Tying Skills: Two Controlled Trials

PURPOSE

Compare the effect of personalized feedback (PF) vs. task demonstration (TD), both delivered via video, on laparoscopic knot-tying skills and perceived workload; and evaluate the effect of repeated practice.

METHOD

General surgery interns and research fellows completed four repetitions of a simulated laparoscopic knot-tying task at one-month intervals. Midway between repetitions, participants received via e-mail either a TD video (demonstration by an expert) or a PF video (video of their own performance with voiceover from a blinded senior surgeon). Each participant received at least one video per format, with sequence randomly assigned. Outcomes included performance scores and NASA Task Load Index (NASA-TLX) scores. To evaluate the effectiveness of repeated practice, scores from these trainees on a separate delayed retention test were compared against historical controls who did not have scheduled repetitions.

RESULTS

Twenty-one trainees completed the randomized study. Mean change in performance scores was significantly greater for those receiving PF (difference = 23.1 of 150 [95% confidence interval (CI): 0, 46.2], P = .05). Perceived workload was also significantly reduced (difference = -3.0 of 20 [95% CI: -5.8, -0.3], P = .04). Compared with historical controls (N = 93), the 21 with scheduled repeated practice had higher scores on the laparoscopic knot-tying assessment two weeks after the final repetition (difference = 1.5 of 10 [95% CI: 0.2, 2.8], P = .02).

CONCLUSIONS

Personalized video feedback improves trainees' procedural performance and perceived workload compared with a task demonstration video. Brief monthly practice sessions support skill acquisition and retention.

Otologic Skills Training

This article presents a summary of the current simulation training for otologic skills. There is a wide variety of educational approaches, assessment tools, and simulators in use, including simple low-cost task trainers to complex computer-based virtual reality systems. A systematic approach to otologic skills training using adult learning theory concepts, such as repeated and distributed practice, self-directed learning, and mastery learning, is necessary for these educational interventions to be effective. Future directions include development of measures of performance to assess efficacy of simulation training interventions and, for complex procedures, improvement in fidelity based on educational goals.

A review of simulation-enhanced, team-based cardiopulmonary resuscitation training for undergraduate students

Cardiopulmonary resuscitation training is an essential element of clinical skill development for healthcare providers. The International Liaison Committee on Resuscitation has described issues related to cardiopulmonary resuscitation and emergency cardiovascular care education. Educational interventions have been initiated to try to address these issues using a team-based approach and simulation technologies that offer a controlled, safe learning environment. The aim of the study is to review and synthesize published studies that address the primary question "What are the features and effectiveness of educational interventions related to simulation-enhanced, team-based cardiopulmonary resuscitation training?" We conducted a systematic review focused on educational interventions pertaining to cardiac arrest and emergencies that addressed this main question. The findings are presented together with a discussion of the effectiveness of various educational interventions. In conclusion, student attitudes toward interprofessional learning and simulation experiences were more positive. Research reports emphasized the importance of adherence to established guidelines, adopting a holistic approach to training, and that preliminary training, briefing, deliberate practices, and debriefing should help to overcome deficiencies in cardiopulmonary resuscitation training.

Simulation Fellowship Programs: An International Survey of Program Directors

PURPOSE

To report on the evolution of simulation-based training (SBT) by identifying the composition and infrastructure of existing simulation fellowship programs, describing the current training practices, disclosing existing program barriers, and highlighting opportunities for standardization.

METHOD

Investigators conducted a cross-sectional survey study among English-speaking simulation fellowship program directors (September 2014-September 2015). They identified fellowships through academic/institutional Web sites, peer-reviewed literature, Web-based search engines, and snowball sampling. They invited programs to participate in the Web-based questionnaire via e-mail and follow-up telephone calls.

RESULTS

Forty-nine programs met the inclusion criteria. Of these, 32 (65%) responded to the survey. Most programs were based in the United States, but others were from Canada, England, and Australia. Over half of the programs started in or after 2010. Across all 32 programs, 186 fellows had graduated since 1998. Fellows and directors were primarily departmentally funded; programs were primarily affiliated with hospitals and/or medical schools, many of which had sponsoring centers accredited by governing bodies. Fellows were typically medical trainees; directors were typically physicians. The majority of programs (over 90%) covered four core objectives, and all endorsed similar educational outcomes. Respondents identified no significant universal barriers to program success. Most directors (18/28 [64%]) advocated standardized fellowship guidelines on a national level.

CONCLUSIONS

Paralleling the fast growth and integration of SBT, fellowship training opportunities have grown rapidly in the United States, Canada, and beyond. This study highlights potential areas for standardization and accreditation of simulation fellowships which would allow measurable competencies in graduates.

Effectiveness and cost-effectiveness of embedded simulation in occupational therapy clinical practice education: study protocol for a randomised controlled trial

Background

Clinical placements are a critical component of the training for health professionals such as occupational therapists. However, with growing student enrolments in professional education courses and workload pressures on practitioners, it is increasingly difficult to find sufficient, suitable placements that satisfy program accreditation requirements. The professional accrediting body for occupational therapy in Australia allows up to 200 of the mandatory 1000 clinical placement hours to be completed via simulation activities, but evidence of effectiveness and efficiency for student learning outcomes is lacking. Increasingly placement providers charge a fee to host students, leading educators to consider whether providing an internal program might be a feasible alternative for a portion of placement hours. Economic analysis of the incremental costs and benefits of providing a traditional versus simulated placement is required to inform decision-making.

Methods/design

This study is a pragmatic, non-inferiority, single-blind, multicentre, two-group randomised controlled trial (RCT) with an embedded economic analysis. The RCT will compare a block of 40 hours of simulated placement (intervention) with a 40-hour block of traditional placement (comparator), with a focus on student learning outcomes and delivery costs. Six universities will instigate the educational intervention within their respective occupational therapy courses, randomly assigning their cohort of students (1:1 allocation) to the simulated or traditional clinical placements. The primary outcome is achievement of professional behaviours (e.g. communication, clinical reasoning) as assessed by a post-placement written examination. Secondary outcomes include proportions passing the placement assessed using the Student Practice Evaluation Form-Revised, changes in student confidence pre-/post-placement, student and educator evaluation of the placement experience and cost-effectiveness of simulated versus traditional clinical placements. Comprehensive cost data will be collected for both the simulated and traditional placement programs at each site for economic evaluation.

Discussion

Use of simulation in health-related fields like occupational therapy is common, but these activities usually relate to brief opportunities for isolated skill development. The simulated clinical placement evaluated in this trial is less common because it encapsulates a 5-day block of integrated activities, designed and delivered in a manner intended to emulate best-practice placement experiences. The planned study is rare due to inclusion of an economic analysis that aims to provide valuable information about the relationship between costs and outcomes across participating sites.

Trial registration

Australian New Zealand Clinical Trials Registry, ACTRN12616001339448. Registered 26 September 2016.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-2087-0) contains supplementary material, which is available to authorized users.

Interprofessional objective structured teaching exercise (iOSTE) to train preceptors

BACKGROUND AND PURPOSE

Interprofessional education (IPE) is important in the education of all health care students, yet limited IPE training has been provided to preceptors who train these students in the clinical setting. Simulation using the standardized student model has been used to train health care preceptors in medicine. To our knowledge, there are no reports utilizing interprofessional objective structured teaching exercises (iOSTE) to train pharmacy preceptors. The primary objectives of this pilot study were to evaluate the effects of iOSTE on the pharmacy preceptors' perceived importance of the Interprofessional Education Collaborative (IPEC) core competencies and confidence in precepting interprofessional students. Additionally, data were collected regarding pharmacy preceptors' prior experiences in simulation and debriefing.

EDUCATIONAL ACTIVITY AND SETTING

Preceptors (n=23) participated in an iOSTE and debriefed with trained standardized nursing and pharmacy students caring for a trained standardized asthma patient.

FINDINGS

Pre- versus post-iOSTE survey data showed statistically significant improvements in all self-confidence related items, including the following abilities: precept students from different disciplines (p=0.004), facilitate a simulation activity (p=0.001), conduct the debriefing process (p<0.001), and discuss with students the IPE core competencies (p=0.001). Additionally, responses to post-iOSTE survey questions assessing the learning activity showed high ratings (median=5, interquartile range=4 to 5). Pharmacy preceptors increased their teaching ability and confidence level in communicating with students from other health care professions.

SUMMARY

These findings indicate that iOSTE is a useful and well-received method for preceptor development.

Improving patient safety through better teamwork: how effective are different methods of simulation debriefing? Protocol for a pragmatic, prospective and randomised study

INTRODUCTION

Medical errors have an incidence of 9% and may lead to worse patient outcome. Teamwork training has the capacity to significantly reduce medical errors and therefore improve patient outcome. One common framework for teamwork training is crisis resource management, adapted from aviation and usually trained in simulation settings. Debriefing after simulation is thought to be crucial to learning teamwork-related concepts and behaviours but it remains unclear how best to debrief these aspects. Furthermore, teamwork-training sessions and studies examining education effects on undergraduates are rare. The study aims to evaluate the effects of two teamwork-focused debriefings on team performance after an extensive medical student teamwork training.

METHODS AND ANALYSES

A prospective experimental study has been designed to compare a well-established three-phase debriefing method (gather-analyse-summarise; the GAS method) to a newly developed and more structured debriefing approach that extends the GAS method with TeamTAG (teamwork techniques analysis grid). TeamTAG is a cognitive aid listing preselected teamwork principles and descriptions of behavioural anchors that serve as observable patterns of teamwork and is supposed to help structure teamwork-focused debriefing. Both debriefing methods will be tested during an emergency room teamwork-training simulation comprising six emergency medicine cases faced by 35 final-year medical students in teams of five. Teams will be randomised into the two debriefing conditions. Team performance during simulation and the number of principles discussed during debriefing will be evaluated. Learning opportunities, helpfulness and feasibility will be rated by participants and instructors. Analyses will include descriptive, inferential and explorative statistics.

ETHICS AND DISSEMINATION

The study protocol was approved by the institutional office for data protection and the ethics committee of Charité Medical School Berlin and registered under EA2/172/16. All students will participate voluntarily and will sign an informed consent after receiving written and oral information about the study. Results will be published.

Academic Primer Series: Key Papers About Teaching with Technology

Introduction

Modern learners have immediate, unlimited access to a wide variety of online resources. To appeal to this current generation of learners, educators must embrace the use of technology. However, educators must balance newer, novel technologies with traditional methods to achieve the best learning outcomes. Therefore, we aimed to review several papers useful for faculty members wishing to incorporate technology into instructional design.

Methods

We identified a broad list of papers relevant to teaching and learning with technology within the online discussions of the Academic Life in Emergency Medicine (ALiEM) Faculty Incubator. This list was augmented with suggestions by a guest expert (BT) and an open call on Twitter (tagged with the #meded and #FOAMed hashtags) yielding 24 papers. We then conducted a modified three-round Delphi process within the authorship group, including junior and senior faculty members, to identify the most impactful papers.

Results

We pared the list of 24 papers to five that were most highly rated. Two were research papers and three were commentaries or editorials. The authorship group reviewed and summarized these papers with specific consideration to their value to junior educators and faculty developers.

Conclusion

This is a key reading list for junior faculty members and faculty developers interested in teaching with technology. The commentary contextualizes the importance of these papers for medical educators, to optimize use of technology in their teaching or incorporate into faculty development.

Structured debriefing: What difference does it make?

Debriefing in simulation is a cornerstone of learning. However, in-depth studies examining simulation debriefing are scarce. This study explored four key debriefing attributes-feedback, reflection, knowledge development, and psychological safety-prior and subsequent to the implementation of a new pedagogical intervention in a pre-clinical scenario simulation course. The scenarios focused on patients with deteriorating conditions and took place at bachelor's nursing degree level. The new intervention for the debriefing sessions contained a detailed observation tool describing specific, correct nursing actions for deteriorating patients; video playback watched only by students acting as nurses, and debriefing organized into two sections. The study design was explorative. To generate data, 12 debriefing sessions were audio and video recorded in 2013 and 11 in 2014. Two student groups participated each year, comprising 16 and 10 students, respectively. Qualitative analysis was performed to examine the transcribed audio and video recordings. Relative to the 2013 cohort, the reflections of observers and the students acting as nurses were more assertive, and students' feedback was more specific and comprehensive in the 2014 cohort. Conducting in-depth studies examining debriefing is important to increase knowledge regarding the impact of pedagogical underpinnings on debriefing content and processes.

Reporting Guidelines for Health Care Simulation Research: Extensions to the CONSORT and STROBE Statements

INTRODUCTION

Simulation-based research (SBR) is rapidly expanding but the quality of reporting needs improvement. For a reader to critically assess a study, the elements of the study need to be clearly reported. Our objective was to develop reporting guidelines for SBR by creating extensions to the Consolidated Standards of Reporting Trials (CONSORT) and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statements.

METHODS

An iterative multistep consensus-building process was used on the basis of the recommended steps for developing reporting guidelines. The consensus process involved the following: (1) developing a steering committee, (2) defining the scope of the reporting guidelines, (3) identifying a consensus panel, (4) generating a list of items for discussion via online premeeting survey, (5) conducting a consensus meeting, and (6) drafting reporting guidelines with an explanation and elaboration document.

RESULTS

The following 11 extensions were recommended for

CONSORT

item 1 (title/abstract), item 2 (background), item 5 (interventions), item 6 (outcomes), item 11 (blinding), item 12 (statistical methods), item 15 (baseline data), item 17 (outcomes/estimation), item 20 (limitations), item 21 (generalizability), and item 25 (funding). The following 10 extensions were recommended for STROBE: item 1 (title/abstract), item 2 (background/rationale), item 7 (variables), item 8 (data sources/measurement), item 12 (statistical methods), item 14 (descriptive data), item 16 (main results), item 19 (limitations), item 21 (generalizability), and item 22 (funding). An elaboration document was created to provide examples and explanation for each extension.

CONCLUSIONS

We have developed extensions for the CONSORT and STROBE Statements that can help improve the quality of reporting for SBR.

Preparation With Web-Based Observational Practice Improves Efficiency of Simulation-Based Mastery Learning

INTRODUCTION

Our current understanding of what results in effective simulation-based training is restricted to the physical practice and debriefing stages, with little attention paid to the earliest stage: how learners are prepared for these experiences. This study explored the utility of Web-based observational practice (OP) -featuring combinations of reading materials (RMs), OP, and collaboration- to prepare novice medical students for a simulation-based mastery learning (SBML) workshop in central venous catheterization.

METHODS

Thirty medical students were randomized into the following 3 groups differing in their preparatory materials for a SBML workshop in central venous catheterization: a control group with RMs only, a group with Web-based groups including individual OP, and collaborative OP (COP) groups in addition to RM. Preparation occurred 1 week before the SBML workshop, followed by a retention test 1-week afterward. The impact on the learning efficiency was measured by time to completion (TTC) of the SBML workshop. Web site preparation behavior data were also collected.

RESULTS

Web-based groups demonstrated significantly lower TTC when compared with the RM group, (P = 0.038, d = 0.74). Although no differences were found between any group performances at retention, the COP group spent significantly more time and produced more elaborate answers, than the OP group on an OP activity during preparation.

DISCUSSION

When preparing for SBML, Web-based OP is superior to reading materials alone; however, COP may be an important motivational factor to increase learner engagement with instructional materials. Taken together, Web-based preparation and, specifically, OP may be an important consideration in optimizing simulation instructional design.

Multidimensional Evaluation of Simulation-Based Course to Enhance Prelicensure Nursing Students' Clinical Skills

This article describes the development of a new simulation-based course designed to remediate the clinical skills of prelicensure nursing students and the multidimensional method used to evaluate this new course. Evaluation of the course was guided by Kirkpatrick levels of training. The Quint Leveled Clinical Competency Tool was used to evaluate students' clinical competencies. Success of the course was found in 3 levels of evaluation.

Shame in Medical Education: A Randomized Study of the Acquisition of Intimate Examination Skills and Its Effect on Subsequent Performance

THEORY

Although medical students are exposed to a variety of emotions, the impact of emotions on learning has received little attention so far. Shame-provoking intimate examinations are among the most memorable events for students. Their emotions, however, are rarely addressed during training, potentially leading to withdrawal and avoidance and, consequently, performance deficits. However, emotions of negative valance such as shame may be particularly valuable for learning, as they might prompt mental rehearsal. We investigated the effect of shame on learning from the perspective of cognitive load theory.

HYPOTHESES

We hypothesized that (a) training modality determines state shame, (b) state shame directly affects the quality of a clinical breast examination as one example of a shame-provoking exam, and (c) students who experience shame during training outperform those who just discuss the emotion during subsequent performance assessments.

METHOD

Forty-nine advanced medical students participated in a randomized controlled, single-blinded study. After a basic, low-fidelity breast examination training, students were randomized to further practice either on a high-fidelity mannequin including a discussion of their emotions or by examining a standardized patient's real breasts. Last, all students conducted a breast examination in a simulated doctor's office. Dependent variables were measures of outcome and process quality and of situational shame.

RESULTS

Students training with a standardized patient experienced more shame during training (p < .001, d = 2.19), spent more time with the patient (p = .005, d = 0.89), and documented more breast lumps (p = .026, d = 0.65) than those training on a mannequin. Shame interacted with training modality, F(1, 45) = 21.484, p < .001, η² = 0.323, and differences in performance positively correlated to decline in state shame (r = .335, p = .022).

CONCLUSIONS

Students experiencing state shame during training do reenact their training and process germane load-in other words, learn. Furthermore, altering simulation modality offers a possibility for educators to adjust the affective component of training to their objectives.

Serious games for health: three steps forwards

Serious games are educational tools which are more and more used in patient and health professional education. In this article, we discuss three main points that developers and educators need to address during the development of a serious game for health. We first explain how to develop motivating serious games by finding a point where the intrinsic and extrinsic motivations of end users can converge. Then, we propose to identify the features of serious games which enhance their learning effectiveness on the basis of a framework derived from cognitive science and called "the four pillars of learning." Finally, we discuss issues and solutions related to the evaluation of serious games.

The practice of simulation-based assessment in respiratory therapy education

Clinical simulation has gained prominence as an educational approach in many Canadian respiratory therapy programs and is strongly associated with improved learning, clinical and nonclinical skill, future performance, and patient outcomes. Traditionally, the primary assessment approach employed in clinical simulation has been formative debriefing for learning. Contextual factors, such as limited opportunities for learning in clinical practice and technologically oriented perspectives on learning in clinical simulation, are converging to prompt a move from using formative debriefing sessions that support learning in simulation to employing high-stakes testing intended to measure entry-to-practice competencies. We adopt the perspective that these factors are intricately linked to the profession's regulatory environment, which may strongly influence how simulation practices become embedded with respiratory therapy educational programs. Through this discussion we challenge the profession to consider how environmental factors, including externally derived requirements, may ultimately impact the effectiveness of simulation-based learning environments.

The simulated clinical environment: Cognitive and emotional impact among undergraduates

CONTEXT

Simulated clinical immersion (SCI) is used in undergraduate healthcare programs to expose the learner to real-life situations in authentic simulated clinical environments. For novices, the environment in which the simulation occurs can be distracting and stressful, hence potentially compromising learning.

OBJECTIVES

This study aims to determine whether SCI (with environment) imposes greater extraneous cognitive load and stress on undergraduate pharmacy students than simulated patients (SP) (without environment). It also aims to explore how features of the simulated environment influence students' perception of learning.

METHODS

In this mixed-methods study, 143 undergraduate pharmacy students experienced both SCI and SP in a crossover design. After the simulations, participants rated their cognitive load and emotions. Thirty-five students met in focus groups to explore their perception of learning in simulation.

RESULTS

Intrinsic and extraneous cognitive load and stress scores in SCI were significantly but modestly higher compared to SP. Qualitative findings reveal that the physical environment in SCI generated more stress and affected students? focus. In SP, students concentrated on clinical reasoning. SCI stimulated a focus on data collection but impeded in-depth problem solving processes.

CONCLUSION

The physical environment in simulation influences what and how students learn. SCI was reported as more cognitively demanding than SP. Our findings emphasize the need for the development of adapted instructional design guidelines in simulation for novices.

Design of simulation-based medical education and advantages and disadvantages of in situ simulation versus off-site simulation

BACKGROUND

Simulation-based medical education (SBME) has traditionally been conducted as off-site simulation in simulation centres. Some hospital departments also provide off-site simulation using in-house training room(s) set up for simulation away from the clinical setting, and these activities are called in-house training. In-house training facilities can be part of hospital departments and resemble to some extent simulation centres but often have less technical equipment. In situ simulation, introduced over the past decade, mainly comprises of team-based activities and occurs in patient care units with healthcare professionals in their own working environment. Thus, this intentional blend of simulation and real working environments means that in situ simulation brings simulation to the real working environment and provides training where people work. In situ simulation can be either announced or unannounced, the latter also known as a drill. This article presents and discusses the design of SBME and the advantage and disadvantage of the different simulation settings, such as training in simulation-centres, in-house simulations in hospital departments, announced or unannounced in situ simulations.

DISCUSSION

Non-randomised studies argue that in situ simulation is more effective for educational purposes than other types of simulation settings. Conversely, the few comparison studies that exist, either randomised or retrospective, show that choice of setting does not seem to influence individual or team learning. However, hospital department-based simulations, such as in-house simulation and in situ simulation, lead to a gain in organisational learning. To our knowledge no studies have compared announced and unannounced in situ simulation. The literature suggests some improved organisational learning from unannounced in situ simulation; however, unannounced in situ simulation was also found to be challenging to plan and conduct, and more stressful among participants. The importance of setting, context and fidelity are discussed. Based on the current limited research we suggest that choice of setting for simulations does not seem to influence individual and team learning. Department-based local simulation, such as simulation in-house and especially in situ simulation, leads to gains in organisational learning. The overall objectives of simulation-based education and factors such as feasibility can help determine choice of simulation setting.

Rapid Development and Deployment of Ebola Readiness Training Across an Academic Health System: The Critical Role of Simulation Education, Consulting, and Systems Integration

STATEMENT

In this article, we describe an Ebola preparedness initiative with implementation across an academic health system. Key stakeholder centers of various disciplines and clinical experts collaborated in the development and design. Subject matter experts in the areas of Centers for Disease Control and Prevention and World Health Organization protocols for personal protective equipment donning and doffing conducted initial train-the-trainer sessions for program instructors. These trainers represented a cross-section of key clinical responders and environmental services. Through a parallel development process, a blended learning curriculum consisting of online modules followed by on-site training sessions was developed and implemented in both the simulation laboratory and the actual clinical care spaces in preparation for a Department of Health inspection. Lessons learned included identification of the need for iterative refinement based on instructor and trainee feedback, the lack of tolerance of practitioners in wearing full-body personal protective equipment for extended periods, and the ability of a large system to mount a rapid response to a potential public health threat through leveraging of expertise of its Simulation Program, Center for Quality, Safety and Innovation as well as a wide variety of clinical departments.

The use of instructional design guidelines to increase effectiveness of postpartum hemorrhage simulation training

OBJECTIVE

To compare learning outcomes of postpartum hemorrhage simulation training based on either instructional design guidelines or best practice.

METHODS

A pretest-post-test non-equivalent groups study was conducted among obstetrics and gynecology residents in Recife, Brazil, from June 8 to August 30, 2013. The instructional design group included 13 teams, whereas the best practice group included seven teams. A standardized task checklist was used for scenario analysis and the proportion of correctly executed tasks compared (post-test minus pretest).

RESULTS

The instructional design group scored higher than the best practice group for total number of tasks completed (median difference 0.46 vs 0.17; P<0.001; effect size [r]=0.72). Similar results were observed for communication (median difference 0.56 vs 0.22; P=0.004; r=0.58), laboratory evaluation (median difference 0.83 vs 0.00; P<0.001; r=0.76), and mechanical management (median difference 0.25 vs -0.15; P=0.048; r=0.39). Speed of learning was also increased. The median differences were 0.20 for the instructional design group compared with 0.05 for the best practice group at 60 seconds (P=0.015; r=0.49), and 0.49 versus 0.26 (P=0.001; r=0.65) at 360 seconds.

CONCLUSION

The use of simulation training for postpartum hemorrhage that was based on instructional design guidelines yielded better learning outcomes than did training based on best practice.

Use of personal protective equipment among health care personnel: Results of clinical observations and simulations

BACKGROUND

Very little is known about how health care personnel (HCP) actually use personal protective equipment (PPE).

METHODS

The clinical PPE practices of 50 HCP from selected units at the University of Pittsburgh Medical Center (UPMC) Presbyterian Hospital were videotaped with HCP consent. For 2 PPE simulation sessions (simple and full-body sets), 82 HCP were recruited throughout the UPMC system. Simulation practices were videotaped and examined using fluorescent powder with ultraviolet lighting. All participants completed an electronic survey. For a follow-up evaluation simulation, 12 HCP were recruited among simulation participants.

RESULTS

Among 130 total sessions from 65 participants, contamination occurred in 79.2% of simulations during the doffing process with various PPE items: simple set (92.3%) and full-body set (66.2%). Among 11 follow-up evaluation participants, contaminations still occurred in 82% after receiving individual feedback, but the overall contamination level was reduced. Using the contamination information gained during the simulation analysis, 66% of potential contamination was estimated for the clinical observation. Concerns and barriers in PPE use from HCP survey responses were as follows: time-consuming, cumbersomeness, and PPE effectiveness.

CONCLUSIONS

Although HCP knew they were being videotaped, contamination occurred in 79.2% of the PPE simulations. Devising better standardized PPE protocols and implementing innovative PPE education are necessary to ensure HCP safety.

A Systematic Review of Serious Games in Training Health Care Professionals

STATEMENT

Serious games are computer-based games designed for training purposes. They are poised to expand their role in medical education. This systematic review, conducted in accordance with PRISMA guidelines, aimed to synthesize current serious gaming trends in health care training, especially those pertaining to developmental methodologies and game evaluation. PubMed, EMBASE, and Cochrane databases were queried for relevant documents published through December 2014. Of the 3737 publications identified, 48 of them, covering 42 serious games, were included. From 2007 to 2014, they demonstrate a growth from 2 games and 2 genres to 42 games and 8 genres. Overall, study design was heterogeneous and methodological quality by MERQSI score averaged 10.5/18, which is modest. Seventy-nine percent of serious games were evaluated for training outcomes. As the number of serious games for health care training continues to grow, having schemas that organize how educators approach their development and evaluation is essential for their success.

Laparoscopic skill assessment of practicing surgeons prior to enrollment in a surgical trial of a new laparoscopic procedure

BACKGROUND

Outcomes of surgical trials hinge on surgeon selection and their underlying expertise. Assessment of expertise is paramount. We investigated whether surgeons' performance measured by the fundamentals of laparoscopic surgery (FLS) assessment program could predict their performance in a surgical trial.

METHODS

As part of a prospective multi-institutional study of minimally invasive inguinal lymphadenectomy (MILND) for melanoma, surgical oncologists with no prior MILND experience underwent pre-trial FLS assessment. Surgeons completed MILND training, began enrolling patients, and submitted videos of each MILND case performed. Videos were scored with the global operative assessment of laparoscopic skills (GOALS) tool. Associations between baseline FLS scores and participant's trial performance metrics were assessed.

RESULTS

Twelve surgeons enrolled patients; their median total baseline FLS score was 332 (range 275-380, max possible 500, passing >270). Participants enrolled 87 patients in the study (median 6 per surgeon, range 1-24), of which 72 (83%) videos were adequate for scoring. Baseline GOALS score was 17.1 (range 9.6-21.2, max possible score 30). Inter-rater reliability was excellent (ICC = 0.85). FLS scores correlated with improved GOALS scores (r = 0.57, p = 0.05) and with decreased operative time (r = -0.6, p = 0.02). No associations were found with the degree of patient recruitment (r = 0.02, p = 0.7), lymph node count (r = 0.01, p = 0.07), conversion rate (r = -0.06, p = 0.38) or major complications(r = -0.14, p = 0.6).

CONCLUSIONS

FLS skill assessment of surgeons prior to their enrollment in a surgical trial is feasible. Although better FLS scores predicted improved operative performance and operative time, other trial outcome measures showed no difference. Our findings have implications for the documentation of laparoscopic expertise of surgeons in practice and may allow more appropriate selection of surgeons to participate in clinical trials.

Development and Testing of a Short Peripheral Intravenous Catheter Insertion Skills Checklist

To date, there is no published, psychometrically validated, short peripheral intravenous catheter (PIVC) insertion skills checklist. Creating a valid, reliable, and generalizable checklist to measure PIVC skill is a key step in assessing baseline competence and skill mastery. Based on recognized standards and best practices, the PIVC Insertion Skills Checklist was developed to measure all the steps necessary for a best practice PIVC insertion. This includes the entire process from reading the prescriber's orders to documentation and, if the first attempt is unsuccessful, a second attempt option. Content validity was established using 3 infusion therapy experts. Evidence in support of response process validity is described. The PIVC Insertion Skills Checklist was used by 8 trained raters to assess the PIVC insertion skills, in a simulated environment, of 63 practicing clinicians working on medical and surgical units in a US teaching hospital. Internal consistency of the PIVC Insertion Skills Checklist was α = 0.84. Individual item intraclass correlation coefficients (ICCs) between rater and gold standard observations ranged from − 0.01 to 1.00 and total score ICC was 0.99 (95% confidence interval, 0.99–0.99). The current study offers validity and reliability evidence to support the use of the PIVC Insertion Skills Checklist to measure PIVC insertion skill of clinicians in a simulated environment.