Unanticipated teaching demands rise with simulation training: strategies for managing faculty workload

Improving Medical Student Performance With Unsupervised Simulation and Remote Asynchronous Feedback

OBJECTIVE

This study aims to assess the effectiveness of training medical students to perform two clinical procedures using unsupervised simulation with remote asynchronous feedback, compared to an intensive workshop with in-person feedback.

DESIGN, SETTING, AND PARTICIPANTS

Third-year medical students were recruited and randomized into 2 groups: Thoracentesis or paracentesis. Within each group, participants were further randomized into either unsupervised simulation with remote asynchronous feedback (experimental group; EG) or a 2-hour workshop (control group; CG). The EG underwent two unsupervised 20-minute training sessions and received remote asynchronous feedback. The CG had a 2-hour workshop where they received in-person feedback. After training, students were assessed using the objective structured assessment of technical skills (OSATS) scale. Twenty students in thoracentesis and 23 in paracentesis training completed the 2 training sessions with remote and asynchronous feedback, and 30 students for both thoracentesis and paracentesis groups completed the 2-hour workshop.

RESULTS

The EG achieved a significantly higher passing rate than the CG on both procedures (thoracentesis 80% vs. 43%, paracentesis 91% vs. 67%, p-value< 0.05).

CONCLUSION

The asynchronous educational method allowed EG students to achieve higher performance than CG students. This novel modality allowed students and instructors to train and assess at their own pace.

Comparing Peer and Faculty Feedback for Asynchronous Laparoscopic Skill Acquisition

OBJECTIVE

Traditionally, expert surgeons have provided surgical trainees with feedback about their simulation performance, including for asynchronous practice. Unfortunately, innumerable time demands may limit experts' ability to provide feedback. It is unknown whether and how peer feedback is an effective mechanism to help residents acquire laparoscopic skill in an asynchronous setting. As such, we aimed to assess the effect of peer feedback on laparoscopic performance and determine how residents perceive giving and receiving peer feedback.

DESIGN

We conducted a convergent mixed methods study. In the quantitative component, we randomized residents to receive feedback on home laparoscopic tasks from peers or faculty. We then held an end-of-curriculum, in-person laparoscopic assessment with members from both groups and compared performance on the in-person assessment between the groups. In the qualitative component, we conducted interviews with resident participants to explore experiences with feedback and performance. Three authors coded and rigorously reviewed interview data using a directed content analysis.

SETTING

We performed this study at a single tertiary academic institution: the University of California, San Francisco.

PARTICIPANTS

We invited 47 junior residents in general surgery, obstetrics-gynecology, and urology to participate, of whom 37 (79%) participated in the home curriculum and 25 (53%) participated in the end-of-curriculum assessment.

RESULTS

Residents in the peer feedback group scored similarly on the final assessment (mean 70.7%; SD 16.1%) as residents in the faculty feedback group (mean 71.8%; SD 11.9%) (p = 0.86). Through qualitative analysis of interviews with 13 residents, we identified key reasons for peer feedback's efficacy: shared mental models, the ability to brainstorm and appreciate new approaches, and a low-stakes learning environment.

CONCLUSIONS

We found that peer and faculty feedback led to similar performance in basic laparoscopy and that residents engaged positively with peer feedback, suggesting that peer feedback can be used when residents learn basic laparoscopy.

Near-peer teaching in simulation

BACKGROUND

Development, implementation and evaluation of a simulation curriculum is time and resource intensive. Limited faculty time and training are cited as primary barriers to adopting simulation into medical education. Near-peer teaching is a potential solution to manage the increased teaching demands that occur with simulation use.

APPROACH

In 2022, we implemented a near-peer simulation curriculum for teaching junior physical medicine and rehabilitation (PM&R) residents high-acuity low-opportunity events commonly seen on the inpatient rehabilitation unit. The curriculum was taught by senior residents to supplement faculty lectures. Senior residents completed facilitator training on simulator logistics, debriefing and formative assessment.

EVALUATION

Residents completed an end-of-course questionnaire evaluating teaching effectiveness and perceived knowledge acquisition. All items were scored on a 5-point Likert-type scale. Learners rated their near-peers as having good clinical teaching effectiveness (mean [SD], 4.66[0.38]). Senior residents (n = 6) disclosed feeling knowledgeable about the topics they instructed (baseline 3.9[3.2-4.4]; after 4.6[4.1-4.9]; p = 0.19), and junior residents (n = 6) felt they gained knowledge and improved their ability to manage patients as a result of the near-peer curriculum (baseline 2.4[2.3-2.5]; after 3.9[3.5-4.2]; p = 0.005).

IMPLICATIONS

This educational programme is an example of how near-peer teaching can be used in simulation. Our simulation curriculum taught by near-peers was valued by learners as well taught and educational. Research is needed that directly compares the effectiveness of near-pear teaching to faculty instruction. We hope that by sharing our work, educators will feel inspired to use near-peer teachers for simulation instruction when faculty availability for teaching is scarce.

Lasater clinical judgment rubric reliability for scoring clinical judgment after observing asynchronous simulation and feasibility/usability with learners

BACKGROUND

There is strong evidence supporting using the Lasater Clinical Judgment Rubric (LCJR) for scoring learners' clinical judgment during in-person simulation performance and clinical experience reflections. However, a gap exists for using LCJR to evaluate clinical judgment after observing asynchronous simulation.

OBJECTIVE

We aimed to determine the reliability, feasibility, and usability of LCJR for scoring learners' written reflections after observing expert-modeled asynchronous simulation videos.

DESIGN/SETTING/PARTICIPANTS

We used a one-group, descriptive design and sampled pre-licensure, junior-level bachelor's learners from the Southwestern United States.

METHODS

Participants observed eight expert-modeled asynchronous simulation videos over one semester and provided written responses to clinical judgment prompts. We scored clinical judgment using LCJR. We studied reliability by measuring internal consistency of 11 clinical judgment prompts and interrater reliability with two raters. This study also investigated feasibility and usability of the asynchronous simulation learning activity using descriptive statistics. Feasibility included time learners spent completing written responses and time raters spent evaluating written responses. Learners reported usability perceptions using an instructor-developed survey.

RESULTS

Sixty-three learners completed 504 written responses to clinical judgment prompts. Cohen's kappa ranged from 0.34 to 0.86 with a cumulative κ = 0.58. Gwet's AC ranged from 0.48 to 0.90, with a cumulative AC = 0.74. Cronbach's alpha was from 0.51 to 0.72. Learners spent on average 28.32 ± 12.99 min per expert-modeling video observation. Raters spent on average 4.85 ± 1.34 min evaluating written responses for each participant. Learners reported the asynchronous learning activity was usable.

CONCLUSIONS

Nurse educators can reliably use LCJR for scoring learners' clinical judgment after observing asynchronous expert-modeled simulation. Logistically, learners complete the reflective learning activity and faculty use LCJR to measure clinical judgment in feasible time. Further, participants perceived the asynchronous learning activity usable. Nurse educators should utilize this learning activity for evaluating and tracking observer clinical judgment development.

Taking advantage of asynchronous digital feedback: development of an at-home basic suture skills training program for undergraduate medical students that facilitates skills retention

Purpose

To date, there are no training programs for basic suturing that allow remote deliberate practice. This study seeks to evaluate the effectiveness of a basic suture skills training program and its 6-month skill retention applying unsupervised practice and remote digital feedback.

Methods

Fourth-year medical-student trainees reviewed instructional videos from a digital platform and performed unsupervised practice as needed at their homes. When they felt competent, trainees uploaded a video of themselves practicing the skill. In < 72 h, they received expert asynchronous digital feedback. The course had two theoretical stages and five video-based assessments, where trainees performed different suturing exercises. For the assessment, a global (GRS) and specific rating scale (SRS) were used, with a passing score of 20 points (max:25) and 15 (max:20), respectively. Results were compared to previously published work with in-person expert feedback (EF) and video-guided learning without feedback (VGL). A subgroup of trainees underwent a 6-month skills retention assessment.

Results

Two-hundred and forty-three trainees underwent the course between March and December 2021. A median GRS of 24 points was achieved in the final assessment, showing significantly higher scores (p < 0.001) than EF and VGL (20.5 and 15.5, respectively). Thirty-seven trainees underwent a 6-month skills retention assessment, improving in GRS (23.38 vs 24.03, p value = 0.06) and SRS (18.59 vs 19, p value = 0.07).

Conclusion

It is feasible to teach basic suture skills to undergraduate medical students using an unsupervised training course with remote and asynchronous feedback through a digital platform. This methodology allows continuous training with the repetition of quality practice, personalized feedback, and skills retention at 6 months.

Use of surgical video-based automated performance metrics to predict blood loss and success of simulated vascular injury control in neurosurgery: a pilot study

OBJECTIVE

Experts can assess surgeon skill using surgical video, but a limited number of expert surgeons are available. Automated performance metrics (APMs) are a promising alternative but have not been created from operative videos in neurosurgery to date. The authors aimed to evaluate whether video-based APMs can predict task success and blood loss during endonasal endoscopic surgery in a validated cadaveric simulator of vascular injury of the internal carotid artery.

METHODS

Videos of cadaveric simulation trials by 73 neurosurgeons and otorhinolaryngologists were analyzed and manually annotated with bounding boxes to identify the surgical instruments in the frame. APMs in five domains were defined-instrument usage, time-to-phase, instrument disappearance, instrument movement, and instrument interactions-on the basis of expert analysis and task-specific surgical progressions. Bounding-box data of instrument position were then used to generate APMs for each trial. Multivariate linear regression was used to test for the associations between APMs and blood loss and task success (hemorrhage control in less than 5 minutes). The APMs of 93 successful trials were compared with the APMs of 49 unsuccessful trials.

RESULTS

In total, 29,151 frames of surgical video were annotated. Successful simulation trials had superior APMs in each domain, including proportionately more time spent with the key instruments in view (p < 0.001) and less time without hemorrhage control (p = 0.002). APMs in all domains improved in subsequent trials after the participants received personalized expert instruction. Attending surgeons had superior instrument usage, time-to-phase, and instrument disappearance metrics compared with resident surgeons (p < 0.01). APMs predicted surgeon performance better than surgeon training level or prior experience. A regression model that included APMs predicted blood loss with an R2 value of 0.87 (p < 0.001).

CONCLUSIONS

Video-based APMs were superior predictors of simulation trial success and blood loss than surgeon characteristics such as case volume and attending status. Surgeon educators can use APMs to assess competency, quantify performance, and provide actionable, structured feedback in order to improve patient outcomes. Validation of APMs provides a benchmark for further development of fully automated video assessment pipelines that utilize machine learning and computer vision.

Faculty support bundle for simulation education

BACKGROUND

Postgraduate training programmes rely on faculty to meet core educational needs, including simulation. Time is arguably the most valuable resource for academic physicians, which presents a challenge for recruiting faculty to provide extra-clinical teaching. To increase faculty engagement in simulation-based education (SBE), we first identified barriers to participation. Next, we sought to overcome barriers using a self-determination theory (SDT) framework to increase motivation using strategies that addressed faculty autonomy, competence and relatedness.

METHODS

Faculty from a single department of emergency medicine were surveyed about factors influencing participation in SBE. Responses were grouped into themes and used to develop the intervention-a faculty support bundle-to overcome common barriers and promote participation. Supports focused on course materials, organisational consistency and peer recognition. Faculty participation in SBE pre- and post-implementation of the support bundle was analysed via chi-squared analysis. Faculty who delivered SBE were resurveyed after the implementation phase to explore how the support bundle affected their experience.

RESULTS

Initial survey response was 41%. Reported barriers to participation in SBE included scheduling issues, preparation time, competing responsibilities, lack of confidence with simulation and lack of interest. Twenty-four faculty participated in SBE during the pre-implementation phase, compared to 39 post implementation (p = 0.03).

DISCUSSION

The faculty support bundle increases faculty participation in SBE. Strategies focused on internal motivators identified using an SDT framework. In contrast to traditional external motivators, these were no cost interventions. Those seeking to increase faculty participation in SBE should consider implementing similar strategies.

Massive open online course (MOOC) learning builds capacity and improves competence for patient safety among global learners: A prospective cohort study

BACKGROUND

Education about patient safety has historically been limited in health professionals curricula and largely inaccessible to the general public. We developed a massive open online course (MOOC), The Science of Safety in Healthcare, to present the foundational patient safety principles to a broad global audience of health professionals, learners, and patients interested in patient safety.

OBJECTIVES

To describe the Science of Safety in Healthcare MOOC, its effects on patient safety competence, and the satisfaction of course participants.

METHODS

The five-week video-based course was delivered in 2013 and 2014, and was organized in five modules: 1) overview of science of safety and safety culture, 2) enabling and contextual factors that influence patient safety and quality, 3) methods to improve safety and quality, 4) translating evidence intro practice and leading change, and 5) summary and opportunities for capacity building. Each module had three or four segments. Participants were introduced to key concepts, and tools and skills to promote patient safety. Participants completed the Health Professional Education in Patient Safety Survey (H-PEPSS), which measures patient safety competence, and a course satisfaction survey. Pre- and post- course H-PEPSS scores were compared using paired t-tests. Course satisfaction surveys were administered at the completion of the course and six months later.

RESULTS

A total of 20,957, and 9679 participants enrolled in the course in 2013 and 2014, respectively. About half of participants were 25-44 years old (57%), and female (54%). Participants joined from over 100 countries. The majority were health professionals (61%) or health professionals in training (7%). Mean H-PEPSS scores improved after course completion, with significant increases on all survey domains in both years (p < 0.01). Mean score differences were: Teamwork 0.68 (95% CI: 0.64, 0.71), Communication 0.70 (95% CI: 0.66, 0.73), Managing Risk 0.79 (95% CI: 0.76, 0.82), Human Environment 0.64 (95% CI: 0.61, 0.68), Recognizing and Responding 0.64 (95% CI: 0.61, 0.68), and Culture 0.72 (95% CI: 0.68, 0.75). About 8% of participants in each cohort earned a certificate of completion. At 6-months post-course, the majority of the participants agreed or agreed strongly that the course content was useful (93%) and that the course positively influenced their clinical practice (69%) and communication (84%).

CONCLUSIONS

The MOOC course allowed educators to reach a large, diverse audience. The course was well-received and participants reported a significant increase in patient safety competence. As with most MOOCs, rates of completion were low.

Development of a Novel and Scalable Simulation-Based Teamwork Training Model Using Within-Group Debriefing of Observed Video Simulation

THE CHALLENGE

Effective teamwork and communication skills are essential for safe and reliable health care. These skills require training and practice. Experiential learning is optimal for training adults, and the industry has recognized simulation training as an exemplar of this approach. Yet despite decades of investment, this training is inaccessible and underutilized for most of the more than 12 million health care professionals in the United States.

DESIGNING A SOLUTION

This report describes the design process of an adapted simulation training created to overcome the key barriers to scaling simulation-based teamwork training: access to technology, time away from clinical work, and availability of trained simulation educators. The prototype training is designed for delivery in one-hour segments and relies on observation of video simulation scenarios and within-group debriefing, which are promising variations on traditional simulation training. To our knowledge, these two simulation approaches have not been previously combined. The resulting prototype minimizes the need for an on-site trained simulation educator. This report details the development of a training model, its subsequent modification based on pilot testing, and the evaluation of the resulting redesigned prototype.

PRELIMINARY EVALUATION

Participant evaluations of the redesigned prototype were highly positive, with 92% reporting that they would like to participate in additional, similar training sessions. Positive results were also found in assessment of feasibility, acceptability, psychological safety, and behavioral intention (reported intention to alter behavior).

Simulation-based curriculum development: lessons learnt in Global Health education

BACKGROUND

Simulation based medical education (SBME) allows learners to acquire clinical skills without exposing patients to unnecessary risk. This is especially applicable to Emergency Medicine training programs where residents are expected to demonstrate proficiency in the management of time critical, low frequency, and highly-morbidity conditions. This study aims to describe the process through which a SBME curriculum was created, in a limited simulation resource setting at a 4-year Emergency Medicine (EM) residency program at the American University of Beirut Medical Center.

METHODS

A case-based pilot simulation curriculum was developed following Kern's 6 step approach to curriculum design. The curricular objectives were identified through an anonymous survey of the program's residents and faculty. Curriculum outcomes were assessed, and the curriculum was revised to address curricular barriers. Evaluations of the revised curriculum were collected during the simulation sessions and through a whole revised curriculum evaluation at the end of the first year of its implementation.

RESULTS

14/20 residents (70%) and 8/8 faculty (100%) completed the needs assessment from which objectives for the pilot curriculum were developed and implemented through 6 2-h sessions over a 1-year period. Objectives were not met and identified barriers included cost, scheduling, resources, and limited faculty time. The revised curriculum addressed these barriers and 24 40-min sessions were successfully conducted during the following year. The sessions took place 3 at a time, in 2-h slots, using the same scenario to meet the objectives of the different learners' levels. 91/91 evaluations were collected from participants with overall positive results. The main differences between the pilot and the revised curricula included: a better understanding of the simulation center resources and faculty's capabilities.

CONCLUSION

Simulation-based education is feasible even with limited-resources. However, understanding the resources available, and advocating for protected educator time are essential to implementing a successful EM simulation curriculum.

Alternatives to High-Fidelity Simulation

Simulation-based education improves health care professionals' performance in managing critical events. Limitations to widespread uptake of high-fidelity simulation include barriers related to training, technology, and time. Alternatives to high-fidelity simulation that overcome these barriers include in situ simulation, classroom-based simulation, telesimulation, observed simulation, screen-based simulation, and game-based simulation. Some settings have limited access to onsite expert facilitation to design, implement, and guide participants through simulation-based education. Alternatives to onsite expert debriefing in these settings include teledebriefing, scripted debriefing, and within-group debriefing. A combination of these alternatives promotes successful implementation and maintenance of simulation-based education for managing critical health care events.

Learning outcomes of the observer role in nursing simulation: a scoping review protocol

OBJECTIVE

The purpose of this study is to map the literature investigating learning outcomes that have been measured in the observer role in simulation. A scoping review will be completed to investigate the research question: What learning outcomes have been reported in nurses or nursing students participating in simulation while assigned to the observer role?

INTRODUCTION

Simulation-based learning experiences (SBLE) are being utilized more frequently to support learning with increasing numbers of nurses. Despite SBLE becoming more popular, resources and manikin availability are not meeting the demand placed on simulation programs in nursing schools and health care facilities. Programs are placing more learners in observational roles without knowing what learning can be expected.

INCLUSION CRITERIA

Studies will be considered if they investigated learning outcomes of any nursing population (e.g. pre-licensure nursing student, graduate nursing student, practicing nurse) placed in an observational role in simulation. Any simulation topic related to nursing care will be considered. The observer role may involve observing one's own or expert behavior. The SBLE may occur in a simulation lab, classroom, community, health care facility, or online setting. Studies that include health care professionals other than nurses will be excluded. Experimental, quasi-experimental, observational, and qualitative studies will be considered.

METHODS

The JBI methodology for scoping reviews will be followed. A three-step search strategy will be used to identify potentially relevant articles. Two independent reviewers will screen titles and abstracts, assess full-text studies against the inclusion criteria, and extract data. Results will be presented in a manner that aligns with the objective of the review.

Changes to the ACGME Common Program Requirements and Their Potential Impact on Emergency Medicine Core Faculty Protected Time

The Accreditation Council for Graduate Medical Education (ACGME), which regulates residency and fellowship training in the United States, recently revised the minimum standards for all training programs. These standards are codified and published as the Common Program Requirements. Recent specific revisions, particularly removing the requirement ensuring protected time for core faculty, are poised to have a substantial impact on emergency medicine training programs. A group of representatives and relevant stakeholders from national emergency medicine (EM) organizations was convened to assess the potential effects of these changes on core faculty and the training of emergency physicians. We reviewed the literature and results of surveys conducted by EM organizations to examine the role of core faculty protected time. Faculty nonclinical activities contribute greatly to the academic missions of EM training programs. Protected time and reduced clinical hours allow core faculty to engage in education and research, which are two of the three core pillars of academic EM. Loss of core faculty protected time is expected to have detrimental impacts on training programs and on EM generally. We provide consensus recommendations regarding EM core faculty clinical work hour limitations to maintain protected time for educational activities and scholarship and preserve the quality of academic EM.

Physician Versus Nonphysician Instruction: Evaluating an Expert Curriculum-Competent Facilitator Model for Simulation-Based Central Venous Catheter Training

INTRODUCTION

Healthcare simulation supports educational opportunities while maintaining patient safety. To reduce costs and increase the availability of training, a randomized controlled study evaluated central venous catheter (CVC) insertion training in the simulation laboratory with nonphysician competent facilitators (NPCFs) as instructors.

METHOD

A group of learners naive to central line placement participated in a blended curriculum consisting of interactive online materials and simulation-based training. Learners were randomized to training with NPCFs or attending physician faculty. The primary outcome was simulated CVC insertion task performance, graded with a validated checklist by blinded physician reviewers. Learner knowledge and satisfaction were also evaluated. Analysis was conducted using noninferiority testing.

RESULTS

Eighty-five students, 11 attending physicians, and 7 NPCFs voluntarily participated. Noninferiority testing of the difference in CVC insertion performance between NPCF-trained learners versus physician-trained learners found no significant difference [rejecting the null hypothesis of inferiority using an 8% noninferiority margin (P < 0.01)]. In addition, there was no difference found between the 2 groups on pre/post knowledge scores, self-reported learner comfort, course satisfaction, or instructor satisfaction.

CONCLUSIONS

An introductory CVC curriculum can be taught to novice learners by carefully trained and supported NPCFs and achieve skill and knowledge outcomes similar to learners taught by physicians.

A video anchored rating scale leads to high inter-rater reliability of inexperienced and expert raters in the absence of rater training

BACKGROUND

Our objective was to assess the impact of incorporating videos in a behaviorally anchored performance rating scale on the inter-rater reliability (IRR) of expert, intermediate and novice raters.

METHODS

The Intra-corporeal Suturing Assessment Tool (ISAT) was modified to include short video clips demonstrating poor, average, and expert performances. Blinded raters used this tool to assess videos of trainees performing suturing on a porcine model. Three attending surgeons, 4 residents, and 4 novice raters participated; no rater training was provided. The IRR was then compared among rater groups.

RESULTS

The IRR using the modified ISAT was high at 0.80 (p < 0.001). Ratings were significantly correlated with trainee objective suturing scores for all rater groups (experts: R = 0.84, residents: R = 0.81, and novices: R = 0.69; p < 0.001).

CONCLUSIONS

Incorporating video anchors (to define performance) in the ISAT led to high IRR and enabled novices to achieve similar consistency in their ratings as experts.

Stakeholder perspectives and status of surgical simulation and skills training in Canadian urology residency program

INTRODUCTION

With the shift to competency-based training, surgical skills lab training (SSLT) may become a mandatory part of Canadian urology residency programs (CURPs). This study aims to identify: 1) the status of SSLT in CURP; 2) stakeholder perspectives on the utility of SSLT; 3) barriers to developing and implementing SSLT; and 4) how to address these barriers.

METHODS

Surveys were developed and issued to three groups of stakeholders: 1) SSLT directors at all 13 CURPs (response rate 100%); 2) teaching faculty (response rate 33%); and 3) urology residents (response rate 24%). Surveys 2 and 3 were sent to 10 English CURPs. Results were collected through email and SurveyMonkey^®.

RESULTS

Nine of 13 CURPs have a dedicated SSLT; 46% of CURP have 1-3 sessions per year, 8% have 5-7, and 30% >7. Most residents have independent lab access, but 80% do so less than once monthly. Over 90% of stakeholders find SSLT useful, of which high-fidelity models are most preferred (faculty rated 3.66/4, residents 3.18/4). Program directors (PDs) identified lack of protected faculty time, funding, and infrastructure as the top three barriers to SSLT implementation. Residents found lack of faculty time, protected academic time, and infrastructure as barriers. PDs viewed protecting faculty time and more funding as potential solutions, while residents suggested protected faculty and academic time, and after-hours lab access.

CONCLUSIONS

Residents, faculty, and PDs in CURPs view SSLT as useful. Most CURPs have defined SSLT; programs without this have labs for resident use but are underused. To continue to develop and progress SSLT, more time, participation, and funding must be made available.

Leveraging Peer Teaching for Global Health Elective Preparation: Implementation of a Resident-Led Global Health Simulation Curriculum

Introduction. As more trainees engage in global health electives, the call for best practices in predeparture preparation grows. However, many residency programs may not have the infrastructure or staffing in place to consistently meet these expectations. Objective. To assess the feasibility, benefits, and limitations of having residents trained to facilitate and implement a global health preparatory simulation curriculum. Approach. In 2016, we had 5 residents undergo online Simulation Use for Global Away Rotations (SUGAR) facilitator training. Since then, we have conducted 3 simulation sessions from 2016 to 2018. Feedback was obtained from 75% of our participants, which was found to be similar to feedback obtained from attending-led models. Lessons Learned. A resident-led global health preparatory curriculum provides a sustainable model for residency programs without placing additional burden on attending faculty.

Objectively Assessing Intraoperative Arthroscopic Skills Performance and the Transfer of Simulation Training in Knee Arthroscopy: A Randomized Controlled Trial

PURPOSE

To objectively investigate the transfer validity of simulation training using wireless elbow-worn motion sensors intraoperatively to assess whether surgical simulation leads to improvements in intraoperative arthroscopic performance.

METHODS

In this randomized controlled trial, postgraduate year 2 to 3 trainees in nationally approved orthopaedic surgery posts were randomized to standard junior residency training (control group) or standard training plus additional weekly simulation training (intervention group). Both groups performed a supervised real-life diagnostic knee arthroscopy in the operating room at 13 weeks. Performance was measured using wireless elbow-worn motion sensors recording objective surgical performance metrics: number of hand movements, smoothness, and time taken. A participant-supervisor performance ratio was used to adjust for variation in case mix and difficulty. The study took place in a surgical simulation suite and the orthopaedic operating rooms of a university teaching hospital.

RESULTS

The intervention group objectively outperformed the control group in all outcome metrics. Procedures performed by the intervention group required fewer hand movements (544 [interquartile range (IQR), 465-593] vs 893 [IQR, 747-1,242]; P < .001), had smoother movements (25,842 ms^-3 [IQR, 20,867-27,468 ms^-3] vs 36,846 ms^-3 [IQR, 29,840-53,949 ms^-3]; P < .001), and took less time (320 seconds [IQR, 294-392 seconds] vs 573 seconds [IQR, 477-860 seconds]; P < .001) than those performed by the control group. The cases were comparable between the groups. Standardized to the supervisor's performance, the intervention group required fewer hand movements (1.9 [IQR, 1.5-2.1] vs 3.3 [IQR, 2.2-4.8]; P = .0091), required less time (1.2 [IQR, 1.1-1.7] vs 2.6 [IQR, 1.6-3.0]; P = .0037), and were smoother (2.1 [IQR, 1.8-2.8] vs 4.3 [IQR, 2.8-5.4]; P = .0037) than the control group, but they did not perform as well as their supervisors.

CONCLUSIONS

This study uses intraoperative motion-analysis technology to objectively show that surgical simulation training improves actual intraoperative technical skills performance.

CLINICAL RELEVANCE

The described wireless objective assessment method complements the subjective observational performance assessments commonly used. Further studies are required to assess how these measures of intraoperative performance correlate to patient outcomes. Intraoperative motion analysis is translatable across surgical specialties, offering potential for objective assessment of progression through competency-based training, revalidation, and talent selection for specialist training.

Resident versus faculty member simulation debriefing

BACKGROUND

Near-peer teaching is effective in graduate medical education, but it has not been compared with faculty member teaching in resident simulation. In this study, we sought to compare debriefing sessions of internal medicine (IM) intern simulation sessions led by academic faculty doctors with those led by senior IM residents in order to measure the effectiveness of near-peer teaching in this setting. Near-peer teaching is effective in graduate medical education, but has not been compared with faculty member teaching in resident simulation METHOD: Internal medicine interns participated in four simulation cases, two of which were debriefed by faculty members and two of which were debriefed by residents. Pre-simulation knowledge assessment was completed prior to the case. Following each debriefing, interns completed a Debriefing Assessment for Simulation in Healthcare (DASH) survey. Post-simulation knowledge assessments were completed 6 months after simulation. Debriefings were recorded and transcribed. Each statement made during debriefing was classified as either correct or erroneous by blinded reviewers.

RESULTS

Fifty interns participated in simulation, and the response rate on the DASH survey was 88%. There was no difference between DASH scores (p = 0.13), post-simulation knowledge assessments or error rates during debriefing (p = 0.31) for faculty member and resident instructors.

CONCLUSION

Our study suggests that residents and faculty members provide a similar quality of simulation instruction based on qualitative and quantitative evaluation.

Simulation Clinical Scenario Design Workshop for Practicing Clinicians

INTRODUCTION

Faculty development is important to developing skilled faculty members who are able to effectively design and deliver educational content. There has been an increase in courses designed to help faculty better teach at the bedside, but fewer options for those interested in developing their skills as simulation-based educators. Our goal was to create a workshop to train prospective simulation educators on the skills and knowledge necessary to design a clinical scenario.

METHODS

Learners participate in a 90-minute workshop utilizing short, didactic teaching and practical hands-on practice. Faculty guide learners through the process of developing targeted goals and learning outcomes, setting the scene for a clinical scenario, and storyboarding the main action. Learners work individually, but engage in peer-to-peer feedback, as well as instructor feedback, throughout the session.

RESULTS

We have run four iterations of this workshop at our institution in the past year, and developed a modified version for an international meeting. A total of 51 learners from our hospital have completed the workshop. Overall, learners agree that the workshop is informative and increases their knowledge. For each educational objective, more than 70% of participants indicated the workshop increased their competence in the area either "extremely" or "quite a bit."

DISCUSSION

This workshop provides learners hand-on practice in developing a scenario for simulation-based education. Learners leave with the tools and knowledge necessary to take the work developed in the session and create a complete scenario that can be used at our simulation center or at another facility.

Foundational Elements of Applied Simulation Theory: Development and Implementation of a Longitudinal Simulation Educator Curriculum

Simulation-based education has gained popularity, yet many faculty members feel inadequately prepared to teach using this technique. Fellowship training in medical education exists, but there is little information regarding simulation or formal educational programs therein. In our institution, simulation fellowships were offered by individual clinical departments. We recognized the need for a formal curriculum in educational theory. Kern’s approach to curriculum development was used to develop, implement, and evaluate the Foundational Elements of Applied Simulation Theory (FEAST) curriculum.

Needs assessments resulted in a 26-topic curriculum; each biweekly session built upon the previous. Components essential to success included setting goals and objectives for each interactive session and having dedicated faculty, collaborative leadership and administrative support for the curriculum. Evaluation data was collated and analyzed annually via anonymous feedback surveys, focus groups, and retrospective pre-post self-assessment questionnaires.

Data collected from 32 fellows over five years of implementation showed that the curriculum improved knowledge, challenged thinking, and was excellent preparation for a career in simulation-based medical education. Themes arising from focus groups demonstrated that participants valued faculty expertise and the structure, practicality, and content of the curriculum.

We present a longitudinal simulation educator curriculum that adheres to a well-described framework of curriculum development. Program evaluation shows that FEAST has increased participant knowledge in key areas relevant to simulation-based education and that the curriculum has been successful in meeting the needs of novice simulation educators. Insights and practice points are offered for educators wishing to implement a similar curriculum in their institution.

Survey of outcomes in a faculty development program on simulation pedagogy

Although many nursing programs use simulation as a teaching-learning modality, there are few systematic approaches to help nursing educators learn this pedagogy. This study evaluates the effects of a simulation pedagogy nursing faculty development program on participants' learning perceptions using a retrospective pre-course and post-course design. Sixteen Thai participants completed a two-day nursing faculty development program on simulation pedagogy. Thirteen questionnaires were used in the final analysis. The participants' self-perceived learning about simulation teaching showed significant post-course improvement. On a five-point Likert scale, the composite mean attitude, subjective norm, and perceived behavioral control scores, as well as intention to use a simulator, showed a significant post-course increase. A faculty development program on simulation pedagogy induced favorable learning and attitudes. Further studies must test how faculty performance affects the cognitive, emotional, and social dimensions of learning in a simulation-based learning domain.