Individualized feedback during simulated laparoscopic training:a mixed methods study

Structured feedback and operative video debriefing with critical view of safety annotation in training of laparoscopic cholecystectomy: a randomized controlled study

BACKGROUND

The learning curve in minimally invasive surgery (MIS) is lengthened compared to open surgery. It has been reported that structured feedback and training in teams of two trainees improves MIS training and MIS performance. Annotation of surgical images and videos may prove beneficial for surgical training. This study investigated whether structured feedback and video debriefing, including annotation of critical view of safety (CVS), have beneficial learning effects in a predefined, multi-modal MIS training curriculum in teams of two trainees.

METHODS

This randomized-controlled single-center study included medical students without MIS experience (n = 80). The participants first completed a standardized and structured multi-modal MIS training curriculum. They were then randomly divided into two groups (n = 40 each), and four laparoscopic cholecystectomies (LCs) were performed on ex-vivo porcine livers each. Students in the intervention group received structured feedback after each LC, consisting of LC performance evaluations through tutor-trainee joint video debriefing and CVS video annotation. Performance was evaluated using global and LC-specific Objective Structured Assessments of Technical Skills (OSATS) and Global Operative Assessment of Laparoscopic Skills (GOALS) scores.

RESULTS

The participants in the intervention group had higher global and LC-specific OSATS as well as global and LC-specific GOALS scores than the participants in the control group (25.5 ± 7.3 vs. 23.4 ± 5.1, p = 0.003; 47.6 ± 12.9 vs. 36 ± 12.8, p < 0.001; 17.5 ± 4.4 vs. 16 ± 3.8, p < 0.001; 6.6 ± 2.3 vs. 5.9 ± 2.1, p = 0.005). The intervention group achieved CVS more often than the control group (1. LC: 20 vs. 10 participants, p = 0.037, 2. LC: 24 vs. 8, p = 0.001, 3. LC: 31 vs. 8, p < 0.001, 4. LC: 31 vs. 10, p < 0.001).

CONCLUSIONS

Structured feedback and video debriefing with CVS annotation improves CVS achievement and ex-vivo porcine LC training performance based on OSATS and GOALS scores.

"Self-Designed Simulation-Based Laparoscopic Training Program for Urology Residents: Results After 6 Years of Experience"

Introduction

Learning laparoscopy (LAP) is challenging and requires different skills to conventional open surgery. There is a recognized need for a standardized laparoscopic training framework within urology to overcome these difficulties and to shift learning curve from patient to skills laboratory. Simulation-based training has been widely commented, but implementation in real day practice is lacking. We present our "LAP training program for residents".

Material

Between 2017 and 2022, 11 residents participated in our self-designed program: Theoretical: (Moodle platform) basic knowledge and multimedia content for initiation into LAP. Evaluated through online exam. Practical: exercises for LAP skills acquisition were proposed and encouraged residents' practice in a box trainer available and experimental surgery sessions on a porcine model. On-site E-BLUS (European Basic Laparoscopic Urologic Skills) examination was performed annually. Feedback was obtained through an anonymous online survey.

Results

All residents positively evaluated the program. Theoretical: 82% passed the online exam. The most valued topics: LAP in special clinical situations, complications, instruments, and configuration of the operating room (OR). Practical: all residents increased dry-lab box practices. A total of 23 experimental surgical sessions were carried out. For 64%, simulation in the experimental OR was a necessary complement to achieve laparoscopic skills and allowed them to feel more confident. Forty-five percent considered it essential to improve their surgical technique. E-BLUS evaluation was valued as a means to achieve dexterity and safer surgery by 90%. Reduction in time and errors were observed through time, although only 2 passed the E-BLUS.

Conclusion

Our program for learning LAP includes the acquisition of knowledge, training of basic skills and surgical technique in a safe environment, as well as an objective evaluation. Encouraged practice of basic skills and surgical technique simulation and improved objective evaluation. It is structured, reproducible, systematic and has been positively valued, although it requires commitment for success.

Tele-mentoring - a way to expand laparoscopic simulator training for medical students over large distances: a prospective randomized pilot study

BACKGROUND

Studies have shown the clinical benefits of laparoscopic simulator training. Decreasing numbers of operations by surgical residents have further increased the need for surgical simulator training. However, many surgical simulators in Sweden are often insufficiently used or not used at all. Furthermore, large geographical distances make access to curriculum-based surgical simulator training at established simulator centres difficult. The aim of this study was to evaluate whether tele-mentoring (TM) could be well tolerated and improve basic laparoscopic surgical skills of medical students 900 km away from the teacher.

METHODS

Twenty students completed an informed consent and a pre-experimental questionnaire. The students were randomized into two groups: (1) TM (N = 10), receiving instructor feedback via video-link and (2) control group (CG, N = 10) with lone practice. Initial warm-up occurred in the Simball Box simulator with one Rope Race task followed by five consecutive Rope Race and three Peg Picker tasks. Afterwards, all students completed a second questionnaire.

RESULTS

The whole group enjoyed the simulator training (prescore 73.3% versus postscore 89.2%, P < 0.0001). With TM, the simulator Rope Race overall score increased (prescore 30.8% versus postscore 43.4%; P = 0.004), and the distance that the laparoscopic instruments moved decreased by 40% (P = 0.015), indicating better precision, whereas in the CG it did not. In Peg Picker, the overall scores increased, whereas total time and distance of the instruments decreased in both groups, indicating better performance and precision.

CONCLUSIONS

Simulation training was highly appreciated overall. The TM group showed better overall performance with increased precision in what we believe to be the visuospatially more demanding Rope Race tasks compared to the CG. We suggest that surgical simulator tele-mentoring over long distances could be a viable way to both motivate and increase laparoscopic basic skills training in the future.

Prediction of laparoscopic skills: objective learning curve analysis

INTRODUCTION

Prediction of proficiency of laparoscopic skills is essential to establish personalized training programs. Objective assessment of laparoscopic skills has been validated in a laparoscopic box trainer with force, motion and time recognition. The aim of this study is to investigate whether acquiring proficiency of laparoscopic skills can be predicted based on performance in such a training box.

METHODS

Surgical residents in their first year of training performed six different tasks in the Lapron box trainer. Force, motion and time data, three objective measures of tissue manipulation and instrument handling, were collected and analyzed for the six different tasks. Linear regression tests were used to predict the learning curve and the number of repetitions required to reach proficiency.

RESULTS

A total of 6010 practice sessions performed by 42 trainees from 13 Dutch hospitals were assessed and included for analysis. Proficiency level was determined as a mean result of seven experts performing 42 trials. Learning curve graphs and prediction models for each task were calculated. A significant relationship between force, motion and time during six different tasks and prediction of proficiency was present in 17 out of 18 analyses.

CONCLUSION

The learning curve of proficiency of laparoscopic skills can accurately be predicted after three repetitions of six tasks in a training box with force, path length and time recognition. This will facilitate personalized training programs in laparoscopic surgery.

Quantitative influence and performance analysis of virtual reality laparoscopic surgical training system

BACKGROUND

Virtual reality (VR) surgery training has become a trend in clinical education. Many research papers validate the effectiveness of VR-based surgical simulators in training medical students. However, most existing articles employ subjective methods to study the residents' surgical skills improvement. Few of them investigate how to improve the surgery skills on specific dimensions substantially.

METHODS

Our paper resorts to physiological approaches to objectively study the quantitative influence and performance analysis of VR laparoscopic surgical training system for medical students. Fifty-one participants were recruited from a pool of medical students. They conducted four pre and post experiments in the training box. They were trained on VR-based laparoscopic surgery simulators (VRLS) in the middle of pre and post experiments. Their operation and physiological data (heart rate and electroencephalogram) are recorded during the pre and post experiments. The physiological data is used to compute cognitive load and flow experience quantitatively. Senior surgeons graded their performance using newly designed hybrid standards for fundamental tasks and Global operative assessment of laparoscopic skills (GOALS) standards for colon resection tasks. Finally, the participants were required to fill the questionnaires about their cognitive load and flow experience.

RESULTS

After training on VRLS, the time of the experimental group to complete the same task could drop sharply (p < 0.01). The performance scores are enhanced significantly (p < 0.01). The performance and cognitive load computed from EEG are negatively correlated (p < 0.05).

CONCLUSION

The results show that the VRLS could highly improve medical students' performance and enable the participants to obtain flow experience with a lower cognitive load. Participants' performance is negatively correlated with cognitive load through quantitative physiological analysis. This might provide a new way of assessing skill acquirement.

The Use of Feedback in Improving the Knowledge, Attitudes and Skills of Medical Students: a Systematic Review and Meta-analysis of Randomized Controlled Trials

This study aims to investigate the effectiveness of the use of different feedback modalities in improving the knowledge, attitudes, and skills of medical students compared to students receiving no feedback or unstructured feedback. A systematic review and meta-analysis of randomized controlled trials was conducted based on a search of the Cochrane, ERIC, PubMed, Scopus, and Web of Science databases. A total of 26 studies were included for the systematic review and 13 for the meta-analysis. The meta-analysis revealed that the use of feedback was associated with better results compared to control groups (SMD = 0.80 [0.56-1.04], p < 0.001), and also when only high-quality studies were included (SMD = 0.86 [0.56-1.16], p < 0.001). Our findings revealed high heterogeneity in the use of feedback in medical education. However, the results of most of the studies and of the meta-analysis were positive, showing that feedback had a positive influence on the education-learning process of the students. PROSPERO registration: CRD42018112688.

Personalized Adaptive Training Improves Performance at a Professional First-Person Shooter Action Videogame

First-Person Shooter (FPS) game experience can be transferred to untrained cognitive functions such as attention, visual short-term memory, spatial cognition, and decision-making. However, previous studies have been using off-the-shelf FPS games based on predefined gaming settings, therefore it is not known whether such improvement of in game performance and transfer of abilities can be further improved by creating a in-game, adaptive in-game training protocol. To address this question, we compared the impact of a popular FPS-game (Counter-Strike:Global-Offensive–CS:GO) with an ad hoc version of the game based on a personalized, adaptive algorithm modifying the artificial intelligence of opponents as well as the overall game difficulty on the basis of individual gaming performance. Two groups of FPS-naïve healthy young participants were randomly assigned to playing one of the two game versions (11 and 10 participants, respectively) 2 h/day for 3 weeks in a controlled laboratory setting, including daily in-game performance monitoring and extensive cognitive evaluations administered before, immediately after, and 3 months after training. Participants exposed to the adaptive version of the game were found to progress significantly faster in terms of in-game performance, reaching gaming scenarios up to 2.5 times more difficult than the group exposed to standard CS:GO (p < 0.05). A significant increase in cognitive performance was also observed. Personalized FPS gaming can significantly speed-up the learning curve of action videogame-players, with possible future applications for expert-video-gamers and potential relevance for clinical-rehabilitative applications.

Baseline characteristics in laparoscopic simulator performance: The impact of personal computer (PC)-gaming experience and visuospatial ability

BACKGROUND

Learning via simulators is under constant development, and it is important to further optimize simulator training curricula. This study investigates the impact of personal computer-gaming experience, visuospatial skills, and repetitive training on laparoscopic simulator performance and specifically on the constituent parameters of the simulator score.

METHODS

Forty-seven medical students completed 3 consecutive Minimally Invasive Surgical Trainer-Virtual Reality simulator trials. Previously, they performed a visuospatial test and completed a questionnaire regarding baseline characteristics and personal computer-gaming experience. Linear regression was used to analyze the relationship between simulator performance and type of personal computer-gaming experience and visuospatial ability.

RESULTS

During the first 2 Minimally Invasive Surgical Trainer-Virtual Reality simulation tasks, there was an association between personal computer-gaming experience and the coordination parameters of the score (eg, EconDiath task 1: P = .0047; EconDiath task 2: P = .0102; EconDiath task 3: P = .0836). The type of game category played seemed to have an impact on the coordination parameters (eg, EconDiath task 1-3 for sport games versus no-sport games: P = .01, P = .0013, and P = .01, respectively). In the first Minimally Invasive Surgical Trainer task, visuospatial ability correlated with Minimally Invasive Surgical Trainer simulator performance but was abolished with repetitive training (overall Minimally Invasive Surgical Trainer score task 1-3: P = .0122, P = .0991, and P = .3506, respectively). Sex-specific differences were noted initially but were abolished with training.

CONCLUSION

Sport games versus no-sport games demonstrated a significantly better Minimally Invasive Surgical Trainer performance. Furthermore, repetitive laparoscopic simulator training may compensate for a previous lack of personal computer-gaming experience, low visuospatial ability, and sex differences.

Use of feedback on medium-term blood pressure measurement skills in medical students: a randomized controlled trial

BACKGROUND

The objective of this study was to assess the effect of feedback on medium-term blood pressure (BP) measurement skills in medical students versus a control group.

METHODS

A randomized controlled trial in first-year medical students was conducted. After theoretical-practical training on measuring BP, the students were randomized into a control group or intervention group. The skill was assessed immediately after the training (T1) by evaluating students with a simulated standardized patient using a checklist. Students in the intervention group received feedback immediately after assessment, whereas the control group received no feedback. After 3 months (T2), each student was reassessed in the same way as for (T1).

RESULTS

Ninety-two first-year medical students took part in the study (45 in control group and 47 in intervention group). At T1, there were no differences in the skill measured. At T2 (after three months), there was a significant difference in the intervention group (score = 23.97 ± 3.82) compared with the control group (score = 20.91 ± 4.87), P < 0.001, d = 0.69. In the intervention group, the scores were maintained at 3 months (T1 = 23.23 and T2 = 23.97, P = 0.335), whereas in the control group, scores declined significantly (T1 = 23.44 and T2 = 20.91; P = 0.002).

CONCLUSION

Receiving feedback promotes retention of learning of BP measurement skills over the medium term. Further studies applying feedback to other skills should be conducted.

Video Coaching Improving Contemporary Technical and Nontechnical Ability in Laparoscopic Education

OBJECTIVE

A video coaching (VC) system has been developed in surgical education. This study compares the educational effect on technical and nontechnical skills of the VC method for teaching laparoscopic surgery.

DESIGN

We conducted a prospectively randomized study of an education program to teach laparoscopic procedures.

SETTING

The study was performed at the Chang Gung Memorial Hospital, a university hospital in Taiwan.

PARTICIPANTS

We enrolled sixteen first- or second-year surgical residents.The participants were randomized into VC and conventional teaching (CT) groups, and their surgical skills were judged by the Global Operation Assessment of Laparoscopic Skills (GOALS) and the Objective Structured Assessment of Technical Skills (OSATS). Nontechnical skills were evaluated by the Non-Technical Skills for Surgeons (NOTSS) assessment and self-efficacy questionnaires (SEQs). After the program, posttraining scores were compared to assess improvements.

RESULTS

The 16 enrolled participants finished the entire course and completed all the videos during the study period. Comparing the VC and CT groups, we found that the pretraining GOALS, OSATS, NOTSS and SEQ scores were similar between both groups. However, after training, the OSATS score gain was higher in the VC groupthan in the CT group (9.25 ± 2.05 vs. 6.50 ± 1.51, p=0.009). Regarding nontechnical skills, the NOTSS score improved more in the VC group than in the CT group (5.50 ± 0.93 vs. 4.25 ± 0.89, p=0.015). The SEQ score was also higher in the VC group (32.13 ± 2.10) than in the CT group (29.50 ± 1.77), with a significant difference (p=0.018).

CONCLUSION

VC can help surgeons build their expertise using a more accessible method. Additionally, VC can shorten the learning curve and improve self-efficacy, thereby contributing to surgeons' education.

Effect of face-to-face verbal feedback compared with no or alternative feedback on the objective workplace task performance of health professionals: a systematic review and meta-analysis

OBJECTIVE

Verbal face-to-face feedback on clinical task performance is a fundamental component of health professions education. Experts argue that feedback is critical for performance improvement, but the evidence is limited. The aim of this systematic review was to investigate the effect of face-to-face verbal feedback from a health professional, compared with alternative or no feedback, on the objective workplace task performance of another health professional.

DESIGN

Systematic review and meta-analysis.

METHODS

We searched the full holdings of Ovid MEDLINE, CENTRAL, Embase, CINAHL and PsycINFO up to 1 February 2019 and searched references of included studies. Two authors independently undertook study selection, data extraction and quality appraisal. Studies were included if they were randomised controlled trials investigating the effect of feedback, in which health professionals were randomised to individual verbal face-to-face feedback compared with no feedback or alternative feedback and available as full-text publications in English. The certainty of evidence was assessed using the Grading of Recommendations, Assessment, Development and Evaluations approach. For feedback compared with no feedback, outcome data from included studies were pooled using a random effects model.

RESULTS

In total, 26 trials met the inclusion criteria, involving 2307 participants. For the effect of verbal face-to-face feedback on performance compared with no feedback, when studies at high risk of bias were excluded, eight studies involving 392 health professionals were included in a meta-analysis: the standardised mean difference (SMD) was 0.7 (95% CI 0.37 to 1.03; p<0.001) in favour of feedback. The calculated SMD prediction interval was -0.06 to 1.46. For feedback compared with alternative feedback, studies could not be pooled due to substantial design and intervention heterogeneity. All included studies were summarised, and key factors likely to influence performance were identified including components within feedback interventions, instruction and practice opportunities.

CONCLUSIONS

Verbal face-to-face feedback in the health professions may result in a moderate to large improvement in workplace task performance, compared with no feedback. However, the quality of evidence was low, primarily due to risk of bias and publication bias. Further research is needed. In particular, we found a lack of high-quality trials that clearly reported key components likely to influence performance.

TRIAL REGISTRATION NUMBER

CRD42017081796.

Trainee Performance After Laparoscopic Simulator Training Using a Blackbox versus LapMentor

BACKGROUND

Training using laparoscopic high-fidelity simulators (LHFSs) to proficiency levels improves laparoscopic cholecystectomy skills. However, high-cost simulators and their limited availability could negatively impact residents' laparoscopic training opportunities. We aimed to assess whether motivation and surgical skill performance differ after basic skills training (BST) using a low-cost (Blackbox) versus LHFS (LapMentor) among medical students.

MATERIALS AND METHODS

Sixty-three medical students from Karolinska Institutet volunteered, completing written informed consent, questionnaire regarding expectations of the simulation training, and a visuospatial ability test. They were randomized into two groups that received BST using Blackbox (n = 32) or LapMentor (n = 31). However, seven students absence resulted in 56 participants, followed by another 9 dropouts. Subsequently, after training, 47 students took up three consecutive tests using the minimally invasive surgical trainer-virtual reality (MIST-VR) simulator, finalizing a questionnaire.

RESULTS

More Blackbox group participants completed all MIST-VR tests (29/31 versus 18/25). Students anticipated mastering LapMentor would be more difficult than Blackbox (P = 0.04). In those completing the simulation training, a trend toward an increase was noted in how well participants in the Blackbox group liked the simulator training (P = 0.07). Subgroup analysis of motivation and difficulty in liking the training regardless of simulator was found only in women (Blackbox [P = 0.02]; LapMentor [P = 0.06]). In the Blackbox group, the perceived difficulty of training, facilitation, and liking the Blackbox training (significant only in women) were significantly correlated with the students' performance in the MIST-simulator. No such correlations were found in the LapMentor group.

CONCLUSIONS

Results indicate an important role for low-tech/low-cost Blackbox laparoscopic BST of students in an otherwise high-tech surrounding. Furthermore, experience of Blackbox BST procedures correlate with students' performance in the MIST-VR simulator, with some gender-specific differences.

Trends in UK endoscopy training in the BSG trainees' national survey and strategic planning for the future

BACKGROUND

Improvements in the structure of endoscopy training programmes resulting in certification from the Joint Advisory Group in Gastrointestinal Endoscopy have been acknowledged to improve training experience and contribute to enhanced colonoscopy performance.

OBJECTIVES

The 2016 British Society of Gastroenterology trainees' survey of endoscopy training explored the delivery of endoscopy training - access to lists; level of supervision and trainee's progression through diagnostic, core therapy and subspecialty training. In addition, the barriers to endoscopy training progress and utility of training tools were examined.

METHODS

A web-based survey (Survey Monkey) was sent to all higher specialty gastroenterology trainees.

RESULTS

There were some improvements in relation to earlier surveys; 85% of trainees were satisfied with the level of supervision of their training. But there were ongoing problems; 12.5% of trainees had no access to a regular training list, and 53% of final year trainees had yet to achieve full certification in colonoscopy. 9% of final year trainees did not feel confident in endoscopic management of upper GI bleeds.

CONCLUSIONS

The survey findings provide a challenge to those agencies tasked with supporting endoscopy training in the UK. Acknowledging the findings of the survey, the paper provides a strategic response with reference to increased service pressures, reduced overall training time in specialty training programmes and the requirement to support general medical and surgical on-call commitments. It describes the steps required to improve training on the ground: delivering additional training tools and learning resources, and introducing certification standards for therapeutic modalities in parallel with goals for improving the quality of endoscopy in the UK.

Adapting Feedback to Individual Residents: An Examination of Preceptor Challenges and Approaches

Background

Feedback conversations between preceptors and residents usually occur in closed settings. Little is known about how preceptors address the challenges posed by residents with different skill sets, performance levels, and personal contexts.

Objective

This study explored the challenges that preceptors experienced and approaches taken in adapting feedback conversations to individual residents.

Methods

In 2015, 18 preceptors participated in feedback simulations portraying residents with variations in skill, insight, confidence, and distress, followed by debriefing of the feedback conversation with a facilitator. These interactions were recorded, transcribed, and analyzed using thematic and framework analysis.

Results

The preceptors encountered common challenges with feedback conversations, including uncertainty in how to individualize feedback to residents and how to navigate tensions between resident- and preceptor-identified goals. Preceptors questioned their ability to enhance skills for highly performing residents, whether they could be directive when residents had insight gaps, how they could reframe the perceptions of the overly confident resident, and whether they should offer support to emotionally distressed residents or provide feedback about performance. Preceptors adapted their approach to feedback, drawing on techniques of coaching for highly performing residents, directing for residents with insight gaps, mediation with overly confident residents, and mentoring with emotionally distressed residents.

Conclusions

Examining the feedback challenges preceptors encounter and the approaches taken to adapt feedback to individual residents can provide insight into how preceptors meet the challenges of competency-based medical education, in which frequent, focused feedback is essential for residents to achieve educational milestones and entrustable professional activity expectations.