Video-augmented feedback for procedural performance

Role of video self-assessment in laparoscopic simulation training: a randomized pilot trial

BACKGROUND

Residency programs have implemented simulation training to compensate for reduced operating room exposure. Video recording is an educational tool that can be utilized for coaching, telepresence, and self-assessment during simulation training. Data is limited on the utility of video recording and self-assessment for laparoscopic training in Ob/Gyn residency programs..

OBJECTIVE

This study aimed to determine the role of video self-assessment as an educational tool in laparoscopic simulation training and to establish the feasibility of our study design for a larger randomized controlled trial.

STUDY DESIGN

This was a prospective pilot study with a parallel, randomized, trial design that occurred in the Department of Obstetrics and Gynecology at the Mount Sinai Hospital. Subject participation took place in a surgical simulation training room. A total of 23 subjects were recruited (7 medical students, 15 residents, 1 fellow) voluntarily. All participants completed the study. All the subjects completed a pretest survey. The surgical simulation room contained a single Fundamentals of Laparoscopic Surgery box trainer and video-recording station. For session #1, each participant performed 2 Fundamentals of Laparoscopic Surgery tasks (A, peg transfer; B, intracorporeal knot tie). Participants were video recorded during session #1 and were randomized to either receive or not receive their video recording. The video group (n=13) and control group (n=10) repeated the Fundamentals of Laparoscopic Surgery tasks 7 to 10 days later (session #2). The primary outcome was percentage change in completion time between sessions. Secondary outcomes were percentage change in peg and needle drops between sessions.

RESULTS

The participant characteristics (video vs control) were as follows: average training level (6.15 vs 4.90 years), self-assessment (1=poor, 10=excellent) of surgical skill (4.8 vs 3.7), and laparoscopic skill (4.4 vs 3.5). Training level was inversely correlated with completion time for tasks A and B (r, -0.79 and -0.87; P<.0001). Less experienced trainees required the maximum time allotted for each task in session #1 (A, 3; B, 13). Regarding the primary outcome, the video group improved less than the control group (A, 16.7% vs 28.3%; B, 14.4% vs 17.3%). After controlling for training level (residents only), the video group improved more in the primary outcome (A, 17% vs 7.4%; B, 20.9% vs 16.5%) and secondary outcomes (A, 0.0% vs -194.1%; B, 41.3% vs 37.6%).

CONCLUSION

Video self-assessment has a potential role in simulation training for obstetrics-gynecology residents. With key improvements, the feasibility of our study design was demonstrated in preparation for a future definitive trial.

Comparison of self versus expert-assisted feedback for cricothyroidotomy training: a randomized trial

BACKGROUND

The self-video feedback method may have the potential to provide a low-cost alternative to physician-driven simulation-based training. This study aimed to assess the utility of two video feedback methods by comparing the improvement in performing cricothyroidotomy procedure following self video feedback (trainees review their performance by themselves) and expert-assisted video feedback (trainees review their performance while an emergency physician provides additional feedback).

METHODS

This study was pretest-posttest and two-group designed research performed at a university simulation center with 89 final-year medical students and used a cricothyroidotomy simulation model. After seeing an educational presentation and a best practice video, trainees were randomized into two groups; self video feedback group (SVFG) and expert-assisted video feedback group (EVFG). They performed the cricothyroidotomy before and after the feedback. The procedures were also recorded and scored by two emergency physicians.

RESULTS

There was a statistically significant improvement between pre-feedback and post-feedback assessments in terms of scores received and time needed for the procedures in both SVFG and EVFG groups (p < 0.05). Additionally, the post-feedback assessment scores were higher and time needed for the procedure was lower in the EVFG when compared with SVFG (p < 0.05 for both).

CONCLUSIONS

Results demonstrated significant improvement in cricothyroidotomy performance with both types of video feedback method. Even though the improvement was better in the EVFG compared to the SVFG, the self video feedback may have value especially in situations where expert-assisted feedback is not possible.

Video-assisted self-reflection of resuscitations for resident education and improvement of leadership skills: A pilot study

INTRODUCTION

One of the most challenging aspects of Emergency Medicine (EM) residency is mastering the leadership skills required during a resuscitation. Use of resuscitation video recording for debriefing is gaining popularity in graduate medical education. However, there are limited studies of how video technology can be used to improve leadership skills in the emergency department. We aim to evaluate the utility of video-assisted self-reflection, compared with self-reflection alone, in the setting of resuscitation leadership.

METHODS

This was a prospective, randomized, controlled pilot study conducted in 2018 at an urban level 1 trauma center with a three-year EM residency program. The trial included postgraduate year (PGY) 2 and 3 residents (n = 10). Each resident acted as an individual team leader for a live real-time resuscitation in the emergency department. The authors classified a patient as a resuscitation if there was an immediate life- or limb-threatening disease process or an abnormal vital sign with an indication of hypoperfusion. Each resident was recorded as the team leader twice. Both control and intervention groups produced written self-reflection after their first recording. The intervention group viewed their resuscitation recording while completing the written reflection. After their reflection, all participants were recorded for a second resuscitation. Two faculty experts, blinded to the study, scored each video using the Concise Assessment of Leader Management (CALM) scale to measure the leadership skills of the resident team leader.

RESULTS

Five PGY‑3 and five PGY‑2 residents participated. The weighted kappa between the two experts was 0.45 (CI 0.34-0.56, p < 0.0001). The median gain score in the control group was -1.5 (IQR) versus 0.5 in the intervention group (IQR).

DISCUSSION

Video-assisted self-reflection showed positive gain score trends in leadership evaluation for residents during a resuscitation compared with the non-video assisted control group. This tool would be beneficial to implement in EM residency.

Group Video Feedback is an Effective and Efficient Tool for Enhancing Skills of Surgical Interns

BACKGROUND

Our group has shown that personalized video feedback (PVF) is better than a task demonstration video at increasing wound closure skills among incoming surgical interns. However, offering PVF can be time-consuming. We sought to compare the educational effects and time required for group video feedback (GVF) and PVF.

METHODS

We have mailed our matched incoming "surgical" interns a "welcome package" in mid-March for the past 3 y. The package includes similar resources each year. Incoming interns were asked to video record themselves performing six tasks three separate times between April 1 and June 15. After each submission in 2016 and 2017, incoming interns received 2 min of personalized feedback on their three separate wound closure videos (PVF). In 2018, incoming interns received 5 min of group-based feedback three separate times covering all six tasks (GVF). We compared performance (July Surgical Olympics) of these six skills against the interns from the previous 2 y who received PVF on only one skill (suturing).

RESULTS

Twenty-three incoming interns received the pre-residency package and participated in the 2018 Surgical Olympics. This 2018 GVF class had a higher overall mean score for six stations (31.5 [standard deviation = ±7.7]) than the 2016 and 2017 PVF classes (25.6 [standard deviation = ±8]; P < 0.0001). Knot tying ability and suturing skill were similar between the classes. The GVF group performed better on the remaining four skills . The total time of surgical staff and educators spent per class in 2018 (GVF class) was 30 min and includes six tasks compared with 276 min of effort in each of the 2016 and 2017 classes (PVF included one task).

CONCLUSIONS

GVF and PVF as a component of preemptive training had the same effect on improving suturing skills among the interns. GVF required less educator editing and voice over time. GVF is effective and efficient in enhancing incoming interns' performance in multiple skills.

Self-Assessment of Surgical Skills: A Systematic Review

OBJECTIVE

Self-assessment is fundamental in surgical training to enhance learning in the absence of trainer feedback. The primary objective of this review was to assess the factors that influence accuracy of self-assessment at technical skills across all surgical specialties. The secondary objective was to assess whether there are any innate factors or attributes to predict those that will carry out effective self-assessment.

DESIGN

A systematic review was carried out in accordance with PRISMA guidelines. A search strategy encompassing MEDLINE, EMBASE, ERIC, WHO, and the Cochrane database was conducted to identify studies investigating self-assessment at any surgical task. Quality was assessed using the Newcastle-Ottawa scale. A summary table was created to describe specialty, participants, task, setting, assessment tool, and correlation coefficient between self and expert assessment. The review protocol was registered in PROSPERO.

RESULTS

Of 24,638 citations, 40 met inclusion criteria. In total 1753 participants performed 68 procedures. Twenty-six studies investigated skills in general surgery with the remaining 14 in various other surgical specialties. Accuracy of self-assessment is superior in those with greater experience and age, and with use of retrospective video playback. Accuracy tends to be reflected by overestimation of performance. Stressful environments reduce accuracy. There is limited evidence in the literature regarding predicting traits for those who will carry out accurate self-assessment.

CONCLUSIONS

The ability to perform accurate self-assessment is an important skill in surgical training, with accuracy being influenced by a multitude of factors. The use of self-assessment from retrospective video playback may be of benefit in surgical training curricula to enhance learning of technical skills. Further studies are required to define predictors of good self-assessment, which will strengthen recruitment and mentoring to assist trainee learning.

Instant replay: Evaluation of instant video feedback in surgical novices for a laparoscopic gallbladder dissection

BACKGROUND

Athletes often use video to improve their technique. We hypothesized that surgical novices given feedback using video-replay would outperform surgical novices given verbal feedback in the performance of a laparoscopic task.

METHODS

Our study used a prospective, randomized control design. The surgical task involved the laparoscopic dissection of a pig gallbladder. Our participants performed a dissection, pre- and post-traditional or video feedback. Each recording was independently scored by two staff surgeons using the previously validated rating tools.

RESULTS

There was no significant difference between video feedback or traditional feedback groups in their mean overall or task specific scores. Both traditional and video-feedback groups had a trend towards improved performance post-feedback.

CONCLUSIONS

No significant difference in performance by both our global assessment metrics or task-specific metrics was observed. Video feedback requires further study to investigate its impact on surgical training.

Comparison of faculty assessment and students' self-assessment of performance during clinical case discussions in a pharmacotherapy capstone course

OBJECTIVES

The primary objective of this study was to compare faculty assessment and third year students' self-assessment of performance in clinical case discussions. The secondary objective was to evaluate if student characteristics influence self-assessments.

METHODS

This retrospective analysis compared faculty and student self-assessment scores for two clinical case discussions using Spearman's correlation and Wilcoxon's signed ranks test. Chi-squared test was used to compare frequency of faculty and student self-assessments indicating the highest possible rating for the pooled score and for each individual component. The pooled score included three individual components: level of engagement, quality of contribution, and professionalism.

RESULTS

Pooled faculty and student self-assessments correlated for both the first (r = 0.41, p < 0.001) and second (r = 0.35; p < 0.001) clinical case discussions. The frequency that faculty and student self-assessment ratings were the highest possible pooled score was similar for both the first (51.3% vs. 44.7%, respectively, p = 0.25) and second (58.6% vs. 47.4%, p = 0.05) clinical case discussions. Student characteristics (age, gender, and grade point average at graduation) did not influence self-assessments.

CONCLUSIONS

Students' self-assessment correlated with faculty assessment of performance during clinical case discussions. Increased use of self-assessments for professional development in pharmacy and other healthcare professional curricula should be considered.