Comparison of faculty versus structured peer-feedback for acquisitions of basic and intermediate-level surgical skills

Comparing Peer and Faculty Feedback for Asynchronous Laparoscopic Skill Acquisition

OBJECTIVE

Traditionally, expert surgeons have provided surgical trainees with feedback about their simulation performance, including for asynchronous practice. Unfortunately, innumerable time demands may limit experts' ability to provide feedback. It is unknown whether and how peer feedback is an effective mechanism to help residents acquire laparoscopic skill in an asynchronous setting. As such, we aimed to assess the effect of peer feedback on laparoscopic performance and determine how residents perceive giving and receiving peer feedback.

DESIGN

We conducted a convergent mixed methods study. In the quantitative component, we randomized residents to receive feedback on home laparoscopic tasks from peers or faculty. We then held an end-of-curriculum, in-person laparoscopic assessment with members from both groups and compared performance on the in-person assessment between the groups. In the qualitative component, we conducted interviews with resident participants to explore experiences with feedback and performance. Three authors coded and rigorously reviewed interview data using a directed content analysis.

SETTING

We performed this study at a single tertiary academic institution: the University of California, San Francisco.

PARTICIPANTS

We invited 47 junior residents in general surgery, obstetrics-gynecology, and urology to participate, of whom 37 (79%) participated in the home curriculum and 25 (53%) participated in the end-of-curriculum assessment.

RESULTS

Residents in the peer feedback group scored similarly on the final assessment (mean 70.7%; SD 16.1%) as residents in the faculty feedback group (mean 71.8%; SD 11.9%) (p = 0.86). Through qualitative analysis of interviews with 13 residents, we identified key reasons for peer feedback's efficacy: shared mental models, the ability to brainstorm and appreciate new approaches, and a low-stakes learning environment.

CONCLUSIONS

We found that peer and faculty feedback led to similar performance in basic laparoscopy and that residents engaged positively with peer feedback, suggesting that peer feedback can be used when residents learn basic laparoscopy.

Student Pharmacists Provide Similar Quality Clinical Reasoning Feedback as Resident Teaching Assistants

OBJECTIVE

Pharmacists utilize clinical reasoning (CR) to improve patient outcomes via medication optimization. It is critical to develop these skills in student pharmacists, yet optimal pedagogies to teach and assess CR are unknown. Peer feedback may be used to develop CR in student pharmacists, but a certain feedback quality must be reached to be effective. This study sought to evaluate if student pharmacists could provide similar quality peer feedback compared to pharmacy resident teaching assistant (TA) feedback.

METHODS

This was a retrospective, mixed-methods pedagogical analysis comparing the quality of first-year student pharmacist peer feedback to resident TA CR feedback. The CR comments were defined using the intellectual standards of CR. Quality was assessed for task specification, gap identification, actionability, and process orientation by 2 independent investigators. Student performance and perceptions were also assessed. Mann-Whitney U, t tests, and descriptive statistics were used to analyze data where appropriate.

RESULTS

Clinical reasoning feedback from peers (N = 805) and TAs (N = 206) were analyzed. Interrater reliability for feedback quality was moderate to substantial. Overall, peer CR feedback was of higher quality regarding task specification and process orientation while TA CR feedback was of higher quality regarding gap identification and actionability. Students receiving peer feedback performed better on a final patient case than those receiving TA feedback (95.2% vs 92.3%). Overall, the peer feedback process was well received by students.

CONCLUSION

Student pharmacists can provide similar quality feedback as resident TAs. Peer feedback offers an alternative to resident TA feedback and has the potential to contribute to improved CR skills.

Evidence supporting performance measures of laparoscopic appendectomy through a novel surgical proficiency assessment tool and low-cost laparoscopic training system

BACKGROUND

Laparoscopic training remains inaccessible for surgeons in low- and middle-income countries, limiting its widespread adoption. We developed a novel tool for assessment of laparoscopic appendectomy skills through ALL-SAFE, a low-cost laparoscopy training system.

METHODS

This pilot study in Ethiopia, Cameroon, and the USA assessed appendectomy skills using the ALL-SAFE training system. Performance measures were captured using the ALL-SAFE verification of proficiency tool (APPY-VOP), consisting of a checklist, modified Objective Structured Assessment of Technical Skills (m-OSATS), and final rating. Twenty participants, including novice (n = 11), intermediate (n = 8), and expert (n = 1), completed an online module covering appendicitis management and psychomotor skills in laparoscopic appendectomy. After viewing an expert skills demonstration video, participants recorded their performance within ALL-SAFE. Using the APPY-VOP, participants rated their own and three peer videos. We used the Kruskal-Wallis test and a Many-Facet Rasch Model to evaluate (i) capacity of APPY-VOP to differentiate performance levels, (ii) correlation among three APPY-VOP components, and (iii) rating differences across groups.

RESULTS

Checklist scores increased from novice (M = 21.02) to intermediate (M = 23.64) and expert (M = 28.25), with differentiation between experts and novices, P = 0.005. All five m-OSATS domains and global summed, total summed, and final rating discriminated across all performance levels (P < 0.001). APPY-VOP final ratings adequately discriminated Competent (M = 2.0), Borderline (N = 1.8), and Not Competent (M = 1.4) performances, Χ2 (2,85) = 32.3, P = 0.001. There was a positive correlation between ALL-SAFE checklist and m-OSATS summed scores, r(83) = 0.63, P < 0.001. Comparison of ratings suggested no differences across expertise levels (P = 0.69) or location (P = 0.66).

CONCLUSION

APPY-VOP effectively discriminated between novice and expert performance in laparoscopic appendectomy skills in a simulated setting. Scoring alignment across raters suggests consistent evaluation, independent of expertise. These results support the use of APPY-VOP among all skill levels inside a peer rating system. Future studies will focus on correlating proficiency to clinical practice and scaling ALL-SAFE to other settings.

Peer Feedback: Recommendations for Behavior Analysts' Training and Supervision

The increase in demand for behavior analysts in recent years has also increased the importance of effective supervision practices in the field of behavior analysis. A critical supervisory skill is performance feedback, which entails proficiency with two distinct, yet inseparable repertoires of technical and nontechnical skills. Supervisors report never receiving explicit training in feedback (Sellers et al., 2019) and graduate training programs provide little to no training in nontechnical skills (LeBlanc, Taylor et al., 2020b; Pastrana et al., 2018). As with any skill, to develop proficiency with feedback delivery and reception, trainees may require ample practice opportunities. One mechanism to provide trainees routine practice opportunities is to use peers as behavior change agents and peer feedback as an instructional method. The utility of peer feedback has been recognized in the organizational behavior management (OBM) literature (e.g., behavior-based safety interventions; Lebbon et al., 2012; Wirth & Sigurdsson, 2008), and has been used successfully in medical student training and evaluation for several decades. In the context of behavior analytic training and supervision peer feedback has yet to be established as a training method. Similarities in the behavioral and medical fields (e.g., significance of professional and interpersonal skills for successful therapeutic relationships) make the medical field a good model from which behavior analysts can learn. Using peer feedback in training and supervision for behavior analysts may provide trainees with similar benefits to those reported in medical student training literature.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40617-022-00761-1.

Reconstructing feedback in graduate medical education: development of the REFLECT scale to measure feedback delivery in medical residency training

BACKGROUND

Feedback plays a pivotal role in graduate medical education, where medical residents are expected to acquire a wide range of practical and professional competencies. Assessing the feedback delivery status is a preliminary step for educators to enhance the quality of feedback provided. This study aims to develop an instrument to assess the various aspects of feedback delivery in medical residency training.

METHODS

The fifteen-item REFLECT (Residency Education Feedback Level Evaluation in Clinical Training) questionnaire was developed. The content validity was evaluated according to a panel member consisting of fourteen clinical professors and medical education instructors. After evaluating the test-retest reliability, the questionnaire was distributed to a sample of 154 medical residents and was further assessed in terms of internal consistency and factor analysis.

RESULTS

Content validity analysis resulted in an appropriate content validity ratio and content validity index for the final 15 items. The test-retest reliability resulted in an ICC of 0.949 (95% C.I. 0.870-0.980), indicating excellent reliability. The Cronbach's alpha for the 15-item questionnaire was α = 0.85, demonstrating good internal consistency. The factor analysis resulted in a four-factor structure: "attitude towards feedback", "quality of feedback", "perceived importance of feedback", and "reaction to feedback".

CONCLUSIONS

REFLECT proved to a reliable tool that could be utilized as a quick assessment method of feedback delivery, making it a suitable aid for educational managers and faculties to design necessary interventions aiming to enhance the quantity and quality of feedback provided.

Using peer feedback to enhance nursing students’ reflective abilities, clinical competencies, and sense of empowerment: a mixed-methods study

AIM

To assess the use of a framework to provide structured peer feedback and compare the effects of peer video feedback, peer verbal feedback versus faculty feedback on nursing students and peer tutors' learning outcomes and experiences BACKGROUND: Peer feedback has been utilized widely in health professions education to fill the gap for timely feedback, but some students were concerned with its quality, leading to perceptions that peer feedback may not be useful.

DESIGN

Sequential explanatory mixed-methods study METHODS: The study took place from January to February 2022. In phase 1, a quasi-experimental pretest-posttest design was used. First-year nursing students (n = 164) were allocated to peer video feedback, peer verbal feedback or faculty feedback arms. Senior nursing students (n = 69) were recruited to be peer tutors or the control group. The Groningen Reflective Ability Scale was used by first-year students to assess their reflective abilities, while the Simulation-based Assessment Tool was used by peer or faculty tutors to evaluate nursing students' clinical competence of a nursing skill during the simulation. The Debriefing Assessment for Simulation in Healthcare-Student Version was used by students to assess their peer/faculty tutors' feedback quality. Senior students' empowerment levels were measured using the Qualities of an Empowered Nurse scale. In phase 2, six semi-structured focus group discussions with peer tutors (n = 29) were conducted and thematically analyzed.

RESULTS

Peer video feedback and peer verbal feedback significantly improved students' reflective abilities but not in the faculty feedback arm. Students' clinical competence in a technical nursing skill significantly improved in all three arms. Improvements were significantly larger in those receiving peer video feedback and peer verbal feedback than faculty feedback, with no significant differences between peer video feedback and peer verbal feedback. Debriefing Assessment for Simulation in Healthcare-Student Version scores were not significantly different among the 3 arms. Empowerment levels of peer tutors significantly improved after providing peer feedback but not those in the control group. Seven themes were generated from the focus group discussions.

CONCLUSIONS

Although peer video feedback and peer verbal feedback were similarly effective in improving clinical competencies, peer video feedback was more time-consuming and stressful to students. The use of structured peer feedback improved peer tutors' feedback practices and were comparable to quality of faculty feedback. It also significantly increased their sense of empowerment. Peer feedback was widely supported by peer tutors who agreed that it should supplement faculty teaching.

Effectiveness of peer teaching in health professions education: A systematic review and meta-analysis

BACKGROUND

The reform in health professions education requires the focus to shift from fact memorization to exploring, analyzing, assimilating, and synthesizing information to promote active and collaborative learning. Peer teaching is one of the educational strategies.

AIMS

This review aimed to explore and synthesize quantitative evidence to determine the overall effect of peer teaching in enhancing students' theoretical knowledge and practical skills (e.g., procedural skills and resuscitation) in health professions education.

METHODS

PubMed, ScienceDirect, CINAHL, ERIC, ProQuest, reference lists of relevant studies, and reviews were searched till November 2021. Results were pooled using random-effects meta-analysis or narrative synthesis.

RESULTS

A total of 44 RCTs were included. This review showed a significant effect of peer teaching on procedural skills improvement and a comparable effect on theoretical knowledge and resuscitation skills acquisition compared to the conventional teaching method. Near-peer teaching seemed to be the most effective method for skill improvement. Subgroup analysis showed no significant differences between peer teaching and conventional teaching groups (e.g., expert/faculty teaching, self-study or lectures).

CONCLUSIONS

Peer teaching seems to be a promising teaching and learning strategy in health professions education, positively affecting theoretical knowledge and procedural skills. Future research should explore the effect of peer teaching in developing countries to provide a comprehensive picture of peer teaching.

Video feedback with error-focused or correct-focused examples in surgical skills distance learning: A randomized trial

BACKGROUND

Although it seems natural that surgical trainees would learn from demonstrations of a correct performance, evidence outside of surgical education has suggested that error-focused examples may promote error detection and improved procedural task performance. We hypothesized that feedback through error-focused videos would improve procedural learning more than correct-focused videos.

METHODS

We conducted a randomized controlled trial of video feedback comparing error-focused versus correct-focused examples. The participants were interviewees at our general surgery residency program in December 2020. All of the interviewees performed suturing and knot tying tasks on their interview day (baseline), with 70 common errors identified. For each error, we created an instructional feedback video in two formats: one video demonstrating the error and another demonstrating correct performance. The study participants received feedback videos based on baseline performance errors, with the format randomly assigned. Two blinded raters assessed the baseline and postintervention performances.

RESULTS

Thirty-seven interviewees enrolled and 17 submitted postintervention videos. The postintervention mean performance scores were significantly higher (P = .02) in the error-focused than the correct-focused example group (suturing [maximum score 18]: 16.9 vs 13.9 [difference 2.9; 95% CI 0.7, 5.1]; knot-tying [maximum score 24]: 21.6 vs 17.8 [difference 3.8; 95% CI 0.5, 7.0]). We found no between-group differences in performance time ([error-focused versus correct-focused] suturing: 246 vs 256 s; knot-tying: 170 vs 138 s; P = .08). Mean satisfaction with feedback was similar between groups (error-focused: mean = 5.3 versus correct-focused: mean = 5.2, out of 7; P = .95).

CONCLUSION

Feedback that highlights errors is associated with better learning of surgical skills than feedback demonstrating correct performance, confirming our hypothesis.

Video-Assisted Peer Teaching for Surgical Skills Training - Innovative Potential for the Medical Curriculum and Beyond: A Randomized Controlled Trial

OBJECTIVE

Hygienic healthcare standards are essential for avoiding hospital infections. However, medical students and staff lack training in this field, which may be due to high personnel resources of present educational approaches. Thus, there is an urgent need for a novel and efficient approach. Aim of the study is to compare a newly developed video-assisted peer feedback (VAPF) method for teaching wound dressings to the traditional teaching method with qualified instructor feedback (QIF) with respect to essential learning outcomes.

DESIGN, SETTING AND PARTICIPANTS

In this randomized controlled noninferiority trial, 251 medical undergraduates were randomly assigned to one of two interventions (QIF n = 127; VAPF n = 124). In QIF, participants received feedback from a qualified instructor. In VAPF, participants video-recorded each other while performing a wound dressing and gave each other feedback assisted by a standardized checklist. Outcome measures were participants' score in an objective structured practical examination (OSPE) and a written exam after the course.

RESULTS

Noninferiority of VAPF (n = 123) compared to QIF (n = 127) was confirmed for both OSPE (QIF: 8.83 ± 1.30; VAPF: 8.88 ± 1.04; mean difference -0.04, 95% CI -0.34 to 0.25) and written exam (QIF: 8.99 ± 1.06; VAPF: 9.14 ± 1.05; mean difference -0.15, 95% CI -0.41 to 0.12).

CONCLUSIONS

VAPF is a cost-efficient and viable alternative to QIF commonly used in medical education. It provides comparable training outcomes to the traditional training method with lower personnel investment. VAPF is a promising educational method for improving essential clinical competencies.

The Feedback Form and Its Role in Improving the Quality of Trauma Care

Background: One of the tasks of a level I trauma center is quality improvement of level II and level III regional hospitals and emergency medical services by means of continuous education and learning processes. One of the tools for this, which provides constant monitoring of the quality of treatment, is feedback. The purpose of the study was to evaluate the effect of feedback on the quality of trauma care. Methods: Retrospective cohort study comprising two periods of time, 2012-2013 and 2017-2018. The study group included physicians and pre-hospital staff who treated patients prior to referral to the level I center. Upon arrival when the trauma teams identified issues requiring improvement, they were asked to fill in feedback forms. Data on patients treated in the trauma shock room for whom feedback forms were filled out were also extracted. Results: A total of 662 feedback forms were completed, showing a significant improvement (p ˂ 0.0001). The majority of the medical personnel who received the most negative feedback were the pre-hospital staff. A significant increase was revealed in the number of feedbacks with reference to mismanagement of backboard spinal fixation, of the pre-hospital staff, in 2012-2013 compared to 2017-2018 (p < 0.001). Improvement in reducing the time of treatment in the field was also revealed, from 15.2 ± 8.3 min in 2012-2013 to 13.4 ± 7.9 min in 2017-2018. Conclusion: The findings show that feedback improves the treatment of injured patients. Furthermore, constantly monitoring the quality of treatment provided by the trauma team is vital for improvement.

Effectiveness and quality of peer video feedback in health professions education: A systematic review

OBJECTIVES

This review aimed to evaluate the effectiveness of peer video feedback (PVF) on healthcare students' reactions and learning outcomes against other feedback methods (e.g., expert- or self-video feedback). It also synthesized the characteristics of PVF within health professions education to identify its effective elements.

DATA SOURCES

Seven databases were systematically searched to identify relevant studies, including CENTRAL, CINAHL, ERIC, Embase, PubMed, PsycINFO, and Scopus.

REVIEW METHODS

This review was conducted based on the PRISMA Statement Guidelines. Reviewers independently extracted data from the included articles and assessed the risk of bias and quality of the studies. The effectiveness of PVF on students' reactions, learning, and quality of peer feedback was summarized.

RESULTS

A total of 22 articles were included. Results showed PVF was a helpful learning tool, and students were satisfied with its overall learning experience. PVF demonstrated its positive effect on skill-based learning. The top concern was its quality (accuracy and content), ascribing to peers' limited knowledge, expertise, or feedback experience.

CONCLUSIONS

This review affirmed the potential effect of PVF on skill-based learning but revealed students' ambivalent feelings towards its quality. Six effective elements were proposed for its best practice. Future studies are needed to investigate further these proposed elements and how they mediate the educational effects of PVF.

Evaluating bowel enterotomy closures in simulated deep body cavities using the reversing half-hitch alternating post and square knots: a randomized controlled trial

Background

Square knots can be difficult to construct in deep body cavities. The reversing half-hitch alternating post (RHAP) surgical knot has noninferior tensile strength and performance characteristics in deep body cavities. We compared the enterotomy repairs of novice learners in simulated deep body cavities using RHAP versus square knots after proficiency-based training.

Methods

Undergraduate students were randomized to RHAP (n = 10) or square knot (n = 10) groups and trained to defined proficiency. They then performed hand-sewn enterotomy repairs of cadaveric porcine small bowels on flat surfaces and in simulated deep body cavities. We recorded time to knot-tying proficiency and to enterotomy repair, and burst pressures for the repair.

Results

Mean time-to-proficiency in knot tying was equivalent between the RHAP and square knot groups (23 [standard deviation (SD) 3] v. 21 [SD 2] min, p = 0.33). Mean time for enterotomy repair in deep cavities was shorter for the RHAP group (16 [SD 2] min v. 21 [SD 1] min, p = 0.02). Mean burst pressures for enterotomy repair were equivalent on flat surfaces (128 [SD 41] v. 101 [SD 36] mm Hg, p = 0.31), and were significantly higher for the RHAP group in simulated deep body cavities (32 [SD 13] v. 105 [SD 37] mm Hg, p = 0.05).

Conclusion

The RHAP knots appear to have superior performance versus square knots when tied in a deep body cavity by novice learners. Future work should focus on demonstrating the clinical relevance and broad utility of the RHAP knot in abdominal surgery. Both knot types should be taught to novice learners.

Using Peer Feedback to Promote Clinical Excellence in Hospital Medicine

Hospitalists provide a significant amount of direct clinical care in both academic and community hospitals. Peer feedback is a potentially underutilized and low resource method for improving clinical performance, which lends itself well to the frequent patient care handoffs that occur in the practice of hospital medicine. We review current literature on peer feedback to provide an overview of this performance improvement tool, briefly describe its incorporation into multi-source clinical performance appraisals across disciplines, highlight how peer feedback is currently used in hospital medicine, and present practical steps for hospital medicine programs to implement peer feedback to foster clinical excellence among their clinicians.

Comparison of Personal Video Technology for Teaching and Assessment of Surgical Skills

BACKGROUND

Improvements in personal technology have made video recording for teaching and assessment of surgical skills possible.

OBJECTIVE

This study compared 5 personal video-recording devices based on their utility (image quality, hardware, mounting options, and accessibility) in recording open surgical procedures.

METHODS

Open procedures in a simulated setting were recorded using smartphones and tablets (MOB), laptops (LAP), sports cameras such as GoPro (SC), single-lens reflex cameras (DSLR), and spy camera glasses (SPY). Utility was rated by consensus between 2 investigators trained in observation of technology using a 5-point Likert scale (1, poor, to 5, excellent).

RESULTS

A total of 150 hours of muted video were reviewed with a minimum 1 hour for each device. Image quality was good (3.8) across all devices, although this was influenced by the device-mounting requirements (4.2) and its proximity to the area of interest. Device hardware (battery life and storage capacity) was problematic for long procedures (3.8). Availability of devices was high (4.2).

CONCLUSIONS

Personal video-recording technology can be used for assessment and teaching of open surgical skills. DSLR and SC provide the best images. DSLR provides the best zoom capability from an offset position, while SC can be placed closer to the operative field without impairing sterility. Laptops provide best overall utility for long procedures due to video file size. All devices require stable recording platforms (eg, bench space, dedicated mounting accessories). Head harnesses (SC, SPY) provide opportunities for "point-of-view" recordings. MOB and LAP can be used for multiple concurrent recordings.