Remote assessment via video evaluation (RAVVE): a pilot study to trial video-enabled peer feedback on clinical performance

Exploring residents and supervisors' workplace learning needs during postgraduate medical education

Objectives

To identify the main enablers and challenges for workplace learning during postgraduate medical education among residents and their supervisors involved in training hospital specialists across different medical specialties and clinical teaching departments.

Methods

A qualitative explorative study using semi-structured focus group interviews was employed. A purposeful sampling method was utilized to invite participants who were involved in postgraduate medical education for hospital specialist medicine at two universities. Hospital physicians in training, also called residents (n=876) and supervisors (n=66), were invited by email to participate. Three focus groups were organized: two with residents and one with supervisors. Due to the COVID-19 pandemic rules prohibiting real group meetings, these focus groups were online and asynchronous. The data was analyzed following an inductive thematic analysis.

Results

The following overarching themes were identified: 1) the dual learning path, which balances working in the hospital and formal courses, 2) feedback, where quality, quantity, and frequency are discussed, and 3) learning support, including residents' self-directed learning, supervisors' guidance, and ePortfolio support.

Conclusions

Different enablers and challenges for postgraduate medical education were identified. These results can guide all stakeholders involved with workplace learning to develop a better understanding of how workplace learning can be optimized to improve the postgraduate medical education experience. Future studies could focus on confirming the results of this study in a broader, perhaps international setting and exploring strategies for aligning residencies to improve quality.

An exploration of "real time" assessments as a means to better understand preceptors' judgments of student performance

Clinical supervisors are known to assess trainee performance idiosyncratically, causing concern about the validity of their ratings. The literature on this issue relies heavily on retrospective collection of decisions, resulting in the risk of inaccurate information regarding what actually drives raters' perceptions. Capturing in-the-moment information about supervisors' impressions could yield better insight into how to intervene. The purpose of this study, therefore, was to gather "real-time" judgments to explore what drives preceptors' judgments of student performance. We performed a prospective study in which physicians were asked to adjust a rating scale in real-time while watching two video-recordings of trainee clinical performances. Scores were captured in 1-s increments, examined for frequency, direction, and magnitude of adjustments, and compared to assessors' final entrustability judgment as measured by the modified Ottawa Clinic Assessment Tool. The standard deviation in raters' judgment was examined as a function of time to determine how long it takes impressions to begin to vary. 20 participants viewed 2 clinical vignettes. Considerable variability in ratings was observed with different behaviours triggering scale adjustments for different raters. That idiosyncrasy occurred very quickly, with the standard deviation in raters' judgments rapidly increasing within 30 s of case onset. Particular moments appeared to generally be influential, but their degree of influence still varied. Correlations between the final assessment and (a) score assigned upon first adjustment of the scale, (b) upon last adjustment, and (c) the mean score, were r = 0.13, 0.32, and 0.57 for one video and r = 0.30, 0.50, and 0.52 for the other, indicating the degree to which overall impressions reflected accumulation of raters' idiosyncratic moment-by-moment observations. Our results demonstrated that variability in raters' impressions begins very early in a case presentation and is associated with different behaviours having different influence on different raters. More generally, this study outlines a novel methodology that offers a new path for gaining insight into factors influencing assessor judgments.

Video-Assisted Peer Teaching for Surgical Skills Training - Innovative Potential for the Medical Curriculum and Beyond: A Randomized Controlled Trial

OBJECTIVE

Hygienic healthcare standards are essential for avoiding hospital infections. However, medical students and staff lack training in this field, which may be due to high personnel resources of present educational approaches. Thus, there is an urgent need for a novel and efficient approach. Aim of the study is to compare a newly developed video-assisted peer feedback (VAPF) method for teaching wound dressings to the traditional teaching method with qualified instructor feedback (QIF) with respect to essential learning outcomes.

DESIGN, SETTING AND PARTICIPANTS

In this randomized controlled noninferiority trial, 251 medical undergraduates were randomly assigned to one of two interventions (QIF n = 127; VAPF n = 124). In QIF, participants received feedback from a qualified instructor. In VAPF, participants video-recorded each other while performing a wound dressing and gave each other feedback assisted by a standardized checklist. Outcome measures were participants' score in an objective structured practical examination (OSPE) and a written exam after the course.

RESULTS

Noninferiority of VAPF (n = 123) compared to QIF (n = 127) was confirmed for both OSPE (QIF: 8.83 ± 1.30; VAPF: 8.88 ± 1.04; mean difference -0.04, 95% CI -0.34 to 0.25) and written exam (QIF: 8.99 ± 1.06; VAPF: 9.14 ± 1.05; mean difference -0.15, 95% CI -0.41 to 0.12).

CONCLUSIONS

VAPF is a cost-efficient and viable alternative to QIF commonly used in medical education. It provides comparable training outcomes to the traditional training method with lower personnel investment. VAPF is a promising educational method for improving essential clinical competencies.

Effectiveness and quality of peer video feedback in health professions education: A systematic review

OBJECTIVES

This review aimed to evaluate the effectiveness of peer video feedback (PVF) on healthcare students' reactions and learning outcomes against other feedback methods (e.g., expert- or self-video feedback). It also synthesized the characteristics of PVF within health professions education to identify its effective elements.

DATA SOURCES

Seven databases were systematically searched to identify relevant studies, including CENTRAL, CINAHL, ERIC, Embase, PubMed, PsycINFO, and Scopus.

REVIEW METHODS

This review was conducted based on the PRISMA Statement Guidelines. Reviewers independently extracted data from the included articles and assessed the risk of bias and quality of the studies. The effectiveness of PVF on students' reactions, learning, and quality of peer feedback was summarized.

RESULTS

A total of 22 articles were included. Results showed PVF was a helpful learning tool, and students were satisfied with its overall learning experience. PVF demonstrated its positive effect on skill-based learning. The top concern was its quality (accuracy and content), ascribing to peers' limited knowledge, expertise, or feedback experience.

CONCLUSIONS

This review affirmed the potential effect of PVF on skill-based learning but revealed students' ambivalent feelings towards its quality. Six effective elements were proposed for its best practice. Future studies are needed to investigate further these proposed elements and how they mediate the educational effects of PVF.

Using Peer Feedback to Promote Clinical Excellence in Hospital Medicine

Hospitalists provide a significant amount of direct clinical care in both academic and community hospitals. Peer feedback is a potentially underutilized and low resource method for improving clinical performance, which lends itself well to the frequent patient care handoffs that occur in the practice of hospital medicine. We review current literature on peer feedback to provide an overview of this performance improvement tool, briefly describe its incorporation into multi-source clinical performance appraisals across disciplines, highlight how peer feedback is currently used in hospital medicine, and present practical steps for hospital medicine programs to implement peer feedback to foster clinical excellence among their clinicians.