General Surgery Residents' Perception of Feedback: We Can Do Better

How well do faculty do in providing general surgery EPA feedback?

INTRODUCTION

Entrustable Professional Activities (EPAs) provide a framework for competency-based assessment in surgery. EPA descriptions include observable behaviors by trainees at progressive levels of autonomy. The American Board of Surgery (ABS) required all General Surgery (GS) residency programs to implement assessment of 18 EPAs at the beginning of academic year 2023-2024. Microassessments provide formative self-reflection by the resident and feedback by faculty upon completion of the EPA. These frequent assessments culminate in a resident performance profile utilized by the trainee for formative growth and the clinical competency committee for summative feedback. Assessor free text comments are an opportunity to provide meaningful, constructive feedback to residents. Our aim was to analyze comments provided by faculty to residents in terms of their alignment with EPA descriptors and provision of actionable feedback.

METHODS

A total of 540 ​GS EPA assessments for inguinal hernia, gallbladder disease, appendicitis, trauma, and surgical consultation were evaluated from 6/2021-12/2022. We assessed free text EPA comments from faculty compared to EPA behavior descriptions for alignment with the selected EPA level of entrustment. The comments were judged on a binary scale of "Align" vs "Not Align" by two independent evaluators, with a third evaluator to address discordance. Comments were then evaluated for resident behavioral descriptions, suggestions for improvement, and positive or negative feedback.

RESULTS

Approximately 77 ​% of EPA microassessments had alignment between level of autonomy and free text feedback. A common example of feedback discordant with level of autonomy was rating a trainee at an intraoperative level 4 (independent practice) with comments such as "required some guidance with retrocecal case and upsizing port." Based on behavior descriptions this would be a level 3 (indirect supervision). Approximately 88 ​% of feedback contained positive comments with minimal negative feedback (e.g., "this did not go well."). Actionable feedback including "work on optimization of retracting hand" or "continue to work clamp/tie technique and square off each knot" was present in 28.3 ​% of feedback.

CONCLUSIONS

The majority of faculty provide feedback that is aligned with the behavioral anchors of the EPAs assessed, but frequently did not provide actionable feedback to the resident regarding how to advance to the next level of entrustment. EPA entrustment behaviors provide a framework for the development of practice-ready behaviors, and if assessors anchor their feedback in the behaviors for a given entrustment level and project how a resident could proceed to the next level, they can provide a clear trajectory for skill development. Faculty development should focus on improving the frequency of actionable free text feedback, outlining how residents can advance in the future.

Evaluation of Educational Feedback in Urology Training: A Survey-Based Assessment of Trainees and Program Directors

OBJECTIVE

Our objective was to evaluate current satisfaction with the feedback provided during post-graduate urological training, including the quality and frequency of feedback, with participants consisting of both trainees and program directors. Additionally, we aimed to identify areas for future improvement in resident and fellow-level urological training.

METHODS

Graduating residents, fellows, and program directors from accredited residency/fellowship programs in the United States were surveyed. A total of 575 surveys were sent out. Information on feedback frequency, quality, form, and satisfaction was collected using applicable multiple-choice responses and a five-point Likert scale. An open-ended question gathered suggestions for improving current feedback processes. A chi-square test of independence was used to compare the responses to individual questions.

RESULTS

Ninety-two respondents answered our survey: 22 residents, 18 fellows, 25 residency program directors (PDs), and 27 fellowship PDs. The distribution of age, race, and gender categories was not significantly different between PDs and trainees. However, there was a significant difference in their subspecialties and American Urological Association (AUA) sections. The majority of fellowship PDs, residency PDs, fellows, and residents (88 total) reported verbal feedback as the predominant method within their practice. This was followed by written (68 total), electronic (54 total), and app-based feedback (19 total).

CONCLUSION

Our study suggests that there may be a need for ongoing improvement or standardization of feedback mechanisms in the field of urological training and highlights the perceived discrepancies between learners and educators.

[Goal-setting in Plastic Surgery using the SMART principle]

INTRODUCTION

The process of continuous acquisition of surgical expertise is a key element in registrar training. The principle of active, self-directed learning, which is regularly applied in medical school, can also be used to gain surgical expertise in registrar training.

METHODS

Surgical training can be compared to acquiring expertise in music or sport, where both intellectual and manual capabilities are required. Specific training principles, including analysis, goal setting, practice and reflection on the process, are commonly encountered in these fields. Smart goal setting is preferred in order to ensure compliance and a successful strategy. This can also be used in plastic surgery training.

APPLICATION

Surgical principles as well as partial or complete surgical procedures can be practiced using the smart principle. The fragmentation of a larger task into smaller units allows rapid acquisition of surgical expertise without impeding patient safety.

DISCUSSION

Surgical training today happens in a setting caught between economic and medicolegal challenges. The reduction of case load requires simulation practice as well as self-directed learning - which has been shown to improve outcomes. The smart principle allows distinct goal setting which can improve compliance of the student and increase success. As this form of learning is not intuitive to all individuals, supervisors and mentor can support trainees in the acquisition of surgical expertise using this principle.

Quality Metrics and Indicators for Surgical Training: A Scoping Review

BACKGROUND

Surgical training quality is critical to ensure that trainees receive adequate preparation to perform surgical procedures independently and that patients receive safe, effective, and high-quality care. Numerous surgical training quality indicators have been proposed, investigated and implemented. However, the existing evidence base for these indicators is limited, with most studies originating from English-speaking, high-income countries.

OBJECTIVES

This scoping review aimed to identify the range of quality indicators that have been proposed and evaluated in the literature, and to critically evaluate the existing evidence base for these indicators.

METHODS

A systematic literature search was conducted using MEDLINE and Embase databases to identify studies reporting on surgical training quality indicators. A total of 68 articles were included in the review.

RESULTS

Operative volume is the most commonly cited indicator and has been investigated for its effects on trainee exam performance and career progression. Other indicators include operative diversity, workplace-based assessments, regular evaluation and feedback, academic achievements, formal teaching, and learning agreements, and direct observation of procedural skills. However, these indicators are largely based on qualitative analyses and expert opinions and have not been validated quantitatively using clear outcome measures for trainees and patients.

CONCLUSIONS

Future research is necessary to establish evidence-based indicators of high-quality surgical training, including in low-resource settings. Quantitative and qualitative studies are required to validate existing indicators and to identify new indicators that are relevant to diverse surgical training environments. Lastly, any approach to surgical training quality must prioritize the benefit to both trainees and patients, ensuring training success, career progression, and patient safety.

Intraoperative feedback: A survey of surgical trainees' perspective

INTRODUCTION

Intraoperative feedback can be associated with improved surgical performance. Quality feedback can reduce the time required by trainees to achieve proficiency in psychomotor skills. Operative training time has become increasingly limited, and it has become imperative to use surgical training time effectively.

AIM

In this survey, we assessed trainees' perspectives of intraoperative feedback. We included several aspects of feedback including its occurrence, quality, and potential barriers.

METHODS

All surgical trainees in a single centre were invited to complete an electronic questionnaire. Participants were anonymised. We summarised data using descriptive statistics.

RESULTS

Most trainees (85%) reported they had the opportunity to share their training goals with trainers. Just under three-quarters of trainees felt they always or sometimes got timely feedback. Only half of the trainees were signposted to feedback and 23% felt feedback was not part of their department's culture. Half of the trainees did not always feel comfortable asking for feedback from their trainers stating their reasons as fear of criticism, lack of time and competing clinical commitments.

CONCLUSION

There is no denying the importance of feedback on operative performance, however, this survey shows that many of the pillars of quality feedback are poorly adhered to.

Exploring the Quality of Narrative Feedback Provided to Residents During Ambulatory Patient Care in Medicine and Surgery

OBJECTIVES

The transition to competency-based medical education (CBME) has increased the volume of residents' assessment data; however, the quality of the narrative feedback is yet to be used as feedback-on-feedback for faculty. Our objectives were (1) to explore and compare the quality and content of narrative feedback provided to residents in medicine and surgery during ambulatory patient care and (2) to use the Deliberately Developmental Organization framework to identify strengths, weaknesses, and opportunities to improve quality of feedback within CBME.

METHODS

We conducted a mixed convergent methods study with residents from the Departments of Surgery (DoS; n = 7) and Medicine (DoM; n = 9) at Queen's University. We used thematic analysis and the Quality of Assessment for Learning (QuAL) tool to analyze the content and quality of narrative feedback documented in entrustable professional activities (EPAs) assessments for ambulatory care. We also examined the association between the basis of assessment, time to provide feedback, and the quality of narrative feedback.

RESULTS

Forty-one EPA assessments were included in the analysis. Three major themes arose from thematic analysis: Communication, Diagnostics/Management, and Next Steps. Quality of the narrative feedback varied; 46% had sufficient evidence about residents' performance; 39% provided a suggestion for improvement; and 11% provided a connection between the suggestion and the evidence. There were significant differences between DoM and DoS in quality of feedback scores for evidence (2.1 [1.3] vs. 1.3 [1.1]; p < 0.01) and connection (0.4 [0.5] vs. 0.1 [0.3]; p  =  0.04) domains of the QuAL tool. Feedback quality was not associated with the basis of assessment or time taken to provide feedback.

CONCLUSION

The quality of the narrative feedback provided to residents during ambulatory patient care was variable with the greatest gap in providing connections between suggestions and evidence about residents' performance. There is a need for ongoing faculty development to improve the quality of narrative feedback provided to residents.