What is feedback in clinical education?

Enhancing clinical skills in pediatric trainees: a comparative study of ChatGPT-assisted and traditional teaching methods

BACKGROUND

As artificial intelligence (AI) increasingly integrates into medical education, its specific impact on the development of clinical skills among pediatric trainees needs detailed investigation. Pediatric training presents unique challenges which AI tools like ChatGPT may be well-suited to address.

OBJECTIVE

This study evaluates the effectiveness of ChatGPT-assisted instruction versus traditional teaching methods on pediatric trainees' clinical skills performance.

METHODS

A cohort of pediatric trainees (n = 77) was randomly assigned to two groups; one underwent ChatGPT-assisted training, while the other received conventional instruction over a period of two weeks. Performance was assessed using theoretical knowledge exams and Mini-Clinical Evaluation Exercises (Mini-CEX), with particular attention to professional conduct, clinical judgment, patient communication, and overall clinical skills. Trainees' acceptance and satisfaction with the AI-assisted method were evaluated through a structured survey.

RESULTS

Both groups performed similarly in theoretical exams, indicating no significant difference (p > 0.05). However, the ChatGPT-assisted group showed a statistically significant improvement in Mini-CEX scores (p < 0.05), particularly in patient communication and clinical judgment. The AI-teaching approach received positive feedback from the majority of trainees, highlighting the perceived benefits in interactive learning and skill acquisition.

CONCLUSION

ChatGPT-assisted instruction did not affect theoretical knowledge acquisition but did enhance practical clinical skills among pediatric trainees. The positive reception of the AI-based method suggests that it has the potential to complement and augment traditional training approaches in pediatric education. These promising results warrant further exploration into the broader applications of AI in medical education scenarios.

Feedback practices in undergraduate clinical teaching in Sri Lanka - a qualitative study

BACKGROUND

Feedback is integral to medical education, enabling students to improve their knowledge, skills, and attitudes. Feedback practices may vary according to prevalent cultural and contextual factors. This study aimed to explore how feedback is conceptualized and practised in the clinical education of medical students in Sri Lanka.

METHODS

The study was conducted in three medical schools and affiliated hospitals that represent the cultural diversity of Sri Lanka. Purposive sampling was utilized to recruit clinical teachers and students who would provide rich information for the study. The study had three components: an observation study, interviews with clinical teachers and focus group discussions with clinical students. During the observation study, video recording was used as a data collection tool to observe feedback in real-life clinical teaching/learning settings. A constructivist grounded theory approach was adapted for analysis to explore current practices and perceptions inductively.

RESULTS

Feedback was conceptualised as spontaneous unidirectional provision of information for the improvement of students. It was often provided in public settings and in student groups. Error correction was the primary focus of feedback, but both teachers and students desired a balanced approach with reinforcement and reflection. Although the direct approach to corrective feedback was found beneficial for student learning, participants agreed that harsh feedback was to be avoided. The hierarchical culture and lack of programmed feedback in the curricula influenced feedback practices, suggesting the need for modification.

CONCLUSIONS

This study highlighted feedback practices in the local context, emphasizing the need to address the hierarchical gap in clinical settings, balance reinforcement and correction, and promote dialogue and reflection in the feedback processes. The findings will help clinical teachers from both the global south as well as the global north to recognize cultural and contextual differences in providing feedback.

Exploring Feedback Mechanics during Experiential Learning in Pharmacy Education: A Scoping Review

(1) Background: This scoping review aims to explore the literature on feedback for pharmacy students during experiential learning, with a focus on identifying the modes of delivery of feedback and the perceived impact of feedback on student learning outcomes. (2) Methods: The scoping review was conducted in accordance with the Joanna Briggs Institute (JBI) methodology and reported following the Preferred Reporting Items for Systematic Reviews Extension for Scoping Reviews (PRISMA-ScR) guidelines. PubMed, Web of Science, Embase, EBSCO, ERIC, and ProQuest Central were searched electronically from their inception until the end of February 2023 using a combination of keywords and MeSH terms related to feedback, pharmacy education, and student learning outcomes. Data were synthesized narratively. (3) Results: This review included 13 studies published between 2008 and 2022. Almost half of the included studies were conducted in the USA (n = 6, 46%) and reported the perspective of undergraduate pharmacy students (n = 6, 46%). Verbal feedback was the most common mode of feedback delivery (n = 6, 46%). The enablers of effective feedback included timely feedback (n = 6, 46%), feedback provided in a goal-oriented and objective manner (n = 5, 40%), and student-specific feedback (n = 4, 30%). On the other hand, the most common impediments to feedback efficacy were providing extremely positive feedback and lack of constructive criticism. (4) Conclusions: Our findings highlight the importance of feedback model implementation in pharmacy education and preceptor training programs to ensure effective and quality feedback to pharmacy students.

Improving feedback literacy in a primary care rotation

Primary care education is a unique clinical experience for medical students. It is community-based and provides an opportunity for students to learn consultation skills with multiple sources of workplace-based feedback. Meaningful and demonstrable utilisation of this feedback by students remains an educational challenge. We showcase achievable changes to educational tasks in an established curriculum, which aim to improve student feedback literacy and create a feedback loop which improves on previous provision of unidirectional, terminal feedback. The changes have been well-received, with student and educator engagement being positive. Students have demonstrated critical reflection on feedback, and development in consultation and clinical reasoning skills.

Designing feedback processes in the workplace-based learning of undergraduate health professions education: a scoping review

BACKGROUND

Feedback processes are crucial for learning, guiding improvement, and enhancing performance. In workplace-based learning settings, diverse teaching and assessment activities are advocated to be designed and implemented, generating feedback that students use, with proper guidance, to close the gap between current and desired performance levels. Since productive feedback processes rely on observed information regarding a student's performance, it is imperative to establish structured feedback activities within undergraduate workplace-based learning settings. However, these settings are characterized by their unpredictable nature, which can either promote learning or present challenges in offering structured learning opportunities for students. This scoping review maps literature on how feedback processes are organised in undergraduate clinical workplace-based learning settings, providing insight into the design and use of feedback.

METHODS

A scoping review was conducted. Studies were identified from seven databases and ten relevant journals in medical education. The screening process was performed independently in duplicate with the support of the StArt program. Data were organized in a data chart and analyzed using thematic analysis. The feedback loop with a sociocultural perspective was used as a theoretical framework.

RESULTS

The search yielded 4,877 papers, and 61 were included in the review. Two themes were identified in the qualitative analysis: (1) The organization of the feedback processes in workplace-based learning settings, and (2) Sociocultural factors influencing the organization of feedback processes. The literature describes multiple teaching and assessment activities that generate feedback information. Most papers described experiences and perceptions of diverse teaching and assessment feedback activities. Few studies described how feedback processes improve performance. Sociocultural factors such as establishing a feedback culture, enabling stable and trustworthy relationships, and enhancing student feedback agency are crucial for productive feedback processes.

CONCLUSIONS

This review identified concrete ideas regarding how feedback could be organized within the clinical workplace to promote feedback processes. The feedback encounter should be organized to allow follow-up of the feedback, i.e., working on required learning and performance goals at the next occasion. The educational programs should design feedback processes by appropriately planning subsequent tasks and activities. More insight is needed in designing a full-loop feedback process, in which specific attention is needed in effective feedforward practices.

Development of a validated assessment tool for medical students using simulated patients: an 8-year panel survey

BACKGROUND

The use of simulated patients (SPs) to assess medical students' clinical performance is gaining prominence, underscored by patient safety perspective. However, few reports have investigated the validity of such assessment. Here, we examined the validity and reliability of an assessment tool that serves as a standardized tool for SPs to assess medical students' medical interview.

METHODS

This longitudinal survey was conducted at Keio University School of Medicine in Japan from 2014 to 2021. To establish content validity, the simulated patient assessment tool (SPAT) was developed by several medical education specialists from 2008 to 2013. A cohort of 36 SPs assessed the performance of 831 medical students in clinical practice medical interview sessions from April 2014 to December 2021. The assessment's internal structure was analyzed using descriptive statistics (maximum, minimum, median, mean, and standard deviation) for the SPAT's 13 item total scores. Structural validity was examined with exploratory factor analysis, and internal consistency with Cronbach's alpha coefficients. The mean SPAT total scores across different SPs and scenarios were compared using one way analysis of variance (ANOVA). Convergent validity was determined by correlating SPAT with the post-clinical clerkship obstructive structured clinical examination (post-CC OSCE) total scores using Pearson's correlation coefficient.

RESULTS

Of the 831 assessment sheets, 36 with missing values were excluded, leaving 795 for analysis. Thirty-five SPs, excluding one SP who quit in 2014, completed 795 assessments, for a response rate of 95.6%. Exploratory factor analysis revealed two factors, communication and physician performance. The overall Cronbach's alpha coefficient was 0.929. Significant differences in SPAT total scores were observed across SPs and scenarios via one-way ANOVA. A moderate correlation (r =.212, p <.05) was found between SPAT and post-CC OSCE total scores, indicating convergent validity.

CONCLUSIONS

Evidence for the validity of SPAT was examined. These findings may be useful in the standardization of SP assessment of the scenario-based clinical performance of medical students.

National pilot of entrustable professional activities in pathology residency training

Entrustable professional activities (EPAs) are observable clinical skills and/or procedures that have been introduced into medical education at the student and resident levels in most specialties to determine readiness to advance into residency or independent practice, respectively. This publication describes the process and outcomes of a pilot study looking at the feasibility of using two anatomic pathology and two clinical pathology EPAs in pathology residency in 6 pathology residency programs that volunteered for the study. Faculty development on EPAs and their assessment was provided to pilot program faculty, and EPA assessment tools were developed and used by the pilot programs. Pre- and post-study surveys were given to participating residents, faculty, and program directors to gauge baseline practices and to gather feedback on the EPA implementation experience. Results demonstrated overall good feasibility in implementing EPAs. Faculty acceptance of EPAs varied and was less than that of program directors. Residents reported a significant increase in the frequency with which faculty provided formative assessments that included specific examples of performance and specific ways to improve, as well as increased frequency with which faculty provided summative assessments that included specific ways to improve. EPAs offered the most benefit in setting clear expectations for performance of each task, for providing more specific feedback to residents, and in increasing Program director's understanding of resident strengths abilities and weaknesses.

Teaching Before, During, and After a Surgical Resuscitation

Teaching during a surgical resuscitation can be difficult due to the infrequency of these events. Furthermore, when these events do occur, the trainee can experience cognitive overload and an overwhelming amount of stress, thereby impairing the learning process. The emergent nature of these scenarios can make it difficult for the surgical educator to adequately teach. Repeated exposure through simulation, role play, and "war games" are great adjuncts to teaching and preparation before crisis. However, surgical educators can further enhance the knowledge of their trainees during these scenarios by using tactics such as talking out loud, targeted teaching, and debriefing.

Peer feedback: A tool to improve PA, NP, and physician collaboration and clinical practice

ABSTRACT

The increasing use of physician associates/assistants (PAs) and NPs on clinical teams creates an opportunity for colleagues to provide feedback that can drive growth and improvement in PA and NP clinical skills. Through a needs assessment, month-long pilot, and buy-in from our academic hospital medicine group, we initiated a peer feedback platform that sought to highlight individual faculty strengths, facilitate thoughtful and constructive feedback, and create effective collaboration among colleagues. Three months after implementation, results indicated that the platform provided important information on clinical skills and identified areas for growth and opportunities for additional training. Peer feedback can also improve working relationships among PAs, NPs, and physicians by highlighting the clinical skill sets and value of all team members.

E-portfolio as an effective tool for improvement of practitioner nurses' clinical competence

BACKGROUND

Clinical competence is essential for providing effective patient care. Clinical Governance (CG) is a framework for learning and assessing clinical competence. A portfolio is a work-placed-based tool for monitoring and reflecting on clinical practice. This study aimed to investigate the effect of using an e-portfolio on the practitioner nurses' competence improvement through the CG framework.

METHODS

This was a quasi-experimental study with 30 nurses in each intervention and control group. After taking the pretests of knowledge and performance, the participants attended the in-person classes and received the educational materials around CG standards for four weeks. In addition, nurses in the intervention group received the links to their e-portfolios individually and filled them out. They reflected on their clinical practice and received feedback. Finally, nurses in both groups were taken the post-tests.

RESULTS

Comparing the pre-and post-test scores in each group indicated a significant increase in knowledge and performance scores. The post-test scores for knowledge and performance were significantly higher in the intervention group than in the control one, except for the initial patient assessment.

CONCLUSION

This study showed that the e-portfolio is an effective tool for the improvement of the nurses' awareness and performance in CG standards. Since the CG standards are closely related to clinical competencies, it is concluded that using portfolios effectively improves clinical competence in practitioner nurses.

The 'Kidney' model for optimising feedback in undergraduate clinical communication: A meta-ethnographic systematic review

OBJECTIVES

Feedback frameworks/models focus on certain aspects of the feedback process, but a coherent and systematic model is lacking. A meta-ethnography was conducted to identify and synthesise guidance for optimising feedback interactions in undergraduate clinical communication simulations.

METHODS

A systematic search of 4 electronic databases and grey literature was conducted. Following Noblit and Hare's seven phases for conducting meta-ethnography, key themes and concepts were synthesised to provide new interpretations of components in effective feedback interactions.

RESULTS

373 publications were identified and 14 included for the final synthesis, which informed the development of a new Feedback Kidney Model. The Model illustrates the interconnections of various components that allow for effective feedback interactions. The main processes include preparation, proactivity, analysis and feedback information, reception and response, and influencing factors.

CONCLUSIONS

This meta-ethnography moves beyond providing an up-to-date synthesis of feedback guidance to proposing the brand-new Feedback Kidney Model, which can guide medical education and future research into how feedback is co-constructed and utilised to promote learning.

PRACTICE IMPLICATIONS

Clinical communication should incorporate meta-cognitive training and using this Model will help students better utilise on-site face-to-face feedback to enhance their learning and improve future communication with patients.

A Qualitative Textual Analysis of Feedback Comments in ePortfolios: Quality and Alignment with the CanMEDS Roles

Introduction

Competency-based education requires high-quality feedback to guide students' acquisition of competencies. Sound assessment and feedback systems, such as ePortfolios, are needed to facilitate seeking and giving feedback during clinical placements. However, it is unclear whether the written feedback comments in ePortfolios are of high quality and aligned with the current competency focus. Therefore, this study investigates the quality of written feedback comments in ePortfolios of healthcare students, as well as how these feedback comments align with the CanMEDS roles.

Methods

A qualitative textual analysis was conducted. 2,349 written feedback comments retrieved from the ePortfolios of 149 healthcare students (specialist medicine, general practice, occupational therapy, speech therapy and midwifery) were analysed retrospectively using deductive content analysis. Two structured categorisation matrices, one based on four literature-derived feedback quality criteria (performance, judgment, elaboration and improvement) and another one on the seven CanMEDS roles (Medical Expert, Communicator, Collaborator, Leader, Health Advocate, Scholar and Professional), guided the analysis.

Results

The minority of the feedback comments (n = 352; 14.9%) could be considered of high quality because they met all four quality criteria. Most feedback comments were of moderate quality and met only two to three quality criteria. Regarding the CanMEDS roles, the Medical Expert role was most frequently represented in the feedback comments, as opposed to the roles Leader and Health Advocate.

Discussion

The results highlighted that providing high-quality feedback is challenging. To respond to these challenges, it is recommended to set up individual and continuous feedback training.

Enhancing learning experiences in pre-clinical restorative dentistry: the impact of virtual reality haptic simulators

BACKGROUND

Utilization of Virtual Reality haptic simulation (VRHS) to aid in the training of various pre-clinical skills is of recent interest. The aim of this study was to evaluate the impact of VRHS in restorative dentistry on the learning experiences and perceptions of dental students.

METHODS

An interventional study design was utilized to recruit third year students. All participants provided informed consents and were randomly divided into two groups. Group 1: Initially performed a Class I cavity preparation with the VRHS, followed by the same exercise using the phantom head/ acrylic typodont teeth in a conventional simulation environment (CSE). Group 2: Initially performed Class I preparations in a CSE, followed by the same exercise using VRHS. Both groups performed the exercises on a lower right first molar. To understand students' perception, an online questionnaire was circulated. Data analysis involved Chi-square tests, independent t-tests and Mann-Whitney U-tests using the R statistical environment package.

RESULTS

A total of 23 dental students participated in this study. Although student's perceptions were similar in both groups, a strong agreement that VRHS training might be used to supplement standard pre-clinical training was noted. Advancements to the VRHS hardware and software are required to bridge the gap and provide a smooth transition to clinics.

CONCLUSION

Novice dental students generally perceived VRHS as a useful tool for enhancing their manual dexterity. Dental institutions should endorse virtual reality technology with caution, ensuring a planned integration into the curriculum to optimize benefit. Feedback is pivotal to effective learning in simulation-based education, and the triangulation of feedback could serve as a powerful aid to maximize the learning experience.

Teaching rotary root canal instrumentation using a Peyton teaching approach

INTRODUCTION

The aim of the study was two evaluate the effectiveness of a Peyton teaching approach for rotary root canal instrumentation, in comparison to the traditional "see one-do one" method.

MATERIAL AND METHODS

Forty undergraduate students were randomly divided into two groups (n = 20). Students of the first group (G1) were taught how to use rotary instrumentation using a modified Peyton method, whilst the second group (G2) watched a teaching video (30 min) on the same subject. Both groups instrumented a plastic block and subsequently both mesial canals of an extracted lower molar. The quality of the root canal instrumentation was analysed by 2 blinded observers on video recordings and x-rays.

RESULTS

Interobserver correlation was 0.917 (p < .0005; Pearson) for the assessment of the video recordings; students of G1 received significantly more total points (83.55 ± 6.82 points) compared to G2 (69.76 ± 13.82) (p = .001; t-test), the gender had no significant effect on the overall results (p = .444; two-way ANOVA). Significant differences were detected for the categories "initial scouting," "coronal enlargement," "glide path preparation," "preparation using X2 file," "preparation using X3 file" (p < .05; t-test) as well as for the subcategories "sequence of rinse, recapitulation, rinse" (p = .001; t-test) and "recapitulation" (p < .002; t-test). No differences between groups were observed for the radiographic evaluation with respect to working length and canal straightening.

CONCLUSION

Teaching rotary instrumentation by using the Peyton approach resulted in improved performance of undergraduate students assessed with a checklist-based process analysis. Enhanced implementation of rotary instrumentation could result in better long-term results of students' root canal treatment.

Becoming a neurosurgeon in France: A qualitative study from the trainees' perspective

Introduction

The training of neurosurgeons is evolving in a world of socio-professional changes, including the technological revolution, administrative pressure on stakeholders, reduced working hours, geographical heterogeneity, generational changes, to name but a few.

Research question

This qualitative study aimed to explore experiences and feedback of French neurosurgical trainees concerning their training.

Material and methods

The grounded theory approach was used with 23 neurosurgical trainees' interviews. Inclusion was continued until data saturation. Six researchers (an anthropologist, a psychiatrist, and four neurosurgeons) thematically and independently analyzed data collected through anonymized interviews.

Results

Data analysis identified three superordinate themes: (1) The Trainee-Senior Dyad, where the respondents describe a similar bipolarity between trainees and faculty (trainees oscillating between those who fit into the system and those who are more reluctant to accept hierarchy, faculty using an ideal pedagogy while others refuse to help or invest in training); (2) The difficulty to learn (describing pressure exercised on trainees that can alter their motivation and degrade their training, including the impact of administrative tasks); (3) A pedagogy of empowerment (trainee' feelings about the pertinent pedagogy in the OR, ideal sequence to progress, progressive empowerment especially during the shifts, and stress of envisioning themselves as a senior neurosurgeon).

Discussion and conclusion

Respondents emphasize the heterogeneity of their training both intra- and inter-university-hospital. Their critical analysis, as well as the formalization of their stress to become autonomous seniors, can be an important link with the reforms and optimizations currently being carried out to improve and standardize the training of young French neurosurgeons.

Evolution of the feedback conversation in anaesthesia education: a narrative review

Over the past century, education has been a core component for improving patient safety. The initial focus was developing a curriculum and an assessment process. In recent decades, the value of work-based learning has come to the fore. Learning from work, or experiential learning, requires reflection, which is critically dependent on external feedback. Conceptions of feedback have moved from a transactional information transfer from the supervisor to the trainee to a learner-centred and collaborative process occurring in a complex socio-cultural environment. In this narrative review we describe the evolution of the feedback conversation, provide a model synthesising the core concepts of feedback, and offer some guidance for the development of effective feedback in anaesthesia education.

Preferred Feedback Styles Among Different Groups in an Academic Medical Center

INTRODUCTION

Feedback is an essential component in complex work environments. Different generations have been shown to have different sets of values, derived from societal and cultural changes. We hypothesize that generational differences may be associated with preferred feedback patterns among medical trainees and faculty in a large academic institution.

METHODS

A survey was distributed to all students, residents/fellows, and faculty at a large academic medical institution from April 2020 through June 2020. Survey questions evaluated feedback methods for six domains: preparedness, performance, attitude, technical procedures, inpatient, and outpatient care. Participants selected a preferred feedback method for each category. Patient demographics and survey responses were described using frequency statistics. We compared differences in feedback preferences based on generation and field of practice.

RESULTS

A total of 871 participants completed the survey. Preferred feedback patterns in the medical field do not seem to align with sociologic theories of generational gaps. Most participants preferred to receive direct feedback after an activity away from their team, irrespective of their age or medical specialty. Individuals preferred direct feedback during an activity in front of their team only for technical procedures. Compared to nonsurgeons, surgeons were more likely to prefer direct feedback in front of team members for preparedness, performance, and attitude.

CONCLUSIONS

Generational membership is not significantly associated with preferred feedback patterns in this complex medical academic environment. Variations in feedback preferences are associated with field of practice that may be due to specialty-specific differences in culture and personality traits present within certain medical specialties, particularly surgery.

Residents as Medical Coaches

OBJECTIVES

With the recent implementation of Competency-based Medical Education (CBME) and emphasis on direct observation of learners, there is an increased interest in the concept of clinical coaching. While there is considerable literature on the role of attending physicians as coaches, little data is available on the role of residents as coaches, and residents' perceptions about effective coaching. We aimed to identify distinct characteristics of residents' coaching, to examine residents' perceptions on what they valued most in clinical coaches, and to explore trainees' ideas about how to optimize this role.

DESIGN

We performed an exploratory qualitative study, using 45 minutes semi-structured interviews. We did a thematic analysis of the interview transcripts using both inductive and deductive coding.

PARTICIPANTS

We invited and interviewed 5 surgical and 5 nonsurgical residents, and 3 surgical and 3 nonsurgical attending staff. Residents were recruited from all post graduate levels and from a variety of programs.

SETTING

Our study was done in a large tertiary teaching hospital.

RESULTS

Residents perceived that they have a significant role as coaches for junior learners, different from the attending's role. The proximity between the coach and the coaches leads to a different supervisor-learner rapport. This was of benefit as learners described feeling more comfortable making mistakes and seeking feedback, which potentiates effective coaching. Residents reported feeling that it was easier to coach their recently-acquired skills as the subtleties of the tasks and the troubleshooting were fresher in memory. Residents expressed appreciating a coach who values autonomy and does not intervene except when patient safety is at risk. Strategies identified to further optimize residents' role as coaches include placing coaching as a priority, ensuring dedicated time, and offering teaching sessions on coaching.

CONCLUSIONS

Residents have distinct roles as coaches, driven by their recent experience being coached and as near peers. More research is needed to evaluate concrete measures to optimize residents' role as coaches and to improve their coaching skills.

What does "Timely" Mean to Residents? Challenging Feedback Assumptions in Postgraduate Education

Introduction

Current orthodoxy states that feedback should be timely and face-to-face, yet the optimal timing and mode of delivery for feedback is unclear. We explored what "optimal timing" means from residents' points of view as feedback providers and receivers, to ultimately inform strategies to optimize feedback in training.

Methods

As near-peers who have dual roles in both providing and receiving feedback, 16 subspecialty (PGY4 and 5) internal medicine residents were interviewed about their perceptions of the optimal timing and format of feedback. Using constructivist grounded theory, interviews were conducted and analyzed iteratively.

Results

Drawing on their experiences as providers and recipients, residents described simultaneously considering and weighing multiple factors when deciding on when and how to provide feedback. These included their own readiness to engage in providing meaningful feedback, the perceived receptiveness of the learner and the apparent urgency of feedback delivery (e.g., if patient safety was at stake). Face-to-face verbal feedback was valued for encouraging dialogue but could be uncomfortable and limited by time constraints. Written feedback could be more honest and concise, and the possibility of asynchronous delivery had potential to overcome issues with timing and discomfort.

Discussion

Participants' perceptions of the optimal timing of feedback challenge current assumptions about the benefits of "immediate" versus "delayed". The concept of "optimal timing" for feedback was found to be complex and context-dependent, defying a formulaic approach. There may be a role for asynchronous and/or written feedback, which has potential to address unique issues identified issues in near-peer relationships.

Development of a Classification System for Live Surgical Feedback

Importance

Live feedback in the operating room is essential in surgical training. Despite the role this feedback plays in developing surgical skills, an accepted methodology to characterize the salient features of feedback has not been defined.

Objective

To quantify the intraoperative feedback provided to trainees during live surgical cases and propose a standardized deconstruction for feedback.

Design, Setting, and Participants

In this qualitative study using a mixed methods analysis, surgeons at a single academic tertiary care hospital were audio and video recorded in the operating room from April to October 2022. Urological residents, fellows, and faculty attending surgeons involved in robotic teaching cases during which trainees had active control of the robotic console for at least some portion of a surgery were eligible to voluntarily participate. Feedback was time stamped and transcribed verbatim. An iterative coding process was performed using recordings and transcript data until recurring themes emerged.

Exposure

Feedback in audiovisual recorded surgery.

Main Outcomes and Measures

The primary outcomes were the reliability and generalizability of a feedback classification system in characterizing surgical feedback. Secondary outcomes included assessing the utility of our system.

Results

In 29 surgical procedures that were recorded and analyzed, 4 attending surgeons, 6 minimally invasive surgery fellows, and 5 residents (postgraduate years, 3-5) were involved. For the reliability of the system, 3 trained raters achieved moderate to substantial interrater reliability in coding cases using 5 types of triggers, 6 types of feedback, and 9 types of responses (prevalence-adjusted and bias-adjusted κ range: a 0.56 [95% CI, 0.45-0.68] minimum for triggers to a 0.99 [95% CI, 0.97-1.00] maximum for feedback and responses). For the generalizability of the system, 6 types of surgical procedures and 3711 instances of feedback were analyzed and coded with types of triggers, feedback, and responses. Significant differences in triggers, feedback, and responses reflected surgeon experience level and surgical task being performed. For example, as a response, attending surgeons took over for safety concerns more often for fellows than residents (prevalence rate ratio [RR], 3.97 [95% CI, 3.12-4.82]; P = .002), and suturing involved more errors that triggered feedback than dissection (RR, 1.65 [95% CI, 1.03-3.33]; P = .007). For the utility of the system, different combinations of trainer feedback had associations with rates of different trainee responses. For example, technical feedback with a visual component was associated with an increased rate of trainee behavioral change or verbal acknowledgment responses (RR, 1.11 [95% CI, 1.03-1.20]; P = .02).

Conclusions and Relevance

These findings suggest that identifying different types of triggers, feedback, and responses may be a feasible and reliable method for classifying surgical feedback across several robotic procedures. Outcomes suggest that a system that can be generalized across surgical specialties and for trainees of different experience levels may help galvanize novel surgical education strategies.

Teacher feedback-based collaborative testing improves students' knowledge gaps of parasitology

Collaborative testing has been demonstrated the ability to improve students' performance, enhance students' learning, and aid in knowledge retention in many different courses. However, this examination mode lacks the process of teacher feedback. Herein, a short teacher feedback from was added immediately after the collaborative testing to improve the students' performance. A parasitology class of 121 undergraduates was randomized into two groups: group A and group B. Collaborative testing was carried out at the end of theoretical teaching. During the test, students would first answer questions as individuals for 20 minutes. Then, students from group A answered the same questions in groups (5 students in each group) for 20 minutes, while the group-testing duration was only 15 minutes in group B. Immediately after the group testing, teachers conducted a 5-minute feedback about the morphology identification according to the analysis of the answers by group B. Four weeks later, a final test was conducted in an individual test. The total scores and scores for each examination content were analyzed. The results showed that there was no significant difference in the final exam scores between both groups (t = -1.278, P = 0.204). However, the morphological and diagnostic test results of the final examination in group B were significantly higher than those of the midterm examination, while there was no significant change in group A (t = 4.333, P = 0.051). The results confirmed that the teacher feedback after the collaborative testing can effectively make up for the students' knowledge gaps.NEW & NOTEWORTHY This study found that collaborative group testing is helpful for teachers to grasp students' knowledge gaps more easily and the teacher feedback after the collaborative group testing can effectively make up for the knowledge gaps of students.

Optimizing the Learner's Role in Feedback: Development of a Feedback-Preparedness Online Application for Medical Students in the Clinical Setting

Feedback is an essential component of medical education, especially during clinical rotations. There is growing interest in learner-related factors that can optimize feedback's efficiency, including goal orientation, reflection, self-assessment, and emotional response. However, no mobile application or curriculum currently exists to specifically address those factors. This technical report describes the concept, design, and learner-based feedback of an innovative online application, available on mobile phones, developed to bridge this gap. Eighteen students in their third or fourth year of medical school provided comments on a pilot version of the application. The majority of learners deemed the module relevant, interesting, and helpful to guide reflection and self-assessment, therefore fostering better preparation before an upcoming feedback session. Minor improvements were suggested in terms of content and format. The learners' initial positive response supports further efforts to engage in validity and evaluation research. Future steps include modifying the mobile application based on learners' comments, evaluating its efficacy in a real clinical setting, and clarifying whether it is most beneficial for mid-rotation or end-of-rotation feedback sessions.

Fostering formative assessment: teachers' perception, practice and challenges of implementation in four Sudanese medical schools, a mixed-method study

Formative assessment (assessment for learning) enhances learning (especially deep learning) by using feedback as a central tool. However, implementing it properly faces many challenges. We aimed to describe the perception of medical teachers towards FA, their practice, challenges of implementing FA and present applicable solutions. A mixed-method, explanatory approach study was applied by administering a validated questionnaire to 190 medical teachers in four medical schools in Sudan. The obtained results were further studied using the Delphi method. Quantitative analysis revealed that medical teachers perceived their grasping of the concept of FAs and their ability to differentiate formative from summative assessments as very well (83.7%) and (77.4%), respectively. However, in contradiction to the former results, it was noteworthy that (41%) of them mistakenly perceived FA as an approach conducted for purposes of grading and certification. The qualitative study defined the challenges into two main themes: lack of understanding of formative assessment and lack of resources. Medical teachers' development and resource allocation were the main recommendations. We conclude that there is misunderstanding and malpractice in implementing formative assessment attributed to the lack of understanding of FA as well as the lack of resources. We as well present suggested solutions derived from the perception of the medical teachers in the study and evolved around three approaches: faculty development, managing the curriculum by allocating time and resources for FA, and advocacy among stakeholders.

Development of a mobile tele-education system to assist remote otolaryngology learning during COVID-19 pandemic

Background

Developing clinical thinking competence (CTC) is crucial for physicians, but effective methods for cultivation and evaluation are a significant challenge. Classroom teaching and paper-and-pencil tests are insufficient, and clinical field learning is difficult to implement, especially during the COVID-19 pandemic. Simulation learning is a useful alternative, but existing methods, e.g., OSCE, 3D AR/VR, and SimMan, have limitations in terms of time, space, and cost.

Objective

This study aims to present the design and development of an Otolaryngology Mobile Tele-education System (OMTS) to facilitate CTC learning, and to evaluate the system's usability with senior otolaryngology experts.

Methods

The OMTS system utilizes the convenience of mobile learning and the touch function of mobile devices to assist users (medical students or post-graduate physicians) in learning CTC remotely. Clinical knowledge and system functions in the OMTS system are defined by senior experts based on required CTC learning cases. Through simulated clinical case scenarios, users can engage in interactive clinical inquiry, practice required physical and laboratory examinations, make treatment decisions based on simulated responses, and understand and correct learning problems through a diagnostic report for effective learning. Usability testing of the OMTS system was evaluated by three senior otolaryngology experts using measurements of content validity, system usability, and mental workload during their available time and location.

Results

Statistical results of experts' evaluation showed that the OMTS system has good content validity, marginal-to-acceptable system usability, and moderate mental workload. Experts agreed that the system was efficient, professional, and usable for learning, although the practicality of the clinical inquiry and hands-on practice functions could be improved further.

Conclusions

Based on the OMTS system, users can efficiently hands-on practice and learn clinical cases in otolaryngology, and understand and correct their problems according to the diagnostic report. Therefore, the OMTS system can be expected to facilitate CTC learning according to experts' evaluation.

From Their Eyes: What Constitutes Quality Formative Written Feedback for Neurosurgery Residents

BACKGROUND

The characteristics of quality feedback from the neurosurgery resident's perspective are not fully elucidated. The Surgical Autonomy Program is an intraoperative assessment tool based on Vygotsky's Zone of Proximal Development (ZPD). SAP facilitates assessment of a resident's operative performance accompanied by written feedback.

OBJECTIVE

The goal of this study was twofold: to identify themes from the written feedback of SAP operative assessments and to examine if these themes influenced the neurosurgery residents' perception of feedback quality.

METHODS

In 2021, SAP data from 2019-2021 at two neurosurgery programs were reviewed. Feedback quality from the SAP was determined by the resident at the time of their assessment. Using a constant comparative technique, the feedback was coded using a thematic analysis. The quality of feedback within each code was analyzed.

RESULTS

There were 2968 SAP entries evaluated. When the ZPD concept was fully used, residents reported high quality feedback 91.4% of the time compared to 58.6% when ZPD was not used (p < 0.001). Qualitative analysis of the written feedback revealed five themes: Non-Specific, Specific General Observations, Key Points, Next Steps, and Independent Practice. Feedback in the Specific General Observations, Key Points, and Independent Practice categories were associated with higher level feedback than leaving the space blank (p < 0.001) or writing Non-Specific comments (p < 0.001).

CONCLUSIONS

Providing comments that discuss the resident's specific performance in the case, key learning points, or their progress towards independence, results in high quality feedback. Utilizing a theory-based tool such as the SAP can provide meaningful feedback to neurosurgical residents.

Evaluation of a tool to improve the quality of preceptor written feedback for family medicine residents: training and use of a CanMEDS-MF competency-based criterion guide

Background

Written feedback is essential in resident teaching, but preceptors are not always well equipped to provide relevant feedback. The purpose of this study was to evaluate the effectiveness of multi-episodic training and the use of a criterion-referenced guide for written feedback for family medicine preceptors in a French-language academic hospital.

Method

Twenty-three (23) preceptors participated in the training and used the criterion-referenced guide to guide them during the written evaluation in an evaluation sheet named "Field Notes." The content of these Field Notes was analyzed according to completion, the rate of specific feedback, and the rate of feedback by CanMEDS-MF role before and after the training over a three-month period.

Results

Based on the analysis of the Field Notes (n = 70 pre-test; n = 138 post-test), an increase in the percentage of completion (50% vs. 92%, z = 2.97, p = 0.0030) and specific feedback (59% vs. 92%, z = 2.47, p=0.0137) was noted. There was no significant increase in feedback by CanMEDS-MF role.

Conclusions

The development of multi-episodic training and a criterion-referenced guide, created according to the CanMEDS-MF repository, suggests an improvement in comprehensive and specific written feedback in family medicine education.

A Practical Review of Mastery Learning

Objective. To review mastery learning and provide recommendations for implementation in a competency-based curriculum.Findings. Mastery learning, introduced in the 1960s, was developed to ensure all students reach a desired level of mastery or competency. In this model, students acquire knowledge, skills, or attitudes and then complete formative assessments on that learning. If they achieve the desired level, they can proceed to enrichment activities. Students who do not meet the desired level of mastery proceed through corrective activities and retesting. Evidence suggests students within a mastery learning model perform better academically than those in nonmastery learning models with moderate effect sizes. Mastery learning may result in better performance due to several theoretical reasons, including aspects of motivation, testing, and feedback.Conclusion. We make several recommendations on how to modernize mastery learning for apply it to the pharmacy education, including the recommendation to use more cumulative testing and assessment of baseline knowledge and skills. In addition, models of successive relearning may be applied to this model.

Assessing Interpersonal Relationships in Medical Education: the Connection Index

OBJECTIVE

The relationship between a resident physician and his/her supervising attending is foundational to graduate medical education and may impact the clinical learning environment and resident well-being. This paper focuses on how to measure connection between a resident and their clinical supervisor. Connection includes the subdomains of psychological safety, empathy, educational alliance, and feedback.

METHODS

After reviewing the literature, the authors designed the 12-item, 7-point Connection Index (CI₁₂) to quantitatively measure connections between a resident and his/her supervisor during a 6-month period (supervision dyad), and based on educational alliance, empathy, psychological safety, and effective feedback. A 9-criteria evaluation framework was applied to assess its reliability and validity on a sample of psychiatry residents at a residency program, July 2016 through June 2018.

RESULTS

Out of a total possible number of 50 residents, 100% participated to rate 41 supervisors over 201 supervision dyads; the CI₁₂ satisfied all eight of the eight testable criteria, including high scalability (H = 0.78), consistency (alpha = 0.98), test-retest validity (ICC = 0.95), and construct validity where CI₁₂ was found to have statistically significant correlations with outcomes measures (greater connection was associated with less negative emotional experiences, less mistreatment or bias, less burnout, and higher attendance to supervision sessions).

CONCLUSION

The authors showed the CI₁₂ can be a valid and reliable instrument to quantify whether a resident and his/her supervisor connects during a 6-month supervision with respect to empathy, psychological safety, educational alliance, and feedback. We recommend assessing connections as part of the overall evaluation of a resident's experience with the clinical learning environment.

Medical Students' Intention to Change After Receiving Formative Feedback: Employing Social Cognitive Theories of Behavior

Introduction

We applied Azjen's theory of planned behavior (TPB) and Triandis' theory of interpersonal behavior (TIB) to understand medical students' intention to change behavior based on feedback received during an obstetrics and gynecology clerkship. Both models presume that behavioral intention is strongly related to actual behavior.

Materials and Methods

We collected free-text responses from students during a year-long Feedback Focused initiative on the obstetrics and gynecology clerkship at Harvard Medical School. Students reported feedback daily and what they would change based on that feedback. We applied TPB and TIB to identify students' motivation to change. We analyzed data using directed content analysis.

Results

We reviewed 1,443 feedback entries from 122 students between July 2, 2018, and May 31, 2019. Self-efficacy was the most commonly represented component, related to a student expressing their own role, ability, or skill integrating the feedback (85%). Some entries (11%) focused on students' attitudes or beliefs about the outcome of the implemented feedback, usually patient focused but sometimes about the learner's outcome. Intentions motivated by social norms and expectations focused on the perceived or stated expectations of others, usually a superior or a team (11%). A small number of entries (1.7%) indicated that students had an emotional response to challenging or meaningful feedback.

Conclusions

While self-efficacy is an important change motivator, faculty development geared toward improving the provision of meaningful feedback that bridges a desired behavior change to an outcome of interest, framed through the attitudes and beliefs or social norms lens, may improve trainee performance.

Process-oriented metrics to provide feedback and assess the performance of students who are learning surgical procedures: The percutaneous dilatational tracheostomy case

PURPOSE

Assessing competency in surgical procedures is key for instructors to distinguish whether a resident is qualified to perform them on patients. Currently, assessment techniques do not always focus on providing feedback about the order in which the activities need to be performed. In this research, using a Process Mining approach, process-oriented metrics are proposed to assess the training of residents in a Percutaneous Dilatational Tracheostomy (PDT) simulator, identifying the critical points in the execution of the surgical process.

MATERIALS AND METHODS

A reference process model of the procedure was defined, and video recordings of student training sessions in the PDT simulator were collected and tagged to generate event logs. Three process-oriented metrics were proposed to assess the performance of the residents in training.

RESULTS

Although the students were proficient in classic metrics, they did not reach the optimum in process-oriented metrics. Only in 25% of the stages the optimum was achieved in the last session. In these stages, the four more challenging activities were also identified, which account for 32% of the process-oriented metrics errors.

CONCLUSIONS

Process-oriented metrics offer a new perspective on surgical procedures performance, providing a more granular perspective, which enables a more specific and actionable feedback for both students and instructors.

Assessing VATS competence based on simulated lobectomies of all five lung lobes

OBJECTIVES

To determine the number of procedures and expert raters necessary to provide a reliable assessment of competence in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy.

METHODS

Three randomly selected VATS lobectomies were performed on a virtual reality simulator by participants with varying experience in VATS. Video recordings of the procedures were independently rated by three blinded VATS experts using a modified VATS lobectomy assessment tool (VATSAT). The unitary framework of validity was used to describe validity evidence, and generalizability theory was used to explore the reliability of different assessment options.

RESULTS

Forty-one participants (22 novices, 10 intermediates, and 9 experienced) performed a total of 123 lobectomies. Internal consistency reliability, inter-rater reliability, and test-retest reliability were 0.94, 0.85, and 0.90, respectively. Generalizability theory found that a minimum of two procedures and four raters or three procedures and three raters were needed to ensure the overall reliability of 0.8. ANOVA showed significant differences in test scores between the three groups (P < 0.001). A pass/fail level of 19 out of 25 points was established using the contrasting groups' standard setting method, leaving one false positive (one novice passed) and zero false negatives (all experienced passed).

CONCLUSION

We demonstrated validity evidence for a VR simulator test with different lung lobes, and a credible pass/fail level was identified. Our results can be used to implement a standardized mastery learning training program for trainees in VATS lobectomies that ensures that everyone reaches basic competency before performing supervised operations on patients.

Gender Differences in Emergency Medicine Attending Physician Comments to Residents: A Qualitative Analysis

IMPORTANCE

Prior studies have revealed gender differences in the milestone and clinical competency committee assessment of emergency medicine (EM) residents.

OBJECTIVE

To explore gender disparities and the reasons for such disparities in the narrative comments from EM attending physicians to EM residents.

DESIGN, SETTING, AND PARTICIPANTS

This multicenter qualitative analysis examined 10 488 narrative comments among EM faculty and EM residents between 2015 to 2018 in 5 EM training programs in the US. Data were analyzed from 2019 to 2021.

MAIN OUTCOMES AND MEASURES

Differences in narrative comments by gender and study site. Qualitative analysis included deidentification and iterative coding of the data set using an axial coding approach, with double coding of 20% of the comments at random to assess intercoder reliability (κ, 0.84). The authors reviewed the unmasked coded data set to identify emerging themes. Summary statistics were calculated for the number of narrative comments and their coded themes by gender and study site. χ2 tests were used to determine differences in the proportion of narrative comments by gender of faculty and resident.

RESULTS

In this study of 283 EM residents, of whom 113 (40%) identified as women, and 277 EM attending physicians, of whom 95 (34%) identified as women, there were notable gender differences in the content of the narrative comments from faculty to residents. Men faculty, compared with women faculty, were more likely to provide either nonspecific comments (115 of 182 [63.2%] vs 40 of 95 [42.1%]), or no comments (3387 of 10 496 [32.3%] vs 1169 of 4548 [25.7%]; P < .001) to men and women residents. Compared with men residents, more women residents were told that they were performing below level by men and women faculty (36 of 113 [31.9%] vs 43 of 170 [25.3%]), with the most common theme including lack of confidence with procedural skills.

CONCLUSIONS AND RELEVANCE

In this qualitative study of narrative comments provided by EM attending physicians to residents, multiple modifiable contributors to gender disparities in assessment were identified, including the presence, content, and specificity of comments. Among women residents, procedural competency was associated with being conflated with procedural confidence. These findings can inform interventions to improve parity in assessment across graduate medical education.

Perception of Shame in the Plastic Surgery Field

Doctors and postgraduate students, especially those in the surgical field, face a highly stressful environment and are exposed to various emotions that have been studied, but the concept of shame-based learning (SBL) is still undergoing investigation, especially in the field of plastic surgery. SBL is a teaching method in which an instructor instills a sense of shame in the student, which may cause depression, anxiety, aggression, and poor job performance, leading to burnout, mental health illness, substance abuse, and suicide.

METHODS

From March to May 2022, two cross-sectional electronic surveys were conducted for residents and consultants in Saudi Arabia, respectively, which used a validated questionnaire to assess SBL.

RESULTS

Among the 70 responses received (29 residents and 41 consultants), 75.9% of the residents and 80.5% of the consultants were shamed. For residents, a wrong answer was the most common trigger for shame (44.8%), and the operating room was the most common place for it (51.7%). Losing self-confidence was the most common result of shaming (37.9%) and (41.4%) dealt with it by keeping it to themselves. Although 27.6% of residents stated that they had no negative effect, 20.7% stated that they were motivated. There are consultants who practice shaming directly or indirectly (65.9%), while some agreed that it is not necessary (80.5%).

CONCLUSIONS

Although both groups agreed that SBL is unnecessary for the field and will not be practiced in the future, most residents and consultants experienced shame. The negative impact of SBL has several effects on the trainer, the teaching environment, and patient care.

The Practice of Feedback in Health Professions Education in the Hierarchical and Collectivistic Culture: a Scoping Review

The global trend in student assessment is moving towards outcome-based assessment that requires multiple systematic data points and continuous feedback. However, in hierarchical and collectivistic cultures, one-way communication is practised more often, leading to less dialogical feedback. This scoping review explored feedback practice in Asian educational setting. Based on the 17 articles selected, the findings were categorised into four themes, i.e. inhibiting or facilitating factors of feedback, influences of cultural factors on feedback, discrepancies between students' and teachers' perceptions of feedback and impact of feedback. Hierarchical and collectivist cultural aspects, such as preference for group feedback, are pertinent to feedback practice, which likely influence the readiness for programmatic assessment.

The Use of a Modified Script Concordance Test in Clinical Rounds to Foster and Assess Clinical Reasoning Skills

The development of clinical reasoning skills is a high priority during clinical service, but an unpredictable case load and limited time for formal instruction makes it challenging for faculty to foster and assess students' individual clinical reasoning skills. We developed an assessment for learning activity that helps students build their clinical reasoning skills based on a modified version of the script concordance test (SCT). To modify the standard SCT, we simplified it by limiting students to a 3-point Likert scale instead of a 5-point scale and added a free-text box for students to provide justification for their answer. Students completed the modified SCT during clinical rounds to prompt a group discussion with the instructor. Student feedback was positive, and the instructor gained valuable insight into the students' thought process. A modified SCT can be adopted as part of a multimodal approach to teaching on the clinic floor. The purpose of this article is to describe our modifications to the standard SCT and findings from implementation in a clinical rounds setting as a method of formative assessment for learning and developing clinical reasoning skills.

The Effect of Anonymity on Quality of Resident Feedback

OBJECTIVE

Quality feedback is critical to facilitate better performance and quicker learning. However, faculty may be hesitant to provide quality constructive feedback, especially in written form, due to fear of retaliation. We evaluated the impact of faculty anonymity on the quality of faculty-to-resident feedback.

DESIGN

A retrospective review was undertaken of faculty evaluation of resident performance from 2017 to 2018, when evaluations were identifiable, compared to 2018-2019, when evaluations were anonymous. Evaluations included 27 individual items with Likert type scoring and 2 open-ended questions. Open-ended responses and overall performance were de-identified and scored by 2 reviewers independently using the task, performance gap, action scoring model. Comparisons between groups were performed with the Wilcoxon-Mann-Whitney test.

SETTING

Tertiary Care Institution, University of Michigan, Ann Arbor, MI PARTICIPANTS: 415 resident performance evaluations were available for analysis with 251 in the identifiable group and 164 in the anonymous group.

RESULTS

The average composite score for the identifiable group was 105.2 and 103.4 in the anonymous group (p = 0.22). The effect size of the impact on composite score was small (Cohen's d 0.084, 95% CI -0.11-0.28). There was excellent inter-rater reliability. There were no differences between feedback groups for any of the 3 components of task, performance gap, and action model. While average scores for all 3 components were low, action scores were lowest.

CONCLUSIONS

Anonymity did not significantly impact faculty evaluations of resident performance. The quality of open-ended feedback on written evaluations was generally poor, especially in identifying actions for continued performance improvement. Additional mechanisms to improve feedback quality should be sought.

Understanding Feedback for Learners in Interprofessional Settings: A Scoping Review

BACKGROUND

Interprofessional feedback is becoming increasingly emphasized within health professions' training programs. The objective of this scoping review is to determine what is known about how learners perceive and interact with feedback in an interprofessional context for learning.

METHODS

A search strategy was developed and conducted in Ovid MEDLINE. Title and abstract screening were performed by two reviewers independently. Next, full texts of selected articles were reviewed by one reviewer to determine the articles included in the review. Data extraction was performed to determine the articles' study population, methodologies and outcomes relevant to the research objective.

RESULTS

Our analysis of the relevant outcomes yielded four key concepts: (1) issues with the feedback process and the need for training; (2) the perception of feedback providers, affecting how the feedback is utilized; (3) professions of the feedback providers, affecting the feedback process; and (4) learners' own attitude toward feedback, affecting the feedback process.

CONCLUSIONS

The learner's perception of interprofessional feedback can be an obstacle in the feedback process. Training around interprofessional feedback should be included as part of interprofessional programs. Research is needed to explore how to address barriers in feedback interaction that stem from misguided perceptions of feedback providers' professions.

Assessment of neurosurgical resident milestone evaluation reporting and feedback processes

OBJECTIVE

Structured performance evaluations are important for the professional development and personal growth of resident learners. This process is formalized by the Accreditation Council for Graduate Medical Education milestones assessment system. The primary aim of this study was to understand the current feedback delivery mechanism by exploring the culture of feedback, the mechanics of delivery, and the evaluation of the feedback itself.

METHODS

Face-to-face interviews were conducted with 10 neurosurgery residents exploring their perceptions of summative feedback. Coded data were analyzed qualitatively for overriding themes using the matrix framework method. A priori themes of definition of feedback, feedback delivery, and impact of feedback were combined with de novo themes discovered during analysis.

RESULTS

Trainees prioritized formative over summative feedback. Summative and milestone feedback were criticized as being vague, misaligned with practice, and often perceived as erroneous. Barriers to implementation of summative feedback included perceived veracity of feedback, high interrater variability, and the inconstant adoption of a developmental progression model. Gender bias was noted in degree of feedback provided and language used.

CONCLUSIONS

Trainee perception of feedback provided multiple areas of improvement. This paper can serve as a baseline to study improvements in the milestone feedback process and optimize learning.

Effect of an interprofessional small-group communication skills training incorporating critical incident approaches in an acute care and rehabilitation clinic specialized for spinal cord injury and disorder

AIM

To investigate the impact of site-specific inter-professional small-group communication skills training (CST) that incorporates critical incident approaches to learning on patient satisfaction with communication.

SETTING

Rehabilitation clinic specialized for spinal cord injury/disorder (SCI/D).

METHODS

Retrospective observational cohort study design using patient and health-professional self-report data. Data for patient satisfaction with communication were collected in 2014 (existing records) and each year from 2015 to 2021 (post-program; volunteers) using the MECON survey.

RESULTS

Fifteen basic (n = 161 participants), 16 refresher (n = 84), and five short (n = 17) CST seminars were conducted. Overall, 262 employees (105 physicians, 63 nurses, 36 physio- and occupational therapists, and 58 others) participated; 92 participants (response rate 37.6%) responded to feedback surveys. They rated the seminars positive concerning the alternation between theory, discussion, and practical exercise in 91.3%, and rated the length of the training ideal in 80.2%. Post-program patient satisfaction overall increased from 83.1% (confidence interval (CI) 2.6%) to 90% (CI 0.8%; R2 = 0.776; p= 0.004). It was higher in specific communication-related topics: "receiving information" (81.1%, CI 3.1-90.2%, CI 1.0%; p = 0.003), "being able to bring in concerns" (83%, CI 1.0-90.8%; R2 = 0.707; p = 0.009) and "being treated with respect" (89.4%, CI 2.6-94.4%, CI 0.8%; R2 = 0.708; p = 0.004).

PRACTICE IMPLICATIONS

Inter-professional CST is feasible and well accepted by professionals from various professional groups. During seven years of continuous training, independent patient ratings of satisfaction with professional communication have improved significantly. Participants attest to the training's high credibility and usefulness in everyday life.

Teacher Feedback and Student Learning-The Students' Perspective

INTRODUCTION

Feedback from teachers to students plays an important role in informing students about the outcome of their assessments. It contributes to students' ongoing learning. The aim of this study was to investigate dental students' perceptions of the feedback given to them by their teachers in Europe.

MATERIALS & METHODS

An online questionnaire was completed by dental students throughout Europe in this quantitative study. Data were collected via Google Forms, transferred to an excel spreadsheet and analysed using SPSS software Version 24.

RESULTS

234 students studying in 9 different European countries completed the questionnaire. These students were born in 36 different countries within and beyond Europe. 84% (n=197) were undergraduate students. 20.3% (n=48) students reported receiving feedback following summative assessments. 81.2% (n=190) students reported constructive criticism as their preferred mode of receiving feedback. 11.3% (n=26) students did not know who delivered the feedback to them. 71% (n=166) students felt that the feedback they received had a significant impact on their future learning.

CONCLUSION

It would appear that there is some diversity in dental students' perceptions of: i) who delivers feedback, ii) when feedback is given, iii) the consistency of feedback received, and iv) the style of feedback they preferred compared to that delivered by tutors. Feedback is being provided to dental students in an appropriate and helpful manner, although there is still room for improvement. Students were aware of the significance of feedback and its impact on future learning.

Comparison of self versus expert-assisted feedback for cricothyroidotomy training: a randomized trial

BACKGROUND

The self-video feedback method may have the potential to provide a low-cost alternative to physician-driven simulation-based training. This study aimed to assess the utility of two video feedback methods by comparing the improvement in performing cricothyroidotomy procedure following self video feedback (trainees review their performance by themselves) and expert-assisted video feedback (trainees review their performance while an emergency physician provides additional feedback).

METHODS

This study was pretest-posttest and two-group designed research performed at a university simulation center with 89 final-year medical students and used a cricothyroidotomy simulation model. After seeing an educational presentation and a best practice video, trainees were randomized into two groups; self video feedback group (SVFG) and expert-assisted video feedback group (EVFG). They performed the cricothyroidotomy before and after the feedback. The procedures were also recorded and scored by two emergency physicians.

RESULTS

There was a statistically significant improvement between pre-feedback and post-feedback assessments in terms of scores received and time needed for the procedures in both SVFG and EVFG groups (p < 0.05). Additionally, the post-feedback assessment scores were higher and time needed for the procedure was lower in the EVFG when compared with SVFG (p < 0.05 for both).

CONCLUSIONS

Results demonstrated significant improvement in cricothyroidotomy performance with both types of video feedback method. Even though the improvement was better in the EVFG compared to the SVFG, the self video feedback may have value especially in situations where expert-assisted feedback is not possible.

Shaping the right conditions in programmatic assessment: how quality of narrative information affects the quality of high-stakes decision-making

BACKGROUND

Programmatic assessment is increasingly being implemented within competency-based health professions education. In this approach a multitude of low-stakes assessment activities are aggregated into a holistic high-stakes decision on the student's performance. High-stakes decisions need to be of high quality. Part of this quality is whether an examiner perceives saturation of information when making a holistic decision. The purpose of this study was to explore the influence of narrative information in perceiving saturation of information during the interpretative process of high-stakes decision-making.

METHODS

In this mixed-method intervention study the quality of the recorded narrative information was manipulated within multiple portfolios (i.e., feedback and reflection) to investigate its influence on 1) the perception of saturation of information and 2) the examiner's interpretative approach in making a high-stakes decision. Data were collected through surveys, screen recordings of the portfolio assessments, and semi-structured interviews. Descriptive statistics and template analysis were applied to analyze the data.

RESULTS

The examiners perceived less frequently saturation of information in the portfolios with low quality of narrative feedback. Additionally, they mentioned consistency of information as a factor that influenced their perception of saturation of information. Even though in general they had their idiosyncratic approach to assessing a portfolio, variations were present caused by certain triggers, such as noticeable deviations in the student's performance and quality of narrative feedback.

CONCLUSION

The perception of saturation of information seemed to be influenced by the quality of the narrative feedback and, to a lesser extent, by the quality of reflection. These results emphasize the importance of high-quality narrative feedback in making robust decisions within portfolios that are expected to be more difficult to assess. Furthermore, within these "difficult" portfolios, examiners adapted their interpretative process reacting on the intervention and other triggers by means of an iterative and responsive approach.

Experiences of simulated patients in providing feedback in communication skills teaching for undergraduate medical students

BACKGROUND

Simulated Patients (SPs) are commonplace in the teaching of communication skills in medical education and can provide immediate feedback to students from the patient's perspective. The experiences of SPs and their perspective on providing feedback is an under-studied area. This study aims to explore SP experiences and views on feedback, factors influencing their feedback and implications for training.

METHODS

Using a constructivist grounded theory approach, we conducted six focus groups with 30 SPs. Participants included experienced simulated patients from a London-based actor agency, used in undergraduate teaching programmes of communication skills. Consistent with the principles of grounded theory, data was collected and analysed in an iterative process to identify themes.

RESULTS

Five over-arching themes were identified: 1.) Feedback processes, 2.) Challenges in providing feedback, 3.) Cumulative experiences, 4.) Web of interpersonal relationships and dynamics and 5.) Portraying the character and patient representations.

DISCUSSION

These SPs regarded the sharing of the emotions they experienced during the consultation as the focus of their feedback. Their preference was for giving a 'sandwich style' of feedback and 'out-of-role' approach. The relationship with facilitators and students and politeness conventions emerged as significant factors when providing feedback. Sensitivity to the social dynamics of groups and implicit facilitator expectations were challenges they experienced as was divergence in views of student performance.

CONCLUSION

This study explored SP experiences and perspectives on providing feedback. Findings reveal complex social and structural dynamics at play in providing feedback which have not been reported so far in the literature. It is recommended that these issues should be addressed in training of both SPs and facilitators, in addition to feedback guidelines.

Coaching the Fundamentals: Exploring the Applicability and Usefulness of a Novel Skills-Based Feedback Modality

OBJECTIVE

Receiving and integrating feedback is a key to medical trainee development. To assist trainees seeking improvement through daily formative feedback and deliberate practice, the authors created a new skills-based framework called microskills, derived from the Accreditation Council for Graduate Medical Education (ACMGE) milestones and entrustable professional activities. The authors then explored pediatric resident perceptions around the applicability and usefulness of microskills.

METHODS

The authors conducted 4 qualitative semistructured focus groups of 28 pediatric residents. Focus group prompts asked participants to reflect on microskills as a new feedback modality and microskills in relation to existing feedback and assessment approaches. Focus group transcripts were analyzed through inductive thematic analysis through an iterative process until theoretical saturation was reached.

RESULTS

Participants felt microskills could facilitate skill-building and improvement, allow for consistent, targeted feedback, and establish a practice of coaching. Participants also perceived microskills' future success to be dependent on how the modality is adopted and utilized alongside existing assessment tools.

CONCLUSIONS

Trainees found microskills to be a granular, context-based, coaching tool that could improve skill-building and the feedback process. Microskills' emphasis on feedback and their delineation of clinical skills that can be repeatedly practiced has the potential to provide a roadmap for trainee growth. Though microskills could fill a current need in the medical training landscape, they are not a substitute for existing assessment frameworks.

Affecting Change: Enhancing Feedback Interactions with Radiology Trainees

Feedback is a critical part of the learning process and is a valuable tool to empower adult learners. Modern feedback theory places the learner at the center of the feedback encounter. Individual and institutional barriers to effective giving and receiving of feedback can be overcome through education and attention to the form and content of feedback. We review the elements of effective feedback and address issues of framing, environmental, and social factors which aid in providing psychological safety and trust, as necessary elements to create a culture of feedback in radiology training programs. We provide practical strategies to empower learners with the necessary skills to solicit, receive, and reflect on feedback.