Feedback for Learners in Medical Education: What Is Known? A Scoping Review

Making narrative feedback meaningful

BACKGROUND

Narrative written feedback given to students by faculty often fails to identify areas for improvement and recommended actions to lead to this improvement. When these elements are missing, it is challenging for students to improve and for medical schools to use narrative feedback in promotion decisions, to guide coaching plans and to pass on meaningful information to residency programs. Large-group faculty development has improved narrative written feedback, but less is known about individualised faculty development to supplement large-group sessions. To fill this gap, we built a curriculum with general and individualised faculty development to improve narrative written feedback from Internal Medicine faculty to clerkship students.

APPROACH

We used Kern's steps to build a curriculum with general and individualised one-on-one faculty development to improve the problem of inadequate narrative written feedback. We used a novel narrative feedback rubric for pre and post-intervention faculty scores.

RESULTS/FINDINGS/EVALUATION

Through general and individualised one-on-one faculty development with peer comparison scores, we were able to improve narrative written feedback from 3.7/6 to 4.6/6, for an increase of 23%.

IMPLICATIONS

We found our faculty development program effective in improving feedback and was easy to implement. Our rubric was easy to use, and faculty were receptive to feedback in one-on-one meetings. We plan to extend this work locally to other divisions/departments and into graduate medical education; it should also be easily extended to other medical disciplines or health professions.

Navigating discourses of feedback: developing a pattern system of feedback

Although feedback is often presented as if it were a well-understood concept in health professions education, in practice it can mean many things. For some, feedback is a conversation about defining and improving performance, while for others it is the information generated by assessments and tools. Indeed, feedback has variously been defined as a process, as data, as a conversation, and as a reflective exercise. As a result, for a concept so central to what educators do, 'feedback' is ambiguous and has multiple meanings. Pattern theory affords opportunities to examine what scholars and practitioners mean when they use the term 'feedback'. Elaborating feedback as a pattern system can connect otherwise disjointed discourses of feedback. In this paper, the authors describe the development of a pattern system of feedback in medical education. Arksey & O'Malley's 5-stages of scoping reviews were adapted to enact a 6-step pattern system development methodology that included (1) Identifying the research question and scope of inquiry; (2) elaborating a strategy for pattern identification; (3) study selection; (4) abductive pattern representation development; (5) pattern system testing; and (6) summarizing and reporting the results. A pattern system of feedback was developed based on review of 218 full text articles and testing against an additional 2833 citations. This pattern system is made up of 36 pattern representations organized under 6 domains: feedback referent, feedback intentions, feedback information, feedback processing, feedback response, and feedback meta. The pattern system was applied to two models of feedback to demonstrate its utility as a lens through which to analyze various instances of feedback and to foreshadow its potential broader applicability as a tool to facilitate knowledge synthesis in the feedback problem space.

Patient Feedback Applied in Undergraduate Dental Education for Individual Student Development and Assessment: A Scoping Review

INTRODUCTION

Active involvement of patients in healthcare professional education is well established, taking a variety of forms. There is a steer towards patient feedback informing the development of dental students and while there is recognition of its potential value to individual students, challenges exist related to collection and use. What is unclear is, within a dental education setting, the extent and use of patient feedback to individual students. A scoping review was conducted to assess and map the volume and characteristics of the research/literature in this area.

METHODS

Systematic searches of bibliographic databases Ovid MEDLINE(R), Scopus, ERIC and Embase were conducted, and wider literature (Google Scholar) was searched. Screening was conducted based on eligibility criteria and a customised data charting form was used.

RESULTS

The electronic and citation tracking searches identified 1021 studies. After duplicates were removed, 778 studies were screened by title and abstract, and 718 studies were found to be irrelevant to the current review. Sixty full-text studies were assessed for eligibility, 46 studies were excluded, and 14 studies were included for data charting.

CONCLUSION

This review has identified that patient feedback has been captured through both simulated and real patient encounters. There was a bias towards feedback generated through simulated patient encounters. Feedback was reported to support the development of a range of skills, most frequently communication and patient management. Challenges that were identified by researchers related to staff/student engagement and available resources.

Real Patient Participation in Workplace-Based Assessment of Health Professional Trainees: A Scoping Review

The aim of this scoping review is to outline the existing landscape of how real patients participate in the workplace-based assessment of trainees across diverse healthcare professions. In 2019-2020, the authors searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, and Web of Science databases for studies that included descriptions of experiences whereby patients received care from a health professional trainee and participated in workplace-based assessments of that trainee. Full-text articles published in English from 2009 to 2020 were included in the search. Of the 8770 studies screened; 77 full-text articles were included. Analysis showed that strategies for patient participation in workplace-based assessment varied widely. Aspects studied ranged from validation of an assessment tool to evaluation of the impact of an educational intervention on trainees' performance. Assessment of patient satisfaction was the most common approach to patient involvement. The majority of studies were conducted in North America and in the context of physician training. Formal patient participation in the assessment of health professional trainees appears heterogeneous across health professions. Gaps in the literature are evident; therefore, this review points to an inclusive approach to workplace-based assessment to ensure patient feedback of the trainees who care for them is represented.

Surgical Residents' Perception of Feedback on Their Education: Protocol for a Scoping Review

BACKGROUND

Feedback is an essential tool for learning and improving performance in any sphere of education, including training of resident physicians. The learner's perception of the feedback they receive is extremely relevant to their learning progress, which must aim at providing qualified care for patients. Studies pertinent to the matter differ substantially with respect to methodology, population, context, and objective, which makes it even more difficult to achieve a clear understanding of the topic. A scoping review on this theme will unequivocally enhance and organize what is already known.

OBJECTIVE

The aim of this study is to identify and map out data from studies that report surgical residents' perception of the feedback received during their education.

METHODS

The review will consider studies on the feedback perception of resident physicians of any surgical specialty and age group, attending any year of residency, regardless of the type of feedback given and the way the perceptions were measured. Primary studies published in English, Spanish, and Portuguese since 2017 will be considered. The search will be carried out in 6 databases and reference lists will also be searched for additional studies. Duplicates will be removed, and 2 independent reviewers will screen the selected studies' titles, abstracts, and full texts. Data extraction will be performed through a tool developed by the researchers. Descriptive statistics and qualitative analysis (content analysis) will be used to analyze the data. A summary of the results will be presented in the form of diagrams, narratives, and tables.

RESULTS

The findings of this scoping review were submitted to an indexed journal in July 2024, currently awaiting reviewer approval. The search was executed on March 15, 2024, and resulted in 588 articles. After the exclusion of the duplicate articles and those that did not meet the eligibility criteria as well as the inclusion of articles through a manual search, 13 articles were included in the review.

CONCLUSIONS

Conducting a scoping review is the best way to map what is known about a subject. By focusing on the feedback perception more than the feedback itself, the results of this study will surely contribute to gaining a deeper understanding of how to proceed to enhance internal feedback and surgical residents' learning progress.

TRIAL REGISTRATION

Open Science Framework yexb; https://osf.io/yexkb.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/56727.

A Language Model-Powered Simulated Patient With Automated Feedback for History Taking: Prospective Study

BACKGROUND

Although history taking is fundamental for diagnosing medical conditions, teaching and providing feedback on the skill can be challenging due to resource constraints. Virtual simulated patients and web-based chatbots have thus emerged as educational tools, with recent advancements in artificial intelligence (AI) such as large language models (LLMs) enhancing their realism and potential to provide feedback.

OBJECTIVE

In our study, we aimed to evaluate the effectiveness of a Generative Pretrained Transformer (GPT) 4 model to provide structured feedback on medical students' performance in history taking with a simulated patient.

METHODS

We conducted a prospective study involving medical students performing history taking with a GPT-powered chatbot. To that end, we designed a chatbot to simulate patients' responses and provide immediate feedback on the comprehensiveness of the students' history taking. Students' interactions with the chatbot were analyzed, and feedback from the chatbot was compared with feedback from a human rater. We measured interrater reliability and performed a descriptive analysis to assess the quality of feedback.

RESULTS

Most of the study's participants were in their third year of medical school. A total of 1894 question-answer pairs from 106 conversations were included in our analysis. GPT-4's role-play and responses were medically plausible in more than 99% of cases. Interrater reliability between GPT-4 and the human rater showed "almost perfect" agreement (Cohen κ=0.832). Less agreement (κ<0.6) detected for 8 out of 45 feedback categories highlighted topics about which the model's assessments were overly specific or diverged from human judgement.

CONCLUSIONS

The GPT model was effective in providing structured feedback on history-taking dialogs provided by medical students. Although we unraveled some limitations regarding the specificity of feedback for certain feedback categories, the overall high agreement with human raters suggests that LLMs can be a valuable tool for medical education. Our findings, thus, advocate the careful integration of AI-driven feedback mechanisms in medical training and highlight important aspects when LLMs are used in that context.

Assessment of validity and reliability of the feedback quality instrument

BACKGROUND

The purpose was to investigate the psychometric features of the Feedback Quality Instrument (FQI) in medical students, emphasizing the instrument's utility for evaluating the quality of feedback provided in clinical contexts and the importance of performing so for medical trainees.

METHODS AND MATERIAL

The Persian version of the FQI was evaluated for content validity through a focus group of medical education experts. The questionnaire's face, content, and construct validity were assessed using Confirmatory Factor Analysis, internal consistency, and inter-rater reliability. The questionnaire was revised and pilot-tested, with medical students' feedback in different clinical situations. The data was analyzed using AMOS26.

RESULTS

The content validity index equaled 0.88(> 0.79). The content validity ratio representing the proportion of participants who agreed on a selected item was 0.69(> 0.42). According to experts, item 25 is the only modified item, while items 23 and 24 are presented as one item. For reliability, Cronbach alpha was equaled to 0.98.

CONCLUSIONS

The Persian version of the Feedback Quality Instrument (FQI) was valid, reliable, and fair in assessing feedback quality in medical students, providing valuable insights for other institutions. Establishing a basis for systematically analyzing how certain educator behaviors affect student outcomes is practical.

Applying a Competency-Based Medical Education Framework to Development of Residents' Feedback Skills

OBJECTIVE

Feedback is a critically important tool in medical education. This pilot program applies and evaluates a competency-based approach to develop residents' skills in providing feedback to medical students.

METHODS

In 2018-2019, a competency-based resident feedback skills program incorporating videorecording of skills, multi-source feedback using assessment tools with validity evidence, and sequential deliberate practice was piloted in a single-center, prospective study at the University of Rochester. Study participants included eight second-year psychiatry residents and 23 third-year clerkship students. After an introduction to foundational feedback concepts in didactic sessions, residents were videorecorded providing feedback to medical students. Recordings were reviewed with a faculty member for feedback. Skills were assessed by students who had received resident feedback, residents, and faculty utilizing a tool with validity evidence. Observations were repeated a total of three times.

RESULTS

Mean feedback scores increased from 2.70 at the first feedback observation, to 2.77 at the second feedback observation, to 2.89 at the third feedback observation (maximum 3.00 points). The differences between the first and third sessions (0.19) and second and third sessions (0.12) were statistically significant (p values were < .001 and .007, with SE of 0.4 and 0.4, respectively).

CONCLUSIONS

The observed competency-based feedback skills training program for residents using sequential, multi-source review and feedback was feasible and effective. Direct observation is a key component of high-quality feedback, and videorecording is an efficient methodology for observations, enabling both direct observation by the assessor and opportunity for enhanced self-assessment by residents viewing themselves in the feedback encounter.

Implementing Formative Assessment in Human Anatomy Practical Sessions: Medical Students' Perception and Effect on Final Exam Performance

Background

Formative assessment with feedback is part of the assessment program in medical education to improve students' learning. Limited research has focused on its application and impact on practical anatomy education.

Methods

This study aimed to examine medical students' perceptions of formative assessment in practical anatomy sessions of body systems-based educational units and explore its influence on final practical exam performance. A descriptive, cross-sectional study was conducted. Data was collected from 173 Year 2 medical students through a survey that addressed their perception of process and importance of formative assessment and feedback. The survey employed a 5-point Likert scale. Two open-ended questions were appended at the end of the survey. Students' performance in Unit 3 (where formative assessment was conducted) was compared to their performance in Unit 2 (where no formative assessment was conducted) and with the performance of the previous academic year's students in Unit 3 (where no formative assessment was conducted). Descriptive statistics were used. The level of statistical significance was set at p-value < 0.05. Responses to open-ended questions (qualitative data) were counted, categorized as themes, and presented as frequencies and percentages.

Results

The survey showed high internal consistency, and its validity was established through exploratory factor analysis. Results showed that the mean mark for the unit with formative assessment and feedback was significantly higher than for the units without formative assessment and feedback. Students showed positive perception of formative assessment and feedback conducted after practical anatomy sessions. They reported useful insights regarding the benefits they gained from formative assessment and feedback as well as constructive suggestions for future improvements.

Conclusion

The study indicates that students positively perceived formative assessment and feedback sessions after practical anatomy sessions. Findings also refer to a positive effect of formative assessment on students' performance in summative practical assessment in anatomy.

Feedback in an Entrustment-Based Objective Structured Clinical Examination: Analysis of Content and Scoring Methods

Background The integration of entrustable professional activities (EPAs) within objective structured clinical examinations (OSCEs) has yielded a valuable avenue for delivering timely feedback to residents. However, concerns about feedback quality persist. Objective This study aimed to assess the quality and content alignment of verbal feedback provided by examiners during an entrustment-based OSCE. Methods We conducted a progress test OSCE for internal medicine residents in 2022, assessing 7 EPAs. The immediate 2-minute feedback provided by examiners was recorded and analyzed using the Quality of Assessment of Learning (QuAL) score. We also analyzed the degree of alignment with EPA learning objectives: competency milestones and task-specific abilities. In a randomized crossover experiment, we compared the impact of 2 scoring methods used to assess residents' clinical performance (3-point entrustability scales vs task-specific checklists) on feedback quality and alignment. Results Twenty-one examiners provided feedback to 67 residents. The feedback demonstrated high quality (mean QuAL score 4.3 of 5) and significant alignment with the learning objectives of the EPAs. On average, examiners addressed in their feedback 2.5 milestones (61%) and 1.2 task-specific abilities (46%). The scoring methods used had no significant impact on QuAL scores (95% CI -0.3, 0.1, P=.28), alignment with competency milestones (95% CI -0.4, 0.1, P=.13), or alignment with task-specific abilities (95% CI -0.3, 0.1, P=.29). Conclusions In our entrustment-based OSCE, examiners consistently offered valuable feedback aligned with intended learning outcomes. Notably, we explored high-quality feedback and alignment as separate dimensions, finding no significant impact from our 2 scoring methods on either aspect.

Programmatic Assessment of Clinical Reasoning: New Opportunities to Meet an Ongoing Challenge

Issue: Clinical reasoning is essential to physicians' competence, yet assessment of clinical reasoning remains a significant challenge. Clinical reasoning is a complex, evolving, non-linear, context-driven, and content-specific construct which arguably cannot be assessed at one point in time or with a single method. This has posed challenges for educators for many decades, despite significant development of individual assessment methods. Evidence: Programmatic assessment is a systematic assessment approach that is gaining momentum across health professions education. Programmatic assessment, and in particular assessment for learning, is well-suited to address the challenges with clinical reasoning assessment. Several key principles of programmatic assessment are particularly well-aligned with developing a system to assess clinical reasoning: longitudinality, triangulation, use of a mix of assessment methods, proportionality, implementation of intermediate evaluations/reviews with faculty coaches, use of assessment for feedback, and increase in learners' agency. Repeated exposure and measurement are critical to develop a clinical reasoning assessment narrative, thus the assessment approach should optimally be longitudinal, providing multiple opportunities for growth and development. Triangulation provides a lens to assess the multidimensionality and contextuality of clinical reasoning and that of its different, yet related components, using a mix of different assessment methods. Proportionality ensures the richness of information on which to draw conclusions is commensurate with the stakes of the decision. Coaching facilitates the development of a feedback culture and allows to assess growth over time, while enhancing learners' agency. Implications: A programmatic assessment model of clinical reasoning that is developmentally oriented, optimizes learning though feedback and coaching, uses multiple assessment methods, and provides opportunity for meaningful triangulation of data can help address some of the challenges of clinical reasoning assessment.

Medical Students' General Beliefs and Specific Perceptions about Patient Feedback Before and after Training in a Clinical Context

Introduction

Despite its high potential, patient feedback does not always result in learning. For feedback to be effective students must engage with it, which partly depends on their perceptions of feedback. To better understand student engagement with patient feedback in a clinical context, this study explored the following research questions: 1) What are medical students' general beliefs about patient feedback and what are their specific perceptions of feedback messages? 2) What is the difference between these general beliefs and feedback message perceptions before and after patient feedback training?

Methods

The study context was a 12-week clerkship combining Pediatrics and Gynecology, which included feedback training for students and asking for patient feedback. Ninety 4th-year medical students completed pre- and post-clerkship questionnaires. The questionnaires (Beliefs about Patient Feedback Questionnaire, Feedback Perception Questionnaire) were adapted from validated peer-feedback questionnaires. Questionnaires were quantitatively analyzed.

Results

Both pre- and post-clerkship, students had positive general beliefs about patient feedback and positive perceptions of the feedback messages they received. However, paired t-tests showed that students' general beliefs and feedback message perceptions became less positive after feedback training and experience.

Discussion

Patient feedback is not an easy means to learn and students do not become feedback literate in terms of patient feedback overnight. We suggest that future researchers further explore reasons for the decline in positive perceptions of patient feedback. We suggest implementing longitudinal feedback training in medical curricula, where students are guided and supported in the complex task of learning from patients through feedback.

Exploring Feedback Mechanics during Experiential Learning in Pharmacy Education: A Scoping Review

(1) Background: This scoping review aims to explore the literature on feedback for pharmacy students during experiential learning, with a focus on identifying the modes of delivery of feedback and the perceived impact of feedback on student learning outcomes. (2) Methods: The scoping review was conducted in accordance with the Joanna Briggs Institute (JBI) methodology and reported following the Preferred Reporting Items for Systematic Reviews Extension for Scoping Reviews (PRISMA-ScR) guidelines. PubMed, Web of Science, Embase, EBSCO, ERIC, and ProQuest Central were searched electronically from their inception until the end of February 2023 using a combination of keywords and MeSH terms related to feedback, pharmacy education, and student learning outcomes. Data were synthesized narratively. (3) Results: This review included 13 studies published between 2008 and 2022. Almost half of the included studies were conducted in the USA (n = 6, 46%) and reported the perspective of undergraduate pharmacy students (n = 6, 46%). Verbal feedback was the most common mode of feedback delivery (n = 6, 46%). The enablers of effective feedback included timely feedback (n = 6, 46%), feedback provided in a goal-oriented and objective manner (n = 5, 40%), and student-specific feedback (n = 4, 30%). On the other hand, the most common impediments to feedback efficacy were providing extremely positive feedback and lack of constructive criticism. (4) Conclusions: Our findings highlight the importance of feedback model implementation in pharmacy education and preceptor training programs to ensure effective and quality feedback to pharmacy students.

Identity influences on medical students' orientation to feedback during third year clinical rotations

Medical students' feedback orientation (their attitudes about and preferences for feedback from preceptors) may change over the course of the third year of medical school and is likely influenced by identity-related factors. This study proposed that both how students view themselves personally (i.e., impostor syndrome) and how they view themselves in relation to the group (i.e., identification with the profession) are identity factors related to related to feedback orientation during clinical rotations. 177 third-year medical students enrolled in a four-phase longitudinal survey study beginning at the start of clinical rotations and continuing every twelve weeks of the academic year thereafter. Feedback orientation was conceptualized and measured as comprising aspects of utility (i.e., feedback is valuable and useful), sensitivity (i.e., feeling intimidated or threatened by corrective feedback), confidentiality (i.e., public/private context of feedback), and retention (i.e., feedback remembered). Results indicated that these aspects of feedback orientation did not significantly change during the third year. Instead, impostor syndrome was at least marginally, significantly associated with all aspects of feedback orientation across phases. Group identity was associated with feedback utility and retention, and female-identifying students reported significantly greater feedback confidentiality and feedback retention. Interventions may be needed to improve medical students' attitudes about feedback, particularly for those who experience impostor syndrome. Fostering group cohesion among medical students may influence how well students remember feedback and find it useful.

Assessing Healthcare Simulation Facilitation: A Scoping Review of Available Tools, Validity Evidence, and Context Suitability for Faculty Development in Low-Resource Settings

SUMMARY STATEMENT

Assessment tools support simulation facilitation skill development by guiding practice, structuring feedback, and promoting reflective learning among educators. This scoping review followed a systematic process to identify facilitation assessment tools used in postlicensure healthcare simulation. Secondary objectives included mapping of the validity evidence to support their use and a critical appraisal of their suitability for simulation faculty development in low-resource settings. Database searching, gray literature searching, and stakeholder engagement identified 11,568 sources for screening, of which 72 met criteria for full text review. Thirty sources met inclusion; 16 unique tools were identified. Tools exclusively originated from simulation practice in high-resource settings and predominantly focused on debriefing. Many tools have limited validity evidence supporting their use. In particular, the validity evidence supporting the extrapolation and implications of assessment is lacking. No current tool has high context suitability for use in low-resource settings.

Development and Pilot Testing of a Longitudinal Skills-Based Feedback and Conflict Resolution Curriculum for Internal Medicine Residents

PROBLEM

Physicians in training are responsible for leading clinical teams, coordinating interdisciplinary management, navigating conflict, and supervising and giving feedback to junior learners. Giving feedback and resolving conflict are key leadership skills for internal medicine (IM) residents, many of whom desire additional training. Although these skills are integral to successful leadership for physicians in training, residents receive little explicit education and existing curricula have not established best practices for skill acquisition.

APPROACH

Study authors designed a pilot longitudinal, skills-based curriculum to teach first- through third-year IM residents at the University of Pittsburgh how to give formative feedback and engage in conflict resolution. From February to May 2021, authors delivered a series of interactive lectures utilizing frameworks, workplace-based scenarios, skills practice, and discussion. Skills transfer was evaluated with novel pre- and postcurriculum objective structured clinical examinations (OSCEs) wherein participants played the role of senior resident. Each OSCE involved 2 feedback and 2 conflict resolution stations. OSCE performances were evaluated using an author-created checklist with a 1-4 rating scale. The exposure group comprised post-OSCE participants who attended the curriculum. Data were analyzed using a mixed effects regression model.

OUTCOMES

Thirty-six residents participated in curriculum evaluation, and 23 were included in postcurriculum data analysis. Within feedback, the skill "explores feedback content" significantly improved for exposure group participants (precurriculum median, 2.64; postcurriculum, 3.24; P < .05). For conflict resolution, among the exposure group, the skill "identifies a common goal, value, or purpose" significantly improved (pre, 3.10; post, 3.62; P < .05).

NEXT STEPS

This curriculum and evaluation can serve as a stepping stone for further evidence-based leadership frameworks, curricula, and evaluations developed specifically for physicians within their unique leadership roles. The feedback and conflict resolution frameworks used in this curriculum can be applied to various medical specialties, with specialty-relevant scenario adaptations for interactive skills practice.

Coaching in Competence by Design: A New Model of Coaching in the Moment and Coaching Over Time to Support Large Scale Implementation

Coaching is an increasingly popular means to provide individualized, learner-centered, developmental guidance to trainees in competency based medical education (CBME) curricula. Aligned with CBME's core components, coaching can assist in leveraging the full potential of this educational approach. With its focus on growth and improvement, coaching helps trainees develop clinical acumen and self-regulated learning skills. Developing a shared mental model for coaching in the medical education context is crucial to facilitate integration and subsequent evaluation of success. This paper describes the Royal College of Physicians and Surgeons of Canada's coaching model, one that is theory based, evidence informed, principle driven and iteratively and developed by a multidisciplinary team. The coaching model was specifically designed, fit for purpose to the postgraduate medical education (PGME) context and implemented as part of Competence by Design (CBD), a new competency based PGME program. This coaching model differentiates two coaching roles, which reflect different contexts in which postgraduate trainees learn and develop skills. Both roles are supported by the RX-OCR process: developing Relationship/Rapport, setting eXpectations, Observing, a Coaching conversation, and Recording/Reflecting. The CBD Coaching Model and its associated RX-OCR faculty development tool support the implementation of coaching in CBME. Coaching in the moment and coaching over time offer important mechanisms by which CBD brings value to trainees. For sustained change to occur and for learners and coaches to experience the model's intended benefits, ongoing professional development efforts are needed. Early post implementation reflections and lessons learned are provided.

Surgical Artificial Intelligence in Urology: Educational Applications

Surgical education has seen immense change recently. Increased demand for iterative evaluation of trainees from medical school to independent practice has led to the generation of an overwhelming amount of data related to an individual's competency. Artificial intelligence has been proposed as a solution to automate and standardize the ability of stakeholders to assess the technical and nontechnical abilities of a surgical trainee. In both the simulation and clinical environments, evidence supports the use of machine learning algorithms to both evaluate trainee skill and provide real-time and automated feedback, enabling a shortened learning curve for many key procedural skills and ensuring patient safety.

Evaluating Feedback Comments in Entrustable Professional Activities: A Cross-Sectional Study

INTRODUCTION

Competency-based medical education (CBME) has transformed postgraduate medical training, prioritizing competency acquisition over traditional time-based curricula. Integral to CBME are Entrustable Professional Activities (EPAs), that aim to provide high-quality feedback for trainee development. Despite its importance, the quality of feedback within EPAs remains underexplored.

METHODS

We employed a cross-sectional study to explore feedback quality within EPAs, and to examine factors influencing length of written comments and their relationship to quality. We collected and analyzed 1163 written feedback comments using the Quality of Assessment for Learning (QuAL) score. The QuAL aims to evaluate written feedback from low-stakes workplace assessments, based on 3 quality criteria (evidence, suggestion, connection). Afterwards, we performed correlation and regression analyses to examine factors influencing feedback length and quality.

RESULTS

EPAs facilitated high-quality written feedback, with a significant proportion of comments meeting quality criteria. Task-oriented and actionable feedback was prevalent, enhancing value of low-stakes workplace assessments. From the statistical analyses, the type of assessment tool significantly influenced feedback length and quality, implicating that direct and video observations can yield superior feedback in comparison to case-based discussions. However, no correlation between entrustment scores and feedback quality was found, suggesting potential discrepancies between the feedback and the score on the entrustability scale.

CONCLUSION

This study indicates the role of the EPAs to foster high-quality feedback within CBME. It also highlights the multifaceted feedback dynamics, suggesting the influence of factors such as feedback length and assessment tool on feedback quality. Future research should further explore contextual factors for enhancing medical education practices.

Confronting new challenges: Faculty perceptions of gaps in current laparoscopic curricula in a changing training landscape

Background

Opportunities for residents to develop laparoscopic skills have decreased with the rise in robotic operations and the development of complex, subspecialized laparoscopic operations. Given the changing training landscape, this study aimed to identify laparoscopic surgeons' perceptions of gaps in current laparoscopic skills in general surgery, obstetrics-gynecology, and urology residency programs.

Methods

Laparoscopic surgeons who operate with residents participated in semi-structured interviews. Questions addressed expectations for resident proficiency, deficits in laparoscopic surgical skills, and barriers to learning and teaching. Two authors independently coded de-identified transcripts followed by a conventional content analysis.

Results

Fourteen faculty members from thirteen subspecialties participated. Faculty identified three main areas to improve laparoscopic training across specialties: foundational knowledge, technical skills, and cognitive skills. They also recognized an overarching opportunity to address faculty development.

Conclusions

This qualitative study highlighted key deficiencies in laparoscopic training that have emerged in the current, changing era of minimally invasive surgery.

Key message

This qualitative study identified laparoscopic educators' perceptions of deficiencies in laparoscopic training. Findings emphasized the importance of incorporating high quality educational practices to optimize training in the current changing landscape of laparoscopic surgery.

Impact of women's feedback on midwifery student learning: A thematic analysis of students' reflections

PROBLEM

Little is known about the educational impact of providing routine, online feedback from women on midwifery student learning and clinical practice.

BACKGROUND

Feedback on students' clinical performance has historically been provided by lecturers and clinical supervisors. Women's feedback is not routinely collected or evaluated for impact on student learning.

AIM

To evaluate the impact of women's feedback about continuity of care experiences with a midwifery student on learning and practice.

DESIGN

Descriptive, exploratory qualitative study.

METHODS

All second-and third-year Bachelor of Midwifery students undertaking clinical placement between February and June 2022 at one Australian university, submitted formative, guided written reflections on de-identified women's feedback they received through their ePortfolio. Data were analysed using reflexive thematic analysis.

FINDINGS

Forty-four of the 69 eligible students (64%) submitted reflections on feedback received. Three themes emerged: 1) Confidence boosting, 2) Deeply integrating Midwifery Metavalues, and 3) Enhancing commitment to continuity. Three subthemes: connection, future practice and advocacy were identified. Women's feedback positively impacts student learning and places the woman in the educational feedback loop.

CONCLUSION

This study is an international first evaluating the impact of feedback from women on midwifery students' learning. Students reported greater confidence in their clinical practice, a deeper understanding of their midwifery philosophy, and an intention to advocate for, and work in, midwifery continuity models after graduation. Routine feedback about women's experiences should be embedded into midwifery education programs.

Undergraduate Learners' Receptiveness to Feedback in Medical Schools: A Scoping Review

Feedback from educators to learners is considered an important element of effective learning in medical school. While early studies were focused on the processes of providing feedback, recent work has showed that factors related to how learners receive feedback seems to be equally important. Considering that the literature on this topic is new in medical education, and studies are diverse and methodologically variable, we sought to conduct a scoping review to map the articles on receptiveness to feedback, to provide an overview of its related factors, to identify the types of research conducted in this area, and to document knowledge gaps in the existing literature. Using the Joanna Briggs Institute scoping review methodology, we searched four databases (CINAHL, Ovid, PubMed, and Web of Science) and screened 9120 abstracts, resulting in 98 articles for our final analysis. In this sample, 80% of studies on the feedback receiver were published in the last 10 years, and there is a vast variation in the studies' methodologies. The main factors that affect medical students' receptiveness to feedback are students' characteristics, feedback content, educators' credibility, and the learning environment. Feedback literacy is a very recent and rarely used term in medical education; therefore, an important area for further investigation. Lastly, we identified some gaps in the literature that might guide future research, such as studying receptiveness to feedback based on academic seniority and feedback literacy's long-term impacts on learning.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-023-01858-0.

Evolution of the feedback conversation in anaesthesia education: a narrative review

Over the past century, education has been a core component for improving patient safety. The initial focus was developing a curriculum and an assessment process. In recent decades, the value of work-based learning has come to the fore. Learning from work, or experiential learning, requires reflection, which is critically dependent on external feedback. Conceptions of feedback have moved from a transactional information transfer from the supervisor to the trainee to a learner-centred and collaborative process occurring in a complex socio-cultural environment. In this narrative review we describe the evolution of the feedback conversation, provide a model synthesising the core concepts of feedback, and offer some guidance for the development of effective feedback in anaesthesia education.

Exploring Fairness in Scholarly Development: Are We Creating Knowledge Storing Zombies or Curious, Creative and Critical Healthcare Professionals?

Scholarly doctors require research knowledge and skills (Ausbildung), as well as an academic mindset, which includes curiosity, creativity, and critical thinking (Bildung). However, in contrast to knowledge and skills, summative assessment of the development of an academic mindset is not so easy in an objective and so-called 'fair' way. As a result, in practice, assessing knowledge and skills tends to dominate in scholarly development. In this perspective, we explore the issues that arise when we give priority to objective assessment of knowledge and skills in scholarly development to safeguard fairness and, consequently, standardize educational procedures and learning pathways. We argue that eventually this approach may even result in hampered development of a true academic mindset and can be considered unfair rather than fair. To solve this, perhaps we should go back to the core business of the university and in the tradition of founder of the modern university Von Humboldt focus on shaping an academic mindset (Bildung). To rebalance Ausbildung and Bildung in academic education, we should go beyond the assumption that objectivity is a prerequisite for achieving fairness in assessment. Shifting the focus from pure objectivity to both objectivity and subjectivity in assessment as well as learning pathways can assist in protecting fairness and, as a result, bring back Bildung to medical education to ensure future doctors to be true scholars.

Teaching Observation as a Faculty Development Tool in Medical Education: A Scoping Review

INTRODUCTION

Health professions education often includes teaching observation to inform faculty development (FD) and indirectly improve student performance. Although these FD approaches are well received by faculty, they remain underused and/or underreported, with limited opportunities to receive feedback in workplace contexts. The goal of our study was to map the depth and breadth of education literature on the use of observation of teaching as a tool of professional development in medical education.

METHODS

Following the methodology by Arksey and O'Malley, we conducted a scoping review and searched four databases for articles published in English (final searches in April 2022).

RESULTS

Of 2080 articles identified, 45 met the inclusion criteria. All observation activities were associated with one of the following FD approaches: peer observation of teaching (23 articles, 51%), peer coaching (12, 27%), peer review (9, 20%), and the critical friends approach (1, 2%). Thirty-three articles (73%) concerned formative versions of the observation model that took place in clinical settings (21, 47%), and they tended to be a voluntary (27, 60%), one-off (18, 40%), in-person intervention (29, 65%), characterized by limited institutional support (13, 29%). Both barriers and challenges of teaching observation were identified.

DISCUSSION

This review identified several challenges and shortcomings associated with teaching observation, such as inadequate methodological quality of research articles, inconsistent terminology, and limited understanding of the factors that promote long-term sustainability within FD programs. Practical strategies to consider when designing an FD program that incorporates teaching observation are outlined.

A systematic review of effective quality feedback measurement tools used in clinical skills assessment

BACKGROUND

Objective Structured Clinical Examination (OSCE) is a valid tool to assess the clinical skills of medical students. Feedback after OSCE is essential for student improvement and safe clinical practice. Many examiners do not provide helpful or insightful feedback in the text space provided after OSCE stations, which may adversely affect learning outcomes. The aim of this systematic review was to identify the best determinants for quality written feedback in the field of medicine.   Methods: PubMed, Medline, Embase, CINHAL, Scopus, and Web of Science were searched for relevant literature up to February 2021. We included studies that described the quality of good/effective feedback in clinical skills assessment in the field of medicine. Four independent reviewers extracted determinants used to assess the quality of written feedback. The percentage agreement and kappa coefficients were calculated for each determinant. The ROBINS-I (Risk Of Bias In Non-randomized Studies of Interventions) tool was used to assess the risk of bias.

RESULTS

14 studies were included in this systematic review. 10 determinants were identified for assessing feedback. The determinants with the highest agreement among reviewers were specific, described gap, balanced, constructive and behavioural; with kappa values of 0.79, 0.45, 0.33, 0.33 and 0.26 respectively. All other determinants had low agreement (kappa values below 0.22) indicating that even though they have been used in the literature, they might not be applicable for good quality feedback. The risk of bias was low or moderate overall.

CONCLUSIONS

This work suggests that good quality written feedback should be specific, balanced, and constructive in nature, and should describe the gap in student learning as well as observed behavioural actions in the exams.  Integrating these determinants in OSCE assessment will help guide and support educators for providing effective feedback for the learner.

What does "Timely" Mean to Residents? Challenging Feedback Assumptions in Postgraduate Education

Introduction

Current orthodoxy states that feedback should be timely and face-to-face, yet the optimal timing and mode of delivery for feedback is unclear. We explored what "optimal timing" means from residents' points of view as feedback providers and receivers, to ultimately inform strategies to optimize feedback in training.

Methods

As near-peers who have dual roles in both providing and receiving feedback, 16 subspecialty (PGY4 and 5) internal medicine residents were interviewed about their perceptions of the optimal timing and format of feedback. Using constructivist grounded theory, interviews were conducted and analyzed iteratively.

Results

Drawing on their experiences as providers and recipients, residents described simultaneously considering and weighing multiple factors when deciding on when and how to provide feedback. These included their own readiness to engage in providing meaningful feedback, the perceived receptiveness of the learner and the apparent urgency of feedback delivery (e.g., if patient safety was at stake). Face-to-face verbal feedback was valued for encouraging dialogue but could be uncomfortable and limited by time constraints. Written feedback could be more honest and concise, and the possibility of asynchronous delivery had potential to overcome issues with timing and discomfort.

Discussion

Participants' perceptions of the optimal timing of feedback challenge current assumptions about the benefits of "immediate" versus "delayed". The concept of "optimal timing" for feedback was found to be complex and context-dependent, defying a formulaic approach. There may be a role for asynchronous and/or written feedback, which has potential to address unique issues identified issues in near-peer relationships.

Investigating feedback orientation in medical learners

BACKGROUND/PURPOSE

Feedback processes in health professions education (HPE) are not always successful. While recommendations to improve feedback provision dominate the literature, studying specific learner attributes that impact feedback uptake may also improve feedback processes. Feedback orientation is a concept from management science involving four dimensions of learner attributes and attitudes that impact their feedback uptake: utility, accountability, social awareness, and feedback self-efficacy. Feedback orientation may represent a valuable concept in HPE. We aimed to understand medical learners' feedback orientation at different stages in their development.

METHODS

We used the Feedback Orientation Scale, a 20-item survey instrument, for a cross-sectional analysis of feedback orientation in medical students and Internal Medicine residents at one large academic center. We performed descriptive statistics and analysis of variance for data analysis.

RESULTS

We found the same factors (dimensions) to feedback orientation in our population as in management science. Overall feedback orientation scores were high and were largely consistent across trainee levels. Utility was the domain that was highest across learners, whereas feedback self-efficacy was lowest.

CONCLUSIONS

Feedback orientation represents a useful concept to explore medical learners' attitudes toward feedback's role in their development. The four domains can help guide further nuanced feedback research and application.[Box: see text].

The framework of Systematic Assessment for Resilience (SAR): development and validation

BACKGROUND

Burnout and depression among health professions education (HPE) students continue to rise, leading to unwanted effects that ultimately jeopardise optimal medical care and patient health. Promoting the resilience of medical students is one solution to this issue. Several interventions have been implemented to foster resilience, but they focus on aspects other than the primary cause: the assessment system. The purpose of this study is to develop a framework to promote resilience in assessment planning and practice.

METHODS

We followed the guidelines suggested by Whetten for constructing a theoretical model for framework development. There were four phases in the model development. In the first phase, different literature review methods were used, and additional students' perspectives were collected through focus group discussions. Then, using the data, we constructed the theoretical model in the second phase. In the third phase, we validated the newly developed model and its related guidelines. Finally, we performed response process validation of the model with a group of medical teachers.

RESULTS

The developed systematic assessment resilience framework (SAR) promotes four constructs: self-control, management, engagement, and growth, through five phases of assessment: assessment experience, assessment direction, assessment preparation, examiner focus, and student reflection. Each phase contains a number of practical guidelines to promote resilience. We rigorously triangulated each approach with its theoretical foundations and evaluated it on the basis of its content and process. The model showed high levels of content and face validity.

CONCLUSIONS

The SAR model offers a novel guideline for fostering resilience through assessment planning and practice. It includes a number of attainable and practical guidelines for enhancing resilience. In addition, it opens a new horizon for HPE students' future use of this framework in the new normal condition (post COVID 19).

Measuring changes in nursing students' workplace performance following feedback encounters: A quasi-experimental study

BACKGROUND

Work-integrated learning and performance is intimately linked with assessment and feedback; however, empirical evidence concerning the impact of feedback on student performance in workplace settings is limited. A novel practice-based feedback intervention-the REMARK (nuRse fEedback iMplementAtion frameworRK) program-was developed and implemented. Sustainable feedback processes that involve both learners and learning partners can ultimately maximise learning outcomes, optimise self-regulation, and impact performance.

OBJECTIVES

To determine the impact of implementing an educational intervention based on best practice principles of feedback on undergraduate nursing student performance during clinical placements in acute healthcare settings.

METHODS

A non-equivalent, quasi-experimental design was used. A pre- and post-assessment strategy for both control and intervention groups was employed. The REMARK program was based on known attributes of effective feedback critiqued in the literature that foster interactions between a learner and learning partner and, hence, support dialogic feedback.

RESULTS

Clinical assessment data of 214 final-year nursing students collected at three time points were analysed. Using multivariate modelling, the results indicated that students participating in the REMARK programme (intervention group) had statistically higher performance scores than students in the control group when time and placement setting were controlled.

CONCLUSIONS

This research provides evidence that engagement with an intervention based on attributes of effective feedback improves nursing students' workplace performance during clinical placement. To achieve the greatest advantage, incorporating strategies that encourage students to adopt an active role in feedback conversations that relate to the student's learning goals are recommended.

Feedback in family medicine clerkships: a qualitative interview study of stakeholders in community-based teaching

Feedback is an important aspect of teaching and learning in medical education. Irrespective of the training environment, too little effective dialogic feedback occurs. Community-based outpatient learning environments, such as general practitioner practices, have heterogeneous framework conditions regarding feedback that decrease feedback quality. To improve feedback in this setting, characteristics of feedback in such learning environments must be considered. This study aims to reveal such characteristics from different perspectives and derive ideas for improving feedback in community-based learning environments. Three stakeholder groups in family medicine clerkships as an example of community-based learning environments (n = 15 students, n = 12 faculty and administrative staff, n = 13 general physician trainers) were interviewed for this study. Transcripts of the interviews were analysed with qualitative content analysis. All stakeholders interviewed note a lack of feedback between groups. Feedback in primary care practices takes place in specific contexts (e.g., during vs after a consultation, during vs at the end of the clerkship) and is provided in different ways (e.g., verbal vs nonverbal). Barriers of effective feedback in community-based settings are: lack of opportunity/initiation, fear of giving feedback, unawareness (of correct feedback and/or lack of prior experience with feedback), and little basis for feedback. Currently, the exchange between the university and community-based learning environments is limited to grading and report writing, with little sharing of meaningful information. The potential of a better exchange between those within community-based learning environments and the university to improve feedback processes is not reached. This exchange and the framework conditions specific for the community-based learning environment should be considered as parts of the structural dimension of feedback. Teachers and course managers of family medicine institutes are in an important position to shape these factors actively, working together with stakeholders of community-based teaching.

Beyond Competency: A Student Perspective on Growth Through Clerkship Feedback

This article proposes a paradigm shift from the competency-based model of clerkship feedback using checklists to a coaching-based, action plan-oriented process that centers on individualized student-oriented goals. Using a student perspective, the authors examine the feedback literature and put forward a proposal to use an impact model whose emphasis is to improve the learning climate for students. Several techniques are reviewed which include goal generation and creation of dynamic action plans. By intentionally focusing on coaching relationships as a platform for feedback, the learners and mentors share goals and the result of feedback becomes action-based behaviors which may help negate personal attribution and bias in the feedback process.

Using the Assessment of Reasoning Tool to facilitate feedback about diagnostic reasoning

OBJECTIVES

Idiosyncratic approaches to reasoning among teachers and limited reliable workplace-based assessment and feedback methods make teaching diagnostic reasoning challenging. The Assessment of Reasoning Tool (ART) was developed to fill this gap, but its utility and feasibility in providing feedback to residents has not been studied. We evaluated how the ART was used to assess, teach, and guide feedback on diagnostic reasoning to pediatric interns.

METHODS

We used an integrated mixed-methods approach to evaluate how the ART facilitates the feedback process between clinical teachers and learners. We collected data from surveys of pediatric interns and interviews of hospital medicine faculty at Baylor College of Medicine from 2019 to 2020. Interns completed the survey each time they received feedback from their attending that was guided by the ART. The preliminary intern survey results informed the faculty interview questions. We integrated descriptive statistics of the survey with the thematic analysis of the transcribed interviews.

RESULTS

Survey data (52 survey responses from 38 interns) and transcribed interviews (10 faculty) were analyzed. The ART framework provided a shared mental model which facilitated a feedback conversation. The ART-guided feedback was highly rated in terms of structure, content, and clarity in goal-setting while enabling new learning opportunities. Barriers to using the ART included limited time and inter-faculty variability of its use.

CONCLUSIONS

The ART facilitated effective and feasible faculty feedback to interns on their diagnostic reasoning skills.

The Practice of Feedback in Health Professions Education in the Hierarchical and Collectivistic Culture: a Scoping Review

The global trend in student assessment is moving towards outcome-based assessment that requires multiple systematic data points and continuous feedback. However, in hierarchical and collectivistic cultures, one-way communication is practised more often, leading to less dialogical feedback. This scoping review explored feedback practice in Asian educational setting. Based on the 17 articles selected, the findings were categorised into four themes, i.e. inhibiting or facilitating factors of feedback, influences of cultural factors on feedback, discrepancies between students' and teachers' perceptions of feedback and impact of feedback. Hierarchical and collectivist cultural aspects, such as preference for group feedback, are pertinent to feedback practice, which likely influence the readiness for programmatic assessment.

Gender bias in the evaluation of interns in different medical specialties: An archival study

INTRODUCTION

The field of medicine is characterized by within-field gender segregation: Gender ratios vary systematically by subdisciplines. This segregation might be, in part, due to gender bias in the assessment of women and men medical doctors.

METHODS

We examined whether the assessments, i.e. overall score, department scores and skills scores, interns receive by their superiors during their internship year, vary as a function of their gender and the representation of women in the field. We analyzed an archival data set from a large hospital in Israel which included 3326 assessments that were given to all interns who completed their internship year between 2015 and 2019.

RESULTS

Women received lower department scores and skills scores in fields with a low (versus high) representation of women. Men received higher scores in fields with a high (versus low) representation of men, yet there was no difference in their skills scores.

CONCLUSIONS

Women are evaluated more negatively in fields with a low representation of women doctors. Similarly, men are evaluated more negatively in fields with a low representation of men, yet this cannot be explained by their skills. This pattern of results might point to a gender bias in assessments. A better understanding of these differences is important as assessments affect interns' career choices and options.

Interprofessional bedside rounding improves quality of feedback to resident physicians

PURPOSE

Obtaining high quality feedback in residency education is challenging, in part due to limited opportunities for faculty observation of authentic clinical work. This study reviewed the impact of interprofessional bedside rounds ('iPACE™') on the length and quality of faculty narrative evaluations of residents as compared to usual inpatient teaching rounds.

METHODS

Narrative comments from faculty evaluations of Internal Medicine (IM) residents both on usual teaching service as well as the iPACE™ service (spanning 2017-2020) were reviewed and coded using a deductive content analysis approach.

RESULTS

Six hundred ninety-two narrative evaluations by 63 attendings of 103 residents were included. Evaluations of iPACE™ residents were significantly longer than those of residents on usual teams (109 vs. 69 words, p < 0.001). iPACE™ evaluations contained a higher average occurrence of direct observations of patient/family interactions (0.72 vs. 0.32, p < 0.001), references to interprofessionalism (0.17 vs. 0.05, p < 0.001), as well as specific (3.21 vs. 2.26, p < 0.001), actionable (1.01 vs. 0.69, p < 0.001), and corrective feedback (1.2 vs. 0.88, p = 0.001) per evaluation.

CONCLUSIONS

This study suggests that the iPACE™ model, which prioritizes interprofessional bedside rounds, had a positive impact on the quantity and quality of feedback, as measured via narrative comments on weekly evaluations.

Teaching and assessment of clinical diagnostic reasoning in medical students

BACKGROUND

Teaching diagnostic reasoning and giving feedback has an important role in medical education. Clinicians who teach may recognise errors, but be unfamiliar with the terminology used to describe them, leading to a lack of consistent and useful student feedback.

OBJECTIVE

This prospective project evaluation study aimed to develop an examiner training package regarding errors in diagnostic reasoning, utilising consistent language and feedback tool, and report on diagnostic reasoning errors in second year medical students over the transition from preclinical to early clinical training at objective structured clinical exams (OSCEs).

RESULTS

Likert questionnaire regarding examining, assessment and feedback pre- and post-training showed improvement in all measures, including examiner feedback confidence post training (p < .001). Students (n = 235) within the cohort were examined at the first preclinical OSCE 12 weeks into the teaching year and 236 students at the end of year OSCE. A range of 0-6 diagnostic reasoning errors were reported for individual students. When comparing mean history station scores at the preclinical OSCE for students who were observed to have diagnostic reasoning errors, students with 'poor pattern recognition' had a 4.2% lower score than those without this error type (p = .04, 95% CI of difference .14, 8.32), while those with 'unfocused data collection' error had a station score 7.7% lower than those without this error (p < .001, 95% CI of difference 3.50, 11.99). At the end of teaching year clinical OSCE, all common error types were associated with poorer performance. Error pattern shifted through the two longitudinal assessments, resulting in 'poor pattern recognition' having reduced and 'too narrow' and 'premature closure' increased rates.

CONCLUSIONS

Incorporating the identification and feedback of common diagnostic reasoning errors into existing clinical assessments was feasible and easy to implement. Understanding, identifying and providing consistent feedback on common errors assists educators and could guide curriculum design.

Multisource feedback in medical students' workplace learning in primary health care

BACKGROUND

In medical students' workplace learning, feedback is important for effective learning regarding communication and clinical skills. The provision of multisource feedback (MSF) in clinical practice with focus on the patient's perspective is rarely addressed in the literature. The overall objective was to explore the experience of MSF in medical students' clinical learning in primary healthcare (PHC).

METHODS

In the study, patients provided feedback by use of the Patient Feedback in Clinical Practice (PFCP) questionnaire. By use of adapted PFCP questionnaire versions peers and clinical supervisors provided feedback and students performed a self-evaluation. The MSF learning activity was evaluated using surveys (4-point Likert scale/open-ended questions), (students (n = 26), peers (n = 9) and clinical supervisors (n = 7)). Data were analysed using descriptive and qualitative content analysis.

RESULTS

Results (mean 4-point Likert scale) from participants evaluation of the MSF learning activity visualises the value of feedback in terms of patient-centred communication (students 3.50, peers 2.44 and clinical supervisors 3.57), guidance for further training (students 3.14, peers 2.89 and clinical supervisors 3.00) and clarification of pedagogical assignment (students 3.14, peers 2.89 and clinical supervisors 3.00). Thematic analysis of participants' free-text answers in the evaluation surveys resulted in three themes: (1) applicability of the MSF, (2) MSF - collaborative learning process and (3) MSF as a facilitator in students' clinical skills development. The participants experienced that the written MSF provided multi-facetted perspectives, which contributed to students' and peers' clinical and communication learning. MSF experience also enhanced clinical supervisors' feedback regarding communication skills, targeting the supervisors' pedagogical assignment.

CONCLUSION

Our findings indicate that MSF provided directly after a patient encounter, using the PFCP questionnaire as feedback provider, could be an adequate learning activity for medical students' workplace learning. The MSF, provided through the PFCP questionnaire, was experienced to neutralise and operationalise the provision of concrete feedback, facilitating peers' learning and clinical supervisors' tuition. The results visualise the importance of patients in MSF, as a valuable resource in students' workplace learning. Our study implies that this learning activity could be an applicable tool to facilitate learning and pedagogic development in clinical education in PHC.

Experiences of simulated patients in providing feedback in communication skills teaching for undergraduate medical students

BACKGROUND

Simulated Patients (SPs) are commonplace in the teaching of communication skills in medical education and can provide immediate feedback to students from the patient's perspective. The experiences of SPs and their perspective on providing feedback is an under-studied area. This study aims to explore SP experiences and views on feedback, factors influencing their feedback and implications for training.

METHODS

Using a constructivist grounded theory approach, we conducted six focus groups with 30 SPs. Participants included experienced simulated patients from a London-based actor agency, used in undergraduate teaching programmes of communication skills. Consistent with the principles of grounded theory, data was collected and analysed in an iterative process to identify themes.

RESULTS

Five over-arching themes were identified: 1.) Feedback processes, 2.) Challenges in providing feedback, 3.) Cumulative experiences, 4.) Web of interpersonal relationships and dynamics and 5.) Portraying the character and patient representations.

DISCUSSION

These SPs regarded the sharing of the emotions they experienced during the consultation as the focus of their feedback. Their preference was for giving a 'sandwich style' of feedback and 'out-of-role' approach. The relationship with facilitators and students and politeness conventions emerged as significant factors when providing feedback. Sensitivity to the social dynamics of groups and implicit facilitator expectations were challenges they experienced as was divergence in views of student performance.

CONCLUSION

This study explored SP experiences and perspectives on providing feedback. Findings reveal complex social and structural dynamics at play in providing feedback which have not been reported so far in the literature. It is recommended that these issues should be addressed in training of both SPs and facilitators, in addition to feedback guidelines.

Five ways for facilitators to get a grip on small group learning

Successful groups do not happen by chance, and they do not depend solely or even mainly on the interpersonal skills of the group members. Cooperative learning (CL) theory tells us that small groups are successful when facilitators structure and organize the small group to include each one of the five elements of cooperative learning. In this article, we have described each of these five elements as a way to get a grip on small group learning: positive interdependence, promotive interaction, group and individual accountability, interpersonal and small group skills, and group processing. To help our readers remember the five key elements of CL, the five ways to get a grip on small group facilitation, we have created an infographic.

Medical residents and attending physicians’ perceptions of feedback and teaching in the United States: a qualitative study

PURPOSE

Residents and attendings agree on the importance of feedback to resident education. However, while faculty report providing frequent feedback, residents often do not perceive receiving it, particularly in the context of teaching. Given the nuanced differences between feedback and teaching, we aimed to explore resident and attending perceptions of feedback and teaching in the clinical setting.

METHODS

We conducted a qualitative study of internal medicine residents and attendings from December 2018 through March 2019 at the Massachusetts General Hospital to investigate perceptions of feedback in the inpatient clinical setting. Residents and faculty were recruited to participate in focus groups. Data were analyzed using thematic analysis to explore perspectives and barriers to feedback provision and identification.

RESULTS

Five focus groups included 33 total participants in 3 attending (n=20) and 2 resident (n=13) groups. Thematic analysis of focus group transcripts identified 7 themes which organized into 3 thematic categories: (1) disentangling feedback and teaching, (2) delivering high-quality feedback, and (3) experiencing feedback in the group setting. Residents and attendings highlighted important themes in discriminating feedback from teaching. They indicated that while feedback is reactive in response to an action or behavior, teaching is proactive and oriented toward future endeavors.

CONCLUSION

Confusion between the critical concepts of teaching and feedback may be minimized by allowing them to each have their intended impact, either in response to prior events or aimed toward those yet to take place.

Profiles for Success: Examining the Relationship between Student Profiles and Clerkship Performance Using Latent Profile Analysis

PHENOMENON

Learning is a complex phenomenon in which learners can vary in their learning orientation, learning approaches, and perception of the learning environment. Learners may be characterized as mastery oriented or performance oriented, and this learning orientation can influence their learning approaches, such as whether to seek feedback or ask for help. The learning environment includes institutional assessment policies and informal interactions that emphasize outcomes and differentiation among students. When learners perceive the learning environment as performance oriented, they feel competition from other learners and fear negative feedback. This complex interplay of learning orientation, its influence on feedback-seeking behavior, and the environmental influence and reinforcement of both can be captured through learner profiles.

APPROACH

In this multi-institutional cross-sectional survey study of students from six U.S. medical schools, we investigated learner profiles that characterize how these domains manifest together across individual learners. We then determined how these profiles are associated with students' clerkship grades. Measures included the Patterns of Adaptive Learning for learning orientation, self-reports for learning approaches, and perceptions of the learning environment. We used latent profile analysis to cluster students who share common characteristics around orientation, behavior, and environment. The relationship between these profiles and the percentage of honors earned was examined using multiple regression analysis.

FINDINGS

We found four distinct learner profiles within 666 of 974 (68%) analyzable responses: (1) performance oriented with negative perceptions of environment, (2) mastery oriented with desirable learning approaches, (3) "average" group, and (4) mastery orientation only. Profile 1 (M = 39%) and Profile 4 (M = 38%) students received fewer clerkship honors compared with other profiles. Profile 2 students earned a significantly higher percentage of honors grades (M = 54%) compared with other students. Profile 3 students (average group) earned the second highest percentage of honors (M = 46%). Effect sizes comparing the percentage of honors varied from 0.32 to 0.65, indicating medium to large differences.

INSIGHTS

The highest performers, as measured by clerkship honors grades, were mastery oriented, perceived a more positive learning environment, and were comfortable asking questions and seeking feedback for their learning. The lowest performing students were performance oriented and had negative perceptions of their learning environments. Contrary to previous findings, we found that the relationship between mastery orientation and learning approaches is not unidirectional but rather seems to be mediated by perceptions of the learning environment.

How Perceived Burnout Alters Frontline Educators' Assessments in the Clinical Learning Environment

OBJECTIVE

This study explores frontline educators' experience with learner burnout in the clinical learning environment (CLE) and how perceived burnout may impact assessment.

METHODS

A survey was sent to 105 Pediatric Hospital Medicine faculty and fellows at 7 sites across the United States representing diverse CLEs. They were invited to participate in an 11-question web-based survey that was developed, edited, and revised in step-wise fashion. It consisted of 5-point Likert scale, forced-choice, and dichotomous questions. Data from the web-based survey were compiled for descriptive purposes and analyzed for trends.

RESULTS

The response rate was 53%. Most respondents (83%) perceive learner burnout at least once per year, but median comfort in identifying burnout was 3 (scale 1-5, 3 = neutral). Factors associated with comfort were education as primary niche (P = .01) and having wellness training (P = .045). In terms of the impact of perceived learner burnout impacts on assessment, 88% report impact on feedback sessions and 60% reported impact on summative assessment; most are more lenient. Stated belief in whether perceived burnout should or should not impact assessment had 60% sensitivity and 84% specificity in predicting whether it does.

CONCLUSIONS

Frontline educators routinely perceive learner burnout in the CLE and it often impacts educators' assessment of a learner. The discrepancy between the expected and actual impact may suggest amplification of known barriers to accurate assessment in the CLE. Comfort associated with wellness training suggests that interventions targeting frontline educators in the CLE may improve their ability to simultaneously support and assess learners.

Sharing perspectives on feedback: a combined resident-faculty workshop

BACKGROUND

Feedback is essential to medical education. Although the need for effective feedback delivery is well known, more recent focus is on understanding and strengthening the faculty-trainee relationship within which the feedback process is carried out. The authors developed and implemented a combined resident-faculty feedback workshop within a psychiatry residency training program to enhance participants' understanding of challenges residents and faculty experience with the feedback process.

METHODS

The one-hour workshop consisted of small group activities and large group discussions, focused on (i) feedback challenges for both residents and faculty and (ii) potential ways to address identified challenges. Participants completed pre-and post-workshop questionnaires to rate their level of understanding of, and answer open-ended questions regarding, feedback challenges. Mixed-methods assessment of questionnaire responses examined quantitative rating changes from pre- to post-workshop, as well as emergent qualitative themes from the open-ended responses.

RESULTS

From a pool of 30 workshop participants, 26 completed each of the pre- and post-workshop questionnaires. Overall, participants were satisfied with the programming. Important considerations for the feedback process were (i) specific/constructive/timely feedback, (ii) meeting logistical/administrative feedback requirements, (iii) setting norms/expectations of effective/routine feedback, and (iv) relational/emotional considerations surrounding feedback. It appeared both faculty and residents were able to increase perspective taking about how the other group perceived the feedback process.

CONCLUSIONS

This pilot project is one of the first to examine a joint resident-faculty workshop focused on understanding how faculty and residents can interact to better understand each other's perspective on the feedback process. Further work in this area is needed to identify common misperceptions and design programming to help correct them. Further research is also needed to examine the impact of such programming on the feedback process.

Training to Promote Empathic Communication in Graduate Medical Education: A Shared Learning Intervention in Internal Medicine and General Surgery

Background:

Objective:

Design:

Setting/Subjects:

Measurements:

Results:

Conclusions:

Application of Practice Oriented-Peer Review for Prosthodontics (PRO-PReP)-A Qualitative Study

OBJECTIVE

Communication skills diminish with time and must be applied and updated frequently. Due to various professional constraints, the dentists may not be able to attend training programs to sharpen their skills. During patient interactions, dentists may face difficult situations which they may be unable to handle and, consequently, make them overreact. Therefore, there is a need to provide a platform to freely discuss their feelings, ideas, and take opinion from peers.

METHODS

Training in communication skills customized for dealing with complete denture patients was conducted for the prosthodontic postgraduates. Based on feedback obtained, it was decided to have periodic meetings and the concept of Practice-Oriented-Peer Review for Prosthodontics (PrO-PReP) was introduced. This novel concept is a combination of the Relationship building, exploring Reactions, exploring Content, and Coaching (R2C2) model of residency education and the Balint method. The meetings were scheduled every one or two months based on the available caseload of the patients treated by the postgraduates.

RESULTS

The thematic analysis of the postgraduates' self-reflection during the sessions and the video recorded observations (assessed using the Kalamazoo scale) revealed that these sessions were effective in positively engaging the postgraduates to discuss their experiences, reflect on their performances, practice their newly gained skills, and learn from peer sharing.

CONCLUSION

The postgraduates felt that they have changed in their working style and were more confident to manage patients. They found such sessions very useful for being updated with the already-learned skills.

A systematic review of effective quality feedback measurement tools used in clinical skills assessment

Background: Objective Structured Clinical Examination (OSCE) is a valid tool to assess the clinical skills of medical students. Feedback after OSCE is essential for student improvement and safe clinical practice. Many examiners do not provide helpful or insightful feedback in the text space provided after OSCE stations, which may adversely affect learning outcomes. The aim of this systematic review was to identify the best determinants for quality written feedback in the field of medicine.   Methods: PubMed, Medline, Embase, CINHAL, Scopus, and Web of Science were searched for relevant literature up to February 2021. We included studies that described the quality of good/effective feedback in clinical skills assessment in the field of medicine. Four independent reviewers extracted determinants used to assess the quality of written feedback. The percentage agreement and kappa coefficients were calculated for each determinant. The ROBINS-I (Risk Of Bias In Non-randomized Studies of Interventions) tool was used to assess the risk of bias. Results: 14 studies were included in this systematic review. 10 determinants were identified for assessing feedback. The determinants with the highest agreement among reviewers were specific, described gap, balanced, constructive and behavioural; with kappa values of 0.79, 0.45, 0.33, 0.33 and 0.26 respectively. All other determinants had low agreement (kappa values below 0.22) indicating that even though they have been used in the literature, they might not be applicable for good quality feedback. The risk of bias was low or moderate overall. Conclusions: This work suggests that good quality written feedback should be specific, balanced, and constructive in nature, and should describe the gap in student learning as well as observed behavioural actions in the exams.  Integrating these determinants in OSCE assessment will help guide and support educators for providing effective feedback for the learner.

The Reliability and Medical Students' Appreciation of Certainty-Based Marking

Certainty-Based Marking (CBM) involves asking students not only the answer to an objective question, but also how certain they are that their answer is correct. In a mixed method design employing an embedded approach with a quasi-experimental design, we have examined the use of CBM during a 5-week Gynaecology and Obstetrics course. The study was conducted as a non-mandatory revision exam with two additional questionnaires on Moodle. Majority of students perceive CBM as fair (78%) and useful (94%). Most students would immediately want CBM to be used for revision exams, but more practice would be needed for CBM to be used in graded exams. The lowest self-evaluation of knowledge was mostly seen by worst (less than 70% Accuracy) and best achievers (more than 90% Accuracy); the worst achievers probably have knowledge gaps, and the best achievers probably correctly guessed at least one question. Our findings conclude that CBM does not discriminate any learner type (p = 0.932) and does not change the general distribution of the exam scores, since there is no significant differences between Certainty-Based Score (M = 80.4%, SD = 10.4%) and Accuracy (M = 79.8%, SD = 11.1%); t(176) = 0.8327, p = 0.4061. These findings are widely applicable, as learner type study models are used extensively in education. In the future, larger samples should be studied and the implementation of CBM on question types other than MCQ should be investigated.

Learning from patients' written feedback: medical students' experiences

METHODS

Fifty-nine medical students evaluated their learning experience of receiving patients' written feedback obtained from the PFCP questionnaire. Students (N = 57) evaluated their experiences by applying a nine-question evaluation survey (Likert scale N = 3 and free-text questions N = 6) and/or participated in a semi-structured interview (N = 6 students). Data were analyzed using descriptive statistics and qualitative content analysis.

RESULTS

The analysis of data from the students' evaluation survey was performed using 4-point Likert scale questions presented in mean, SD and range; ability to apply patient-centred communication (3.3, 0.74, 2-4), guidance for further clinical training of clinical skills (3.2, 1.31, 1-4) and visualization of the pedagogical assignment during an encounter (3.0, 1.68, 1-4). A content analysis of the free-text questions from the students' evaluation surveys and interviews resulted in three themes: (1) confidence in clinical practice, (2) application of patient-centred communication and (3) identification of learning needs.

CONCLUSIONS

The results indicate that patients' feedback facilitated a reflective self-directed learning process with the identification of learning needs and increased awareness of the patient as a collaborative partner during the encounter. Patients' written feedback adjacent to a patient encounter is identified as a valuable additional learning tool in medical students' workplace learning. Further studies are required to explore how patients' written feedback can be operationalized in different clinical contexts, for example, in in-patient care.