Guidelines: the do's, don'ts and don't knows of feedback for clinical education

Quality of dictated feedback associated with SIMPL operative assessments of pediatric surgical trainees

BACKGROUND

SIMPL is a workplace-based operative performance assessment tool which allows for dictated feedback (DF). To better understand the value of DF, we sought to characterize the type and quality of DF generated during SIMPL evaluations.

METHODS

Thematic analysis of DF from SIMPL assessments between June 2017 and December 2018 at a single pediatric surgery fellowship program was performed. Comments were categorized as specific, encouraging or corrective. Categories were combined to determine DF quality as effective, mediocre or ineffective.

RESULTS

Of 781 SIMPL assessments (21 faculty, 5 trainees), 451 (57%) had DF. Most comments were encouraging (93%) and specific (65%). Only 21% were corrective, 17% had entrustment features, and 8% had an explicit learning plan. Feedback quality was deemed mediocre (45%), ineffective (33%) and effective (21%).

CONCLUSION

SIMPL dictated feedback was mostly encouraging and specific. To improve quality, feedback should incorporate learning plans as well as corrective and entrustment features.

Best Approaches to Evaluation and Feedback in Post-Graduate Medical Education

PURPOSE OF REVIEW

The objectives of this literature review are to appraise current approaches and assess new technologies that have been utilized for evaluation and feedback of residents, with focus on surgical trainees.

RECENT FINDINGS

In 1999, the Accreditation Council for Graduate Medical Education introduced the Milestone system as a tool for summative evaluation. The organization allows individual program autonomy on how evaluation and feedback are performed. In the past, questionnaire evaluations and informal verbal feedback were employed. However, with the advent of technology, they have taken a different shape in the form of crowdsourcing, mobile platforms, and simulation. Limited data is available on new methods but studies show promise citing low cost and positive impact on resident education. No one "best approach" exists for evaluation and feedback. However, it is apparent that a multimodal approach that is based on the ACGME Milestones can be effective and aid in guiding programs.

2019 EULAR points to consider for the assessment of competences in rheumatology specialty training

BACKGROUND AND AIM

Striving for harmonisation of specialty training and excellence of care in rheumatology, the European League Against Rheumatism (EULAR) established a task force to develop points to consider (PtCs) for the assessment of competences during rheumatology specialty training.

METHODS

A systematic literature review on the performance of methods for the assessment of competences in rheumatology specialty training was conducted. This was followed by focus groups in five selected countries to gather information on assessment practices and priorities. Combining the collected evidence with expert opinion, the PtCs were formulated by the multidisciplinary task force, including rheumatologists, medical educationalists, and people with rheumatic and musculoskeletal diseases. The level of agreement (LoA) for each PtC was anonymously voted online.

RESULTS

Four overarching principles and 10 PtCs were formulated. The overarching principles highlighted the importance of assessments being closely linked to the rheumatology training programme and protecting sufficient time and resources to ensure effective implementation. In the PtCs, two were related to overall assessment strategy (PtCs 1 and 5); three focused on formative assessment and portfolio (PtCs 2-4); three focused on the assessment of knowledge, skills or professionalism (PtCs 6-8); one focused on trainees at risk of failure (PtC 9); and one focused on training the trainers (PtC 10). The LoA (0-10) ranged from 8.75 to 9.9.

CONCLUSION

These EULAR PtCs provide European guidance on assessment methods throughout rheumatology training programmes. These can be used to benchmark current practices and to develop future strategies, thereby fostering continuous improvement in rheumatology learning and, ultimately, in patient care.

Formalizing Feedback: Introducing a Structured Approach in an Outpatient Resident Clinic

OBJECTIVE

Delivering feedback is an integral part of graduate medical education. This paper will present how feedback research informed the development of a new feedback model and discuss its implementation and evaluation by residents in an outpatient psychiatry clinic.

METHOD

After reviewing research, a new feedback model of self-determined goal setting with guided objectives and quarterly formal in-person feedback sessions was implemented with 10 psychiatry residents during their 12-month outpatient experience in postgraduate year (PGY)-3. Residents received a pre-intervention survey to assess existing opinions of feedback and goal setting and a post-intervention survey to evaluate experiences with the new feedback model.

RESULTS

On the pre-intervention survey, 3 of 8 resident respondents indicated they had previously set goals, and only 4 of 8 predicted goal setting would be helpful, with average helpfulness rating of 3.62 (scale of 1 to 5). Cumulatively, 10 PGY-3 residents set 31 goals over the academic year. On the post-intervention survey, resident respondents rated the helpfulness of goal setting at 4.71 and quarterly, formal, in-person feedback meetings at 4.86. Success at reaching their self-determined goals was rated at 5 by all respondents.

CONCLUSIONS

Utilizing self-determined goals and formal in-person feedback sessions seemed to provide the framework for an effective feedback model in an outpatient resident clinic. This pilot project suggests that introducing formal feedback models can have a positive impact on resident clinical and educational growth. The data support expanding the model to assess its generalizability with the goal of furthering development of evidence-based feedback models.

Necessary but not sufficient: identifying conditions for effective feedback during internal medicine residents' clinical education

Competency-based medical education and programmatic assessment intend to increase the opportunities for meaningful feedback, yet these conversations remain elusive. By comparing resident and faculty perceptions of feedback opportunities within one internal medicine residency training program, we sought to understand whether and how principles underlying meaningful feedback could be supported or constrained across a variety of feedback opportunities. Using case-study qualitative methodology, interviews and focus groups were conducted to explore 19 internal medicine residents' and 7 faculty members' perceptions of feedback across a variety of feedback opportunities: coaching, mini-CEXs, in-training evaluation reports and routine clinical supervision. Our data analysis moved iteratively between developing conceptual understandings and fine-grained analyses, while attending to both deductive and inductive analysis. Our results suggest that all feedback opportunities, including those created through formalized assessments, can foster meaningful feedback if faculty establish a trusting relationship with the resident, base their feedback on direct observation and support resident learning. However, formalized assessments were often perceived as inhibiting the conditions for meaningful feedback. A coaching program provided a context in which meaningful feedback could arise, in part because faculty were supported in shifting their focus from patient to resident. Meaningful feedback in clinical education may be fostered across a variety of feedback opportunities, however, it is often constrained by assessment. We must consider whether increasing the frequency of formative assessments may inhibit efforts to improve our feedback cultures while, in contrast, freeing up faculty to focus on supporting resident learning could improve these cultures.

Quality of written feedback given to medical students after introduction of real-time audio monitoring of clinical encounters

BACKGROUND

Direct observation is necessary for specific and actionable feedback, however clinicians often struggle to integrate observation into their practice. Remotely audio-monitoring trainees for periods of time may improve the quality of written feedback given to them and may be a minimally disruptive task for a consultant to perform in a busy clinic.

METHODS

Volunteer faculty used a wireless audio receiver during the second half of students' oncology rotations to listen to encounters during clinic in real time. They then gave written feedback as per usual practice, as did faculty who did not use the listening-in intervention. Feedback was de-identified and rated, using a rubric, as strong/medium/weak according to consensus of 2/3 rating investigators.

RESULTS

Monitoring faculty indicated that audio monitoring made the feedback process easier and increased confidence in 95% of encounters. Most students (19/21 respondents) felt monitoring contributed positively to their learning and included more useful comments. 101 written evaluations were completed by 7 monitoring and 19 non-monitoring faculty. 22/23 (96%) of feedback after monitoring was rated as high quality, compared to 16/37 (43%) (p < 0.001) for monitoring faculty before using the equipment (and 20/78 (26%) without monitoring for all consultants (p < 0.001)). Qualitative analysis of student and faculty comments yielded prevalent themes of highly specific and actionable feedback given with greater frequency and more confidence on the part of the faculty if audio monitoring was used.

CONCLUSIONS

Using live audio monitoring improved the quality of written feedback given to trainees, as judged by the trainees themselves and also using an exploratory grading rubric. The method was well received by both faculty and trainees. Although there are limitations compared to in-the-room observation (body language), the benefits of easy integration into clinical practice and a more natural patient encounter without the observer physically present lead the authors to now use this method routinely while teaching oncology students.

An optimal environment for placement learning: listening to the voices of speech and language therapy students

BACKGROUND

Placements are a core component of learning within speech and language therapy (SLT) undergraduate and postgraduate university programmes. They facilitate the development of students' professional and clinical competencies and socialization into the profession. It is indicated that an optimal environment for placement learning often pivots on the mentoring, supervision and feedback the student receives.

AIMS

To explore the perspectives of student SLTs in the Republic of Ireland in relation to factors that would best support their learning and competency development on placement.

METHODS & PROCEDURES

Qualitative data were collected from an anonymous online student survey that consisted of eight open-ended questions. Thematic analysis was applied to the data. Excerpts from the data were selected to illustrate the themes constructed.

OUTCOMES & RESULTS

A total of 117 students responded. Four salient themes were generated that capture the students' perspectives of an optimal environment for placement learning and competency development.

CONCLUSIONS & IMPLICATIONS

This study supports quality assurance within the practice education of student SLTs and highlights aspects of an optimal learning environment that practice educators can strive to develop. In parallel, this study points to the need for improved supports from university personnel and placement site managers, and an increased need for student preparedness and self-reflection. Implications for continuing professional development specific to the role of a practice educator is described, in addition to the recommendation of an expanded perspective of supervision within SLT. What this paper adds What is already known on the subject Placements are an integral component of SLT undergraduate and postgraduate university programmes that enable students to translate theory to practice. Placement involves a transition from structured and predictable learning of the classroom to more dynamic learning environment within the placement site. Assessment of placement is carried out by practice educators using competency assessment tools. An optimal environment for placement learning often pivots on the mentoring, supervision and feedback that the student receives from their practice educator. What this paper adds to existing knowledge This study explores student SLTs' voices in relation to what they consider an optimal environment for placement learning should be and what they believe may best support them in their journey to develop their clinical competencies. It supports quality assurance of the practice education of our future colleagues. What are the potential or actual clinical implications of this work? The findings of this enquiry emphasize the distinction between competence as a SLT and competence as a practice educator and have subsequent implications for the content of continuing professional development for practice educators. In particular, an expanded perspective of the models and frameworks of supervision to promote and implement within the practice education of SLT students is presented.

Learning Conversations: An Analysis of the Theoretical Roots and Their Manifestations of Feedback and Debriefing in Medical Education

Feedback and debriefing are experience-informed dialogues upon which experiential models of learning often depend. Efforts to understand each have largely been independent of each other, thus splitting them into potentially problematic and less productive factions. Given their shared purpose of improving future performance, the authors asked whether efforts to understand these dialogues are, for theoretical and pragmatic reasons, best advanced by keeping these concepts unique or whether some unifying conceptual framework could better support educational contributions and advancements in medical education.The authors identified seminal works and foundational concepts to formulate a purposeful review and analysis exploring these dialogues' theoretical roots and their manifestations. They considered conceptual and theoretical details within and across feedback and debriefing literatures and traced developmental paths to discover underlying and foundational conceptual approaches and theoretical similarities and differences.Findings suggest that each of these strategies was derived from distinct theoretical roots, leading to variations in how they have been studied, advanced, and enacted; both now draw on multiple (often similar) educational theories, also positioning themselves as ways of operationalizing similar educational frameworks. Considerable commonality now exists; those studying and advancing feedback and debriefing are leveraging similar cognitive and social theories to refine and structure their approaches. As such, there may be room to merge these educational strategies as learning conversations because of their conceptual and theoretical consistency. Future scholarly work should further delineate the theoretical, educational, and practical relevance of integrating feedback and debriefing.

Student perspectives on competency-based portfolios: Does a portfolio reflect their competence development?

INTRODUCTION

Portfolio-based assessments require that learners' competence development is adequately reflected in portfolio documentation. This study explored how students select and document performance data in their portfolios and how they perceive these data to be representative for their competence development.

METHODS

Students uploaded performance data in a competency-based portfolio. During one clerkship period, twelve students also recorded an audio diary in which they reflected on experiences and feedback that they perceived to be indicants of their competence development. Afterwards, these students were interviewed to explore the extent to which the performance documentation in the portfolio corresponded with what they considered illustrative evidence of their development. The interviews were analyzed using thematic analysis.

RESULTS

Portfolios provide an accurate but fragmented picture of student development. Portfolio documentation was influenced by tensions between learning and assessment, student beliefs about the goal of portfolios, student performance evaluation strategies, the learning environment and portfolio structure.

DISCUSSION

This study confirms the importance of taking student perceptions into account when implementing a competency-based portfolio. Students would benefit from coaching on how to select meaningful experiences and performance data for documentation in their portfolios. Flexibility in portfolio structure and requirements is essential to ensure optimal fit between students' experienced competence development and portfolio content.

The Effect of Foregrounding Intended Use on Observers' Ratings and Comments in the Assessment of Clinical Competence

PURPOSE

Some educational programs have adopted the premise that the same assessment can serve both formative and summative goals; however, how observers understand and integrate the intended uses of assessment may affect the way they execute the assessment task. The objective of this study was to explore the effect of foregrounding a different intended use (formative vs summative learner assessment) on observer contributions (ratings and comments).

METHOD

In this randomized, experimental, between-groups, mixed-methods study (May-September 2017), participants observed 3 prerecorded clinical performances under formative or summative assessment conditions. Participants rated performances using a global rating tool and provided comments. Participants were then asked to reconsider their ratings from the alternative perspective (from which they were originally blinded). They received the opportunity to alter their ratings and comments and to provide rationales for their decision to change or preserve their original ratings and comments. Outcomes included participant-observers' comments, ratings, changes to each, and stated rationales for changing or preserving their contributions.

RESULTS

Foregrounding different intended uses of assessment data for participant-observers did not result in differences in ratings, number or type of comments (both emphasized evaluative over constructive statements), or the ability to differentiate among performances. After adopting the alternative perspective, participant-observers made only small changes in ratings or comments. Participant-observers reported that they engage in the process in an evaluative manner despite different intended uses.

CONCLUSIONS

Foregrounding different intended uses for assessments did not result in significant systematic differences in the assessment data generated. Observers provided more evaluative than constructive statements overall, regardless of the intended use of the assessment. Future research is needed to explore whether these results hold in social/workplace-based contexts and how they might affect learners.

Report from an Effort to Optimize Feedback in Undergraduate Psychiatry Training

OBJECTIVE

Few studies report about the feedback students receive in undergraduate psychiatry training. The aim of this study was to evaluate an effort to optimize feedback to medical students during rotations in psychiatry.

METHODS

A structured feedback tool was constructed and introduced during psychiatric rotations. At the end of each week during 3 weeks of clinical psychiatry training, 86 medical students anonymously reported whether they had received any feedback, if the new feedback tool had been used and if the received feedback felt valuable. At the end of rotations, students were also asked to leave text comments about their experience of the feedback tool. Course evaluations concerning perceived feedback, before and after introducing a feedback tool, were compared.

RESULTS

The 86 students reported about feedback at three occasions, leaving a total of 212 reports (82% response rate). In all reports, the students affirmed having received some feedback, either with or without the feedback tool. In the 123 (58%) reports where feedback was received with the feedback tool, 110 (89%) answered that the feedback was experienced as valuable. Among those 89 (42%) reports where feedback had been delivered without the feedback tool, 41 (46%) described the feedback as valuable. Course evaluation of perceived feedback improved.

CONCLUSIONS

Feedback to medical students during psychiatry rotations seems to be optimized by using a structured feedback tool.

Perceived Anxiety is Negligible in Medical Students Receiving Video Feedback During Simulated Core Practical Skills Teaching: A Randomised Trial Comparing Two Feedback Modalities

Introduction

The ability to undertake simple practical procedures is essential for graduating medical students and is typically assessed using simulated models. Feedback is a key component of the learning process in developing proficiency in these key skills. Video feedback (VF) has previously shown promise, however, negative effects of VF-related anxiety on performance have been previously reported. Our aim was to investigate for a difference in participant anxiety between supervised individualised video feedback (SIVF) and unsupervised generic video feedback (UGVF) when undertaking simulated basic practical procedures.

Methods

Undergraduate medical students participating in a clinical skills study to compare UGVF and SIVF completed a Likert scale questionnaire detailing perceived anxiety. During the study, students were recorded performing three basic surgical skills (simple interrupted suturing, intravenous cannulation, urinary catheterisation). Feedback was then provided by one of two methods: (1) SIVF - participant video footage reviewed together with a tutor providing targeted feedback, and (2) UGVF - participant video footage reviewed alone with concurrent access to a generic pre-recorded ‘expert tips' video clip for comparison. Each participant received SIVF and UGVF at least once.

Results

The majority of participants did not find either SIVF (81.7%) or UGVF (78.8%) stressful. Students had a strong preference for SIVF (77.5%) and disagreed that similar ‘face-to-face’ feedback had impaired learning in the past (80.3%).

Conclusion

Medical student-perceived anxiety is negligible when video feedback is employed during simulated core practical skill training.

Impressions on Reliability and Students' Perceptions of Learning in a Peer-Based OSCE

BACKGROUND

Peer assessment of performance in the objective structured clinical examination (OSCE) is emerging as a learning instrument. While peers can provide reliable scores, there may be a trade-off with students' learning. The purpose of this study is to evaluate a peer-based OSCE as a viable assessment instrument and its potential to promote learning and explore the interplay between these two roles.

METHODS

A total of 334 medical students completed an 11-station OSCE from 2015 to 2016. Each station had 1-2 peer examiners (PE) and one faculty examiner (FE). Examinees were rated on a 7-point scale across 5 dimensions: Look, Feel, Move, Special Tests and Global Impression. Students participated in voluntary focus groups in 2016 to provide qualitative feedback on the OSCE. Authors analysed assessment data and transcripts of focus group discussions.

RESULTS

Overall, PE awarded higher ratings compared with FE, sources of variance were similar across 2 years with unique variance consistently being the largest source, and reliability (r _φ ) was generally low. Focus group analysis revealed four themes: Conferring with Faculty Examiners, Difficulty Rating Peers, Insider Knowledge, and Observing and Scoring.

CONCLUSIONS

While peer assessment was not reliable for evaluating OSCE performance, PE's perceived that it was beneficial for their learning. Insight gained into exam technique and self-appraisal of skills allows students to understand expectations in clinical situations and plan approaches to self-assessment of competence.

Finding common ground: meta-synthesis of communication frameworks found in patient communication, supervision and simulation literature

BACKGROUND

Effective communication between patients-clinicians, supervisors-learners and facilitators-participants within a simulation is a key priority in health profession education. There is a plethora of frameworks and recommendations to guide communication in each of these contexts, and they represent separate discourses with separate communities of practice and literature. Finding common ground within these frameworks has the potential to minimise cognitive load and maximise efficiency, which presents an opportunity to consolidate messages, strategies and skills throughout a communication curriculum and the possibility of expanding the research agenda regarding communication, feedback and debriefing in productive ways.

METHODS

A meta-synthesis of the feedback, debriefing and clinical communication literature was conducted to achieve these objectives.

RESULTS

Our analysis revealed that the concepts underlying the framework can be usefully categorised as stages, goals, strategies, micro-skills and meta-skills. Guidelines for conversations typically shared a common structure, and strategies aligned with a stage. Core transferrable communication skills (i.e., micro-skills) were identified across various types of conversation, and the major differences between frameworks were related to the way that power was distributed in the conversation and the evolution of conversations along the along the path of redistributing power. As part of the synthesis, an overarching framework "prepare-EMPOWER enact" was developed to capture these shared principles across discourses.

CONCLUSIONS

Adopting frameworks for work-based communication that promote dialogue and empower individuals to contribute may represent an important step towards learner-centred education and person-centred care for patients.

The do's, don'ts and don't knows of establishing a sustainable longitudinal integrated clerkship

INTRODUCTION

The longitudinal integrated clerkship is a model of clinical medical education that is increasingly employed by medical schools around the world. These guidelines are a result of a narrative review of the literature which considered the question of how to maximize the sustainability of a new longitudinal integrated clerkship program.

METHOD

All four authors have practical experience of establishing longitudinal integrated clerkship programs. Each author individually constructed their Do's, Don'ts and Don't Knows and the literature that underpinned them. The lists were compiled and revised in discussion and a final set of guidelines was agreed. A statement of the strength of the evidence is included for each guideline.

RESULTS

The final set of 18 Do's, Don'ts and Don't Knows is presented with an appraisal of the evidence for each one.

CONCLUSION

Implementing a longitudinal integrated clerkship is a complex process requiring the involvement of a wide group of stakeholders in both hospitals and communities. The complexity of the change management processes requires careful and sustained attention, with a particular focus on the outcomes of the programs for students and the communities in which they learn. Effective and consistent leadership and adequate resourcing are important. There is a need to select teaching sites carefully, involve students and faculty in allocation of students to sites and support students and faculty though the implementation phase and beyond. Work is needed to address the Don't Knows, in particular the question of how cost-effectiveness is best measured.

Using Longitudinal Milestones Data and Learning Analytics to Facilitate the Professional Development of Residents: Early Lessons From Three Specialties

PURPOSE

To investigate the effectiveness of using national, longitudinal milestones data to provide formative assessments to identify residents at risk of not achieving recommended competency milestone goals by residency completion. The investigators hypothesized that specific, lower milestone ratings at earlier time points in residency would be predictive of not achieving recommended Level (L) 4 milestones by graduation.

METHOD

In 2018, the investigators conducted a longitudinal cohort study of emergency medicine (EM), family medicine (FM), and internal medicine (IM) residents who completed their residency programs from 2015 to 2018. They calculated predictive values and odds ratios, adjusting for nesting within programs, for specific milestone rating thresholds at 6-month intervals for all subcompetencies within each specialty. They used final milestones ratings (May-June 2018) as the outcome variables, setting L4 as the ideal educational outcome.

RESULTS

The investigators included 1,386 (98.9%) EM residents, 3,276 (98.0%) FM residents, and 7,399 (98.0%) IM residents in their analysis. The percentage of residents not reaching L4 by graduation ranged from 11% to 31% in EM, 16% to 53% in FM, and 5% to 15% in IM. Using a milestone rating of L2.5 or lower at the end of post-graduate year 2, the predictive probability of not attaining the L4 milestone graduation goal ranged from 32% to 56% in EM, 32% to 67% in FM, and 15% to 36% in IM.

CONCLUSIONS

Longitudinal milestones ratings may provide educationally useful, predictive information to help individual residents address potential competency gaps, but the predictive power of the milestones ratings varies by specialty and subcompetency within these 3 adult care specialties.

Beyond 'driving': The relationship between assessment, performance and learning

OBJECTIVE

Is the statement 'assessment drives learning' a myth?

BACKGROUND

Instructors create assessments and students respond to these assessments. Although such responses are often labelled indications of learning, the responses educators observe can also be considered a performance. When responses are aligned with generating stable changes, then assessment drives learning. When responses are not aligned with stable changes, we must consider them to be something else: a performance put on partially or fully for the sake of implying capability rather than actual learning. The alignment between the assessments educators create and the way students respond to these assessments is determined by the actions students take in our curriculum, in preparation for our assessments and after engaging with our assessments.

CONCLUSIONS

Not all assessments need to or should support learning, but when we assume all assessments 'drive learning', we endorse the myth that assessment is necessarily a formative aspect of our curricula. When we create assessments that encourage performance activities such as cramming, competing for tutorial airtime and impression management in the clinical setting we drive students to a performance. By thinking about how our students, institutions, curricula and assessments support learning and how well they support performance, we can modify and more fully align our curricular and assessment efforts to support learners in achieving their (and our) desired outcome. So, is the phrase 'assessment drives learning' a myth? This paper will conclude that it often is but we as educators must, through our leadership, move this myth towards a reality.

Exploring Faculty Perceptions of Competency-Based Medical Education and Assessing Needs for Implementation in Obstetrics and Gynaecology Residency

OBJECTIVE

The shift to competency-based medical education (CBME) is associated with changes in the way residents are taught and assessed. Although there are many purported benefits of CBME, an understanding of the preparedness of faculty to meet the needs of this new paradigm is lacking. The aim of this study was to characterize faculty needs to support the transition to CBME.

METHODS

An online survey was designed with the aim of characterizing faculty understanding of the principles of CBME and common trainee assessment methods, as well as exploring barriers to the implementation of CBME in obstetrics and gynaecology residency programs across Canada. The survey was sent to faculty across Canada in English and French.

RESULTS

A total of 284 responses were collected between September 2015 and December 2016. Although most faculty viewed CBME as a positive change, there were gaps in their knowledge about CBME and workplace-based assessment methods. Barriers to the implementation of CBME included lack of training in assessment of residents and feedback, financial implications, and time constraints.

CONCLUSION

To facilitate the transition to CBME, institutions may need to consider establishing faculty training programs and implementing systemic change aimed at addressing faculty needs and barriers during this fundamental shift in the structure of residency training.

Guidelines: The dos, don'ts and don't knows of remediation in medical education

INTRODUCTION

Two developing forces have achieved prominence in medical education: the advent of competency-based assessments and a growing commitment to expand access to medicine for a broader range of learners with a wider array of preparation. Remediation is intended to support all learners to achieve sufficient competence. Therefore, it is timely to provide practical guidelines for remediation in medical education that clarify best practices, practices to avoid, and areas requiring further research, in order to guide work with both individual struggling learners and development of training program policies.

METHODS

Collectively, we generated an initial list of Do's, Don'ts, and Don't Knows for remediation in medical education, which was then iteratively refined through discussions and additional evidence-gathering. The final guidelines were then graded for the strength of the evidence by consensus.

RESULTS

We present 26 guidelines: two groupings of Do's (systems-level interventions and recommendations for individual learners), along with short lists of Don'ts and Don't Knows, and our interpretation of the strength of current evidence for each guideline.

CONCLUSIONS

Remediation is a high-stakes, highly complex process involving learners, faculty, systems, and societal factors. Our synthesis resulted in a list of guidelines that summarize the current state of educational theory and empirical evidence that can improve remediation processes at individual and institutional levels. Important unanswered questions remain; ongoing research can further improve remediation practices to ensure the appropriate support for learners, institutions, and society.

Assessment in the context of problem-based learning

Arguably, constructive alignment has been the major challenge for assessment in the context of problem-based learning (PBL). PBL focuses on promoting abilities such as clinical reasoning, team skills and metacognition. PBL also aims to foster self-directed learning and deep learning as opposed to rote learning. This has incentivized researchers in assessment to find possible solutions. Originally, these solutions were sought in developing the right instruments to measure these PBL-related skills. The search for these instruments has been accelerated by the emergence of competency-based education. With competency-based education assessment moved away from purely standardized testing, relying more heavily on professional judgment of complex skills. Valuable lessons have been learned that are directly relevant for assessment in PBL. Later, solutions were sought in the development of new assessment strategies, initially again with individual instruments such as progress testing, but later through a more holistic approach to the assessment program as a whole. Programmatic assessment is such an integral approach to assessment. It focuses on optimizing learning through assessment, while at the same gathering rich information that can be used for rigorous decision-making about learner progression. Programmatic assessment comes very close to achieving the desired constructive alignment with PBL, but its wide adoption-just like PBL-will take many years ahead of us.

Students' Perspectives on Their Experience in Clinical Placements: Using a Modified Delphi Methodology to Engage Physiotherapy Stakeholders in Revising the National Form

Purpose: We developed an evidence-informed Student Evaluation of the Clinical Placement form. This form gives students the opportunity to share their feedback and perceptions of their clinical placement experiences and provides meaningful data to all stakeholders. Method: We used a modified Delphi process to engage a sample of national stakeholders: physiotherapy clinical education leads of academic departments, centre coordinators of clinical education, clinical instructors, and students. An expert consultant panel, in addition to the investigators, reviewed the responses from each round and helped develop the questionnaire for the subsequent round and finalize the evaluation form. Results: The response rate was 65.3% (47 of 72) for Round 1, 76.6% (36 of 47) for Round 2, and 100% (36 of 36) for Round 3. After three rounds of questionnaires, 89% of participants thought that the evaluation form met their needs. Conclusions: We developed a revised Student Evaluation of the Clinical Placement form that is informed by the literature and meaningful to all stakeholders. This form is being implemented in physiotherapy university programmes across Canada to enable students to share their experiences at clinical sites.

Assessment of communication skills

OBJECTIVE

This paper addresses how communication skills can best be assessed. Since assessment and learning are strongly connected, the way communication skills are best learned is also described.

RESULTS

Communication skills are best learned in a longitudinal fashion with ample practice in an authentic setting. Confrontation of behavior initiates the learning process and should be supported by meaningful feedback through direct observation. When done appropriately a set of (learned) communication skills become integrated skilled communication, being versatilely used in purposeful goal-oriented clinical communication. The assessment of communication skills should follow a modern approach to assessment where the learning function of assessment is considered a priority. Individual assessments are feedback-oriented to promote further learning and development. The resulting rich information may be used to make progression decisions, usually in a group or committee decision.

CONCLUSION

This modern programmatic approach to assessment fits the learning of skilled communication well.

PRACTICE IMPLICATIONS

Implementation of a programmatic assessment approach to communication will entail a major innovation to education.

Seeing is believing: an exploration of what residents value when they receive feedback

OBJECTIVE

The primary purpose of this paper is to report on psychiatry residents' perceptions of what is important when receiving feedback from evaluators.

METHODS

In January 2018, as part of the Harvard South Shore Psychiatry Residency Training Program's (HSS) ongoing local quality improvement efforts to enhance the delivery and effectiveness of feedback that residents receive from faculty during training, the authors disseminated a survey to psychiatry residents (n = 31) at HSS. Residents rated the level of importance of 17 statements pertaining to the way feedback is delivered. Two open-ended prompts also allowed respondents to share examples of growth-oriented and unhelpful feedback they have received during residency.

RESULTS

Twenty-seven residents responded (87% response rate). Eighty-one percent rated "the evaluator models the same behavior they're encouraging" as "extremely important" when receiving feedback. Many residents also rated the following survey items as "extremely important": "confidence in the evaluator's clinical and interpersonal skills" (63.0%), "amount of time the evaluator observed the resident" (51.9%), "there is a way to fix a performance deficit" (51.8%), and "specific feedback based off the resident's work" (48.1%). Conversely, only 11.1% of residents rated the feedback sandwich as "extremely important."

CONCLUSIONS

Despite a small sample size, this project demonstrated that, when receiving feedback, the majority of psychiatry residents strongly value when evaluators model the targeted behavior. The feedback sandwich was least important to residents. This project underscores the importance of evaluators serving as role models in the context of feedback, and findings can be used in faculty development activities focused on feedback delivery best practices.

Unearthing Faculty and Trainee Perspectives of Feedback in Internal Medicine: the Oral Case Presentation as a Model

BACKGROUND

The case presentation is a fundamental activity used in both patient care and trainee education, partly due to feedback from supervisor to trainee. Although feedback in medical education is well studied, prior studies have not focused on the perceptions of feedback by Internal Medicine supervisors and trainees as it relates to clinical activities like the case presentation.

METHODS

Semi-structured interviews were conducted with eight Internal Medicine physicians, and 18 Internal Medicine trainees (5 medical students, 13 residents) at the University of Toronto. Purposive sampling was used. Interviews were conducted and coded iteratively within a constructivist grounded theory approach until saturation was reached.

RESULTS

Supervisors and trainees recognized feedback as an important part of the case presentation that can be (1) explicit, labeled feedback or (2) implicit, unlabeled feedback. Both trainees and supervisors perceived that not enough feedback occurs, likely stemming from a hesitancy by supervisors to label implicit feedback, calling it an interruption instead. Although trainees were keenly aware of non-verbal feedback from their supervisors as implicit feedback, they often interpreted explicit constructive feedback negatively. Interestingly, the same feedback from senior residents was regarded as highly educational, as it was uncoupled from assessment.

CONCLUSION

Feedback occurs more frequently in case presentations than previously described, particularly in an implicit, unlabeled format. Even though under-recognized, trainees identify and utilize implicit feedback from supervisors, and coaching from senior residents, to develop learned behaviors. This is reassuring in the age of Competency-Based Medical Education, as feedback has an essential role in workplace-based assessment and promotion.

Resuscitation Education Science: Educational Strategies to Improve Outcomes From Cardiac Arrest: A Scientific Statement From the American Heart Association

The formula for survival in resuscitation describes educational efficiency and local implementation as key determinants in survival after cardiac arrest. Current educational offerings in the form of standardized online and face-to-face courses are falling short, with providers demonstrating a decay of skills over time. This translates to suboptimal clinical care and poor survival outcomes from cardiac arrest. In many institutions, guidelines taught in courses are not thoughtfully implemented in the clinical environment. A current synthesis of the evidence supporting best educational and knowledge translation strategies in resuscitation is lacking. In this American Heart Association scientific statement, we provide a review of the literature describing key elements of educational efficiency and local implementation, including mastery learning and deliberate practice, spaced practice, contextual learning, feedback and debriefing, assessment, innovative educational strategies, faculty development, and knowledge translation and implementation. For each topic, we provide suggestions for improving provider performance that may ultimately optimize patient outcomes from cardiac arrest.

'I'm never going to change unless someone tells me I need to': fostering feedback dialogue between general practice supervisors and registrars

Feedback is often unidirectional and focused on learners receiving feedback. Learning relationships are viewed as influential to promoting feedback dialogue. The aim of this study was to explore factors that facilitate or impede feedback between general practice supervisors and registrars. An in-depth qualitative study was conducted. Data collection featured semistructured interviews with registrars (n = 9) and supervisors (n = 5). Interviews were audio recorded and analysed interpretatively. Feedback was affected by personal (i.e. resilience, humility), relational (i.e. strength of supervisory relationship, power differentials) and contextual (i.e. culture) factors. Registrars are not accustomed to providing feedback and supervisors do not typically request feedback. Past feedback experiences affect registrar engagement in feedback exchanges. A culture of feedback dialogue within training organisations and training practices is essential. Power imbalance needs to be addressed, particularly for feedback by registrars. Strategies to develop feedback skills and promote an open feedback culture are essential.

Using conversation analysis to explore feedback on resident performance

Feedback on clinical performance of residents is seen as a fundamental element in postgraduate medical education. Although literature on feedback in medical education is abundant, many supervisors struggle with providing this feedback and residents experience feedback as insufficiently constructive. With a detailed analysis of real-world feedback conversations, this study aims to contribute to the current literature by deepening the understanding of how feedback on residents' performance is provided, and to formulate recommendations for improvement of feedback practice. Eight evaluation meetings between program directors and residents were recorded in 2015-2016. These meetings were analyzed using conversation analysis. This is an ethno-methodological approach that uses a data-driven, iterative procedure to uncover interactional patterns that structure naturally occurring, spoken interaction. Feedback in our data took two forms: feedback as a unidirectional activity and feedback as a dialogic activity. The unidirectional feedback activities prevailed over the dialogic activities. The two different formats elicit different types of resident responses and have different implications for the progress of the interaction. Both feedback formats concerned positive as well as negative feedback and both were often mitigated by the participants. Unidirectional feedback and mitigating or downplaying feedback is at odds with the aim of feedback in medical education. Dialogic feedback avoids the pitfall of a program director-dominated conversation and gives residents the opportunity to take ownership of their strengths and weaknesses, which increases chances to change resident behavior. On the basis of linguistic analysis of our real-life data we suggest implications for feedback conversations.

Student feedback experiences in a cross-border medical education curriculum

OBJECTIVES

To determine non-Western situated health professional student experiences and preferences for feedback in workplace-based settings.

METHODS

We conducted five focus groups with 27 students of Arab-origin enrolled in a Canadian-accredited cross-border pharmacy program in Qatar. Transcripts of recorded discussions were analyzed using the framework method.  Hofstede's and Hall's cultural dimension models were employed to understand described feedback encounters and behaviours.

RESULTS

We identified three themes associated with cultural influences on student feedback experiences, namely: 1) collectivism; 2) power distance; and 3) context.  Trainees described clinical supervisors who inadequately recognized individual performance, rejected critique, and insufficiently documented feedback onto the written in-training evaluation report. Conversely, students expected specific and timely feedback, invited criticism for learning, and desired clear written commentary.

CONCLUSIONS

Feedback behaviours of clinical supervisors, but not those of trainees, were consistent with local cultural norms as described by Hofstede and Hall.  Instead, feedback expectations of pharmacy students in Qatar largely echo those of other trainees enrolled in professional curricula situated outside the Middle East. Principles for optimal feedback in clinical training largely arise from Western perspectives but are not necessarily universal. Our work demonstrates that practices, in part, may be subject to local socio-cultural influences.  This is of particular importance in the experiential training component of cross-border medical education programs adopted by overseas institutions. Our findings also further add to the growing body of literature reporting suboptimal feedback in workplace-based learning, reinforcing the need to cultivate more student-centered practices in health professional training globally.

FEEDBK: a novel approach for providing feedback

BACKGROUND

High-quality feedback is fundamental to students' learning. We describe and evaluate FEEDBK, a novel feedback tool that encompasses the focus of learning, student self-evaluation (reflection), the encounter with the patient (professionalism), one task that the student should continue to do (reinforcement), one task that they could do better (improvement) and a key take-home message for self-directed learning.

METHODS

Medical students received regular bedside teaching and feedback during a 9-week clinical placement at a teaching hospital. Teaching fellows gave feedback in their usual style during weeks 1-3 and then used the FEEDBK tool during weeks 4-9. Questionnaires and focus groups were used to evaluate the students' experience of feedback before and after the introduction of FEEDBK.

RESULTS

FEEDBK significantly enhanced the feedback experience across six domains: identifying learning objectives, chance to reflect, feedback on communication skills/professionalism, specific suggestions for improvement, clear take-home message and personalised feedback. The analysis of the feedback from focus groups yielded four categories: (i) FEEDBK influenced the feedback process (delivering timely feedback); (ii) the nature of the feedback (enhancing both the quantity and the quality of the feedback received); (iii) the structure of the feedback; and (iv) the evolution of teaching and learning.

DISCUSSION

FEEDBK may enhance both the quantity and the quality of feedback and has the potential to improve the quality of teaching. It provides teachers with a quick and easy-to-follow framework to signpost curriculum-aligned feedback.

Where philosophy meets culture: exploring how coaches conceptualise their roles

CONTEXT

Although conceptually attractive, coaching in medicine remains ill-defined, with little examination of the transferability of coaching principles from other fields. Here we explore how coaching is enacted both within and outside of medicine; we aim to understand both the elements required for coaching to be useful and the factors that may influence its translation to the medical education context.

METHODS

In this constructivist grounded theory study, we interviewed 24 individuals across three groups: physicians who consider themselves coaches in clinical learning settings (n = 8), physicians with experience as sports, arts or business coaches (n = 10), and sports coaches without medical backgrounds (n = 6). Data collection and analysis were conducted iteratively using constant comparison to identify themes and explore their relationships.

RESULTS

We identified a shared philosophy of coaching, comprising three core elements that our participants endorsed regardless of the coaching context: (i) mutual engagement, with a shared orientation towards growth and development; (ii) ongoing reflection involving both learners and coaches, and (iii) an embrace of failure as a catalyst for learning. Enacting these features appeared to be influenced by culture, which affected how coaching was defined and developed, how the coaching role was positioned within the learning context, and how comfortably vulnerability could be expressed. Participants struggled to clearly define the coaching role in medicine, instead acknowledging that the lines between educational roles were often blurred. Further, the embrace of failure appeared challenging in medicine, where showing vulnerability was perceived as difficult for both learners and teachers.

CONCLUSIONS

Medical education's embrace of coaching should be informed by an understanding of both coach and learner behaviours that need to be encouraged and trained, and the cultural and organisational supports that are required to foster success.

Constructing a Shared Mental Model for Feedback Conversations: Faculty Workshop Using Video Vignettes Developed by Residents

INTRODUCTION

Providing feedback is a fundamental principle in medical education; however, as educators, our community lacks the necessary skills to give meaningful, impactful feedback to those under our supervision. By improving our feedback-giving skills, we provide concrete ways for trainees to optimize their performance, ultimately leading to better patient care.

METHODS

In this faculty development workshop, faculty groups used six feedback video vignettes scripted, enacted, and produced by residents to arrive at a shared mental model of feedback. During workshop development, we used qualitative analysis for faculty narratives combined with the findings from a focused literature review to define dimensions of feedback.

RESULTS

Twenty-three faculty (physical medicine and rehabilitation and neurology) participated in seven small-group workshops. Analysis of group discussion notes yielded 343 codes that were collapsed into 25 coding categories. After incorporating the results of a focused literature review, we identified 48 items grouped into 10 dimensions of feedback. Online session evaluation indicated that faculty members liked the workshop's format and thought they were better at providing feedback to residents as a result of the workshop.

DISCUSSION

Small faculty groups were able to develop a shared mental model of dimensions of feedback that was also grounded in medical education literature. The theme of specificity of feedback was prominent and echoed recent medical education research findings. Defining performance expectations for feedback providers in the form of a practical and psychometrically sound rubric can enhance reliable scoring of feedback performance assessments and should be the next step in our work.

On the Same Page: Building Best Practices of Peer Coaching for Medical Educators Using Nominal Group Technique

This article was migrated. The article was marked as recommended. Introduction: Peer coaching is a faculty development approach that improves teaching practice. Elements include peer observation of teaching, feedback, and collegial exchange. Peer coaching supports reflection on teaching, cultivates workplace learning, and fosters learning cultures. Yet, limited resources are available to guide faculty developers in designing and implementing peer coaching initiatives. This gap may lead to initiatives that fail to optimize teaching effectiveness. Methods: The authors convened a focus group of seven participant experts, via video-teleconference, to arrive at consensus on best practices of peer coaching for medical educators. The focus group utilized Nominal Group Technique, a consensus building methodology. Process steps included an introduction, silent idea generation, idea sharing, group discussion, and voting. Consensus was reached with over 50% agreement. Data were qualitatively analyzed using inductive content analysis, and quotes were extracted to support the identification of best practices. Results: Seventeen best practices were identified. All participant experts recommended a framework for the peer observation process including a pre-observation meeting and post-observation debrief. The participant experts stressed the importance of confidentiality and behaviorally-based feedback. To promote collegial exchange, most agreed peer coaching should be a formative process conducted in an environment that is safe and nonthreatening. Finally, peer coaching should be supported at multiple levels within an organization. Conclusion: Expert consensus generated 17 best practices of peer coaching for medical educators that optimize teaching effectiveness. The results provide a practical resource for faculty developers. Future researchers should explore common pitfalls and barriers to the implementation of peer coaching initiatives from the perspectives of academic leadership, peer coaches, and observed educators.

Influences on the implementation of Mini-CEX and DOPS for postgraduate medical trainees' learning: A grounded theory study

Introduction: In order for Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) to actually have a positive effect on trainees' learning, the way in which the tools are implemented is of key importance. However, there are many factors influencing their implementation. In this study, we aim to develop a comprehensive model of such factors. Methods: Using a constructivist grounded theory approach, we performed eight focus groups. Participants were postgraduate trainees and supervisors from three different specialties; all were experienced with Mini-CEX and/or DOPS. Data were analyzed for recurring themes, underlying concepts and their interactions using constant comparison. Results: We developed a model demonstrating how the implementation of Mini-CEX and DOPS for trainees' learning is influenced by 13 factors relating to four categories: organizational culture (e.g. value of teaching and feedback), work structure (e.g. time for Mini-CEX and DOPS, faculty development), instruments (e.g. content of assessment), and users (e.g. relationship between trainees and supervisors), and their interaction. Conclusions: We developed a complex model of influencing factors relating to four categories. Consideration of this model might support successful implementation and trainees' learning with Mini-CEX and DOPS.

Curated Collections for Educators: Eight Key Papers about Feedback in Medical Education

Feedback is an essential part of learning, growth, and academic success. Junior faculty members are often unfamiliar with the grounding literature that defines feedback. Many times they receive little education on providing and receiving feedback, resulting in unhelpful "feedback" for both learners and program leadership alike. This article aims to summarize eight key papers on feedback, to outline relevant information for emerging clinician educators, and identify ways to use these resources for the faculty development.  In order to generate a list of key papers that describes the importance and significance of feedback, the authors conducted a consensus-building process to identify the top papers. In August and September, 2018, the 2018-2019 Academic Life in Emergency Medicine (ALiEM) Faculty Incubator program discussed the topic of feedback in medical education. A number of papers on the topic was highlighted. This list of papers was further augmented using the suggestions and expertise of guest experts who are leaders in the field of medical education and feedback. The authors also used social media to conduct an open call on Twitter for important papers regarding feedback (utilizing #meded, #Feedback hashtags). Via this process, a list of 88 key papers was identified on the topic of feedback in medical education. After compiling these papers, the authorship group engaged in a modified Delphi approach to build consensus on the top eight papers on feedback. These papers were deemed essential by the authors and have been summarized with respect to their relevance to junior faculty members and to faculty developers. In this manuscript, we present eight key papers addressing feedback in medical education with discussions and applications for junior faculty members and faculty developers. This list of articles that can serve to help junior clinician educators grow in their ability to give effective feedback and also serve as resources upon which senior faculty can design the faculty development sessions.

Engaging Patients in Trainee Evaluations in Clinical Nutrition: a Pilot Intervention

BACKGROUND

Health professions trainee feedback rarely includes the patient voice, despite patients being experts in their own experience. Research supports including the patient voice; however, literature on the value of in-practice patient feedback is lacking.

ACTIVITY

A form was designed and implemented for patients to evaluate dietetic interns teaching a group class. Interns were surveyed to provide feedback.

RESULTS

Interns reported finding the feedback valuable and unique. The evaluations were felt to be credible.

DISCUSSION

Results support engaging patients in feedback for dietetic interns. Research should address different contexts, competencies other than communication and professionalism, and the impact to patients.

Provision of feedback to medical teachers on their educational performance: perspectives of internal medicine teachers

PURPOSE

Teachers' evaluation and giving feedback on their educational performance has been a great concern of all medical universities, especially in clinical settings. The opinion of teachers (as feedback recipients) about the feedback process and its usefulness is very critical for the feedback to be effective. This study aimed to identify the perspectives of internal medicine teachers on receiving feedback about their educational performance.

MATERIALS AND METHODS

In a qualitative content analysis study, from October 2016 to July 2017, 16 semi-structured interviews were conducted with the internal medicine teachers selected purposefully. Recorded and transcribed interviews were analyzed using an inductive content analysis approach.

RESULTS

Three themes emerged: "Feedback elements", "Feedback outcomes", and "Effective feedback strategies". The first includes two categories: human and background elements. "Human elements" consists of three subcategories including "Sources of feedback", "Feedback provider", and "Feedback recipient". "Background elements" includes three subcategories which are the underlying conditions in the feedback process: "Performing performance", "Data collection", and "Giving feedback". The third theme contains three categories: "Developing a systemic approach to feedback", "Appropriate educational policies for teacher recruitment and promotion", and "Using others' experiences".

CONCLUSION

The current study represents clinical teachers' opinions about the feedback elements, issues affecting the feedback process, and their recommendations for effective feedback. All these helped the researchers to develop a feedback cycle, which starts with "performing the performance" and continues with "performance data collection", "data analysis and interpretation", "feedback presentation", and "feedback follow-up". The researchers recommend the feedback officials to study teachers' opinions obtained in the current study and use the feedback cycle to provide more effective feedback.

Specific feedback makes medical students better communicators

BACKGROUND

Feedback is regarded a key element in teaching communication skills. However, specific aspects of feedback have not been systematically investigated in this context. Therefore, the aim of this study was to investigate the effectiveness of communication skills training (CST) integrating specific, structured and behavioral feedback.

METHODS

We condensed best practice recommendations for feedback in a CST for undergraduate medical students and compared the effect of specific, structured and behavior-orientated feedback (intervention group CST-behav) to general, experience-orientated feedback (CST- exp. as our control group) in a randomized controlled trial (RCT). We investigated changes on communication skills evaluated by independent raters, and evaluated by standardized patients (SP). To do that, every student was video-recorded in a pre and post assessment.

RESULTS

Sixty-six undergraduate medical students participated voluntarily in our study. Randomization did not result in equally skilled groups at baseline, so valid inter-group comparisons were not possible. Therefore, we analyzed the results of 34 students of our intervention group (CST-behav). Five out of seven domains in communication skills as evaluated by independent raters improved significantly, and there was a significant change in the global evaluation by SP.

CONCLUSIONS

Although we were unable to make between-group comparisons, the results of the within group pre-post evaluation suggest that specific feedback helps improve communication skills.

Giving feedback on others' writing

None

Assessment, feedback and the alchemy of learning

CONTEXT

Models of sound assessment practices increasingly emphasise assessment's formative role. As a result, assessment must not only support sound judgements about learner competence, but also generate meaningful feedback to guide learning. Reconciling the tension between assessment's focus on judgement and decision making and feedback's focus on growth and development represents a critical challenge for researchers and educators.

METHODS

We synthesise the literature related to this tension, framed around four trends in education research: (i) shifting perspectives on assessment; (ii) shifting perspectives on feedback; (iii) increasing attention on learners' perceptions of assessment and feedback, and (iv) increasing attention on the influence of culture on assessment and feedback. We describe factors that produce and sustain this tension.

RESULTS

The lines between assessment and feedback frequently blur in medical education. Models of programmatic assessment deliberately use the same data for both purposes: low-stakes individual data points are used formatively, but then are added together to support summative judgements. However, the translation of theory to practice is not straightforward. Efforts to embed meaningful feedback in programmes of learning face a multitude of threats. Learners may perceive assessment with formative intent as summative, restricting their engagement with it as feedback, and thus diminishing its learning value. A learning culture focused on assessment may limit learners' sense of safety to explore, to experiment, and sometimes to fail.

CONCLUSIONS

Successfully blending assessment and feedback demands clarity of purpose, support for learners, and a system and organisational commitment to a culture of improvement rather than a culture of performance.

A novel 6D-approach to radically transform undergraduate medical education: preliminary reflections from MBRU

BACKGROUND

Designers of undergraduate medical education (UME) need to address the exponentially expanding volume and variability of scientific knowledge, where by didactic teaching techniques need to be augmented by innovative student-centric pedagogical strategies and implementation of milieus, where information, communication and technology-enabled tools are seamlessly integrated, and lifelong information gathering, assimilation, integration and implementation is the ultimate goal. In UME, the basic sciences provide a solid scaffold allowing students to develop their personal critical decisional framework as well as define the understanding of normal human physiology, pivotal for the identification, categorization and management of pathophysiology. However, most medical schools confine themselves to "stagnant curricula", with the implementation of traditional "teacher centered" pedagogical techniques in the dissemination of the courses pertaining to basic sciences in UME.

METHOD

To tackle the above paucity, we present a novel "6D-Approach" for the dissemination of concepts in basic sciences through mentored journal-clubs. The approach is informed by a teaching principle derived from Constructivism. The technique in which the 6D-approach can be implemented in UME, is shown using an example from a first-year course of Molecular Biology and Principles of Genetics at our medical school. A reflection on the impact of 6D-Approach for students as well as instructors is also presented.

RESULT

The 6D-approach was positively received by the students and the formal feedback for the course: Molecular Biology and Principles of Genetics, where the approach was repeatedly employed, indicated that students expressed satisfaction with the teaching strategies employed in the course, with ~ 89% of the students in the cohort strongly agreeing with the highest grading score "extremely satisfied". Further, the approach through the use of mentored journal clubs encourages retention of knowledge, critical thinking, metacognition, collaboration and leadership skills in addition to self-evaluation and peer feedback.

CONCLUSION

Hence, through the 6D-Approach, our attempt is to initiate, advance and facilitate critical thinking, problem-solving and self-learning in UME, demonstrated by graduating accomplished, competent and safe medical practitioners.

The perspectives of physiotherapists on managing nonspecific low back pain following a training programme in cognitive functional therapy: A qualitative study

BACKGROUND

It has long been acknowledged that nonspecific chronic low back pain (NSCLBP) is associated with a complex combination of biopsychosocial (BPS) factors, and recent guidelines advocate that the management of back pain should reflect this multidimensional complexity. Cognitive functional therapy (CFT) is a behaviourally oriented intervention that targets patients' individual BPS profiles. Although the efficacy of CFT has been demonstrated in primary care, little evidence exists about the training requirements of this approach.

METHODS

Qualitative semistructured interviews were conducted with 10 physiotherapists working in primary care, who had undergone a formal training programme in CFT. A purposive sampling method was employed to seek the broadest perspectives. Thematic analysis was used to analyse the interview transcripts and capture the emergent themes.

RESULTS

Five main themes emerged: (i) the learning challenge; (ii) self-reported changes in confidence; (iii) self-reported changes in communication practice; (iv) self-reported changes in attitudes and understanding; (v) the physiotherapists felt that CFT was more effective than their usual approach for NSCLBP but identified barriers to successful implementation, which included a lack of time and difficulties in engaging patients with strong biomedical beliefs.

CONCLUSIONS

The study suggested that training in CFT has the capacity to produce self-reported changes in physiotherapists' attitudes, confidence and practice. The provision of such training has implications in terms of time and costs; however, this this may be warranted, given the physiotherapists' strong allegiance to the approach compared with their usual practice.

Exploring Faculty Approaches to Feedback in the Simulated Setting: Are They Evidence Informed?

INTRODUCTION

Feedback in clinical education and after simulated experiences facilitates learning. Although evidence-based guidelines for feedback exist, faculty experience challenges in applying the guidelines. We set out to explore how faculty approach feedback and how these approaches align with current recommendations.

METHODS

There is strong evidence for the following four components of feedback: feedback as a social interaction, tailoring content, providing specific descriptions of performance, and identifying actionable items. Faculty preceptors participated in feedback simulations followed by debriefing. The simulations were video recorded, transcribed, and analyzed qualitatively using template analysis to examine faculty approaches to feedback relative to evidence-informed recommendations.

RESULTS

Recorded encounters involving 18 faculty and 11 facilitators yielded 111 videos. There was variability in the extent to which feedback approaches aligned with recommended practices. Faculty behaviors aligned with recommendations included a conversational approach, flexibly adapting feedback techniques to resident context, offering rich descriptions of observations with specific examples and concrete suggestions, achieving a shared understanding of strengths and gaps early on to allow sufficient time for problem-solving, and establishing a plan for ongoing development. Behaviors misaligned with guidelines included prioritizing the task of feedback over the relationship, lack of flexibility in techniques applied, using generic questions that did not explore residents' experiences, and ending with a vague plan for improvement.

CONCLUSIONS

Faculty demonstrate variability in feedback skills in relation to recommended practices. Simulated feedback experiences may offer a safe environment for faculty to further develop the skills needed to help residents progress within competency-based medical education.

Exploring the relationship between examiners' memories for performances, domain separation and score variability

Background: OSCE examiners' scores are variable and may discriminate domains of performance poorly. Examiners must hold their observations of OSCE performances in "episodic memory" until performances end. We investigated whether examiners vary in their recollection of performances; and whether this relates to their score variability or ability to separate disparate performance domains. Methods: Secondary analysis was performed on data where examiners had: 1/scored videos of OSCE performances showing disparate student ability in different domains; and 2/performed a measure of recollection for an OSCE performance. We calculated measures of "overall-score variance" (the degree individual examiners' overall scores varied from the group mean) and "domain separation" (the degree to which examiners separated different performance domains). We related these variables to the measure of examiners' recollection. Results: Examiners varied considerably in their recollection accuracy (recognition beyond chance -5% to +75% for different examiners). Examiners' recollection accuracy was weakly inversely related to their overall score accuracy (R = -0.17, p < 0.001) and related to their ability to separate domains of performance (R = 0.25, p < 0.001). Conclusions: Examiners vary substantially in their memories for students' performances which may offer a useful point of difference to study processing and integration phases of judgment. Findings could have implication for the utility of feedback.

Bridging the gap to effective feedback in residency training: perceptions of trainees and teachers

BACKGROUND

Clinical feedback is an important part of residency training, yet literature suggests this complex interaction is not completely understood. In particular, little is known about what resident versus attending physicians expect as feedback. This study investigates this gap in knowledge by examining differences in interactions that residents and attendings view as feedback.

METHODS

Surveys containing sample clinical feedback scenarios were distributed to residents and attending physicians in emergency medicine and general surgery at a large academic medical center. Respondents were asked to decide whether useful feedback was provided in each scenario, and responses were compared between the two groups. Continuous features were summarized with medians, interquartile ranges (IQRs), and ranges; categorical features were summarized with frequency counts and percentages. Comparisons of features between residents and attendings were evaluated using Wilcoxon rank sum, chi-square, and Fisher exact tests. Statistical analyses were performed using version 9.4 of the SAS software package (SAS Institute, Inc.; Cary, NC). All tests were two-sided and p-values < 0.05 were considered statistically significant.

RESULTS

Seventy-two individuals responded to the survey out of approximately 110 invitations sent (65%), including 35 (49%) residents and 37 (51%) attendings. Of 35 residents, 31 indicated their level of training, which included 13 (42%) PGY-1, 9 (29%) PGY-2, 6 (19%) PGY-3, and 3 (10%) PGY-4, respectively. Of 37 attendings, 34 indicated the number of years since completion of residency or last fellowship, at a median of 9 years (IQR 4-14; range 1-31). No significant difference was found in residents' and attendings' perceptions of what constituted feedback in the sample scenarios.

CONCLUSIONS

While this study did not find a statistical difference in perception of feedback between residents and attendings, additional factors should be considered when investigating perceived feedback deficiencies. Further research is needed to better understand and improve the clinical feedback process.