Unfulfilled promise, untapped potential: feedback at the crossroads

Strategies to develop an efficient feedback system in medical education and its outcome - An interventional study

OBJECTIVE

The main aim of this study is to know the loopholes in the feedback system, strategies to remove these loopholes, and to measure the effects of an efficient feedback system in an institution.

METHODS

This research has three stages. In the first stage, we tried to find out the flaws in the feedback system of the institution based on published benchmarks. Then we tried to remove these flaws with the collaboration of different stakeholders. In the last stage of the study, we tried to measure the effectiveness of these corrective procedures by certain key performance indicators (KPIs).

RESULTS

After improvement strategies, there was a significant improvement in the KPIs like, the knowledge and attitude toward the feedback improved from 3.1 to 4.23 in students, and from 3.46 to 4.34 in faculty members, on a five-point scale, and the teaching standards improved as measured by students' performance in different courses in terms of learning objectives achievement, for example in internal medicine from 70.8% to 75.3%. Similar improvement was recorded in other KPIs as well.

CONCLUSION

An efficient feedback system is the backbone of education and by adopting effective feedback, individuals and institutions can excel in the field of medical education.

Feedback-based learning from viewpoints of surgical nursing students: A mixed-method study

BACKGROUND

Feedback-based learning (FBL) focuses on guiding the learning process according to educational objectives and the student's needs. This study aimed to investigate surgical nursing students' perceptions and explore their experiences of FBL.

METHOD

The present study used a mixed-methods sequential explanatory design that was conducted in the quantitative and qualitative phases. Surgical nursing students participated in the quantitative phase (n = 105). In the first phase, students completed two questionnaires about FBL and clinical feedback. Semi-structured face-to-face interviews were used to collect qualitative data in the second phase. Graneheim and Lundman's inductive approaches were used to analyse the qualitative data.

RESULTS

The mean (SD) score for students' perception of FBL was 3.99 ± 0.70. The qualitative results were explored in two themes, "motivational support for improvement" and "unpleasant learning".

CONCLUSION

In this study, the positive and negative aspects of FBL were explained. FBL is perceived as a motivational support mechanism to improve students' capabilities during their academic courses and also prepare them for future careers. Conversely, FBL may experience unpleasant learning due to negative feedback and negative emotions.

What do you think of when you hear the word 'feedback'? A reflective thematic analysis study of interviews

PURPOSE

Although most teaching around feedback focuses on the delivery, one must consider that the word 'feedback' is not a neutral word. It inflicts a range of emotions that, when used, may influence the effectiveness of the feedback process. A more profound understanding of health professions educators' perceptions regarding the word 'feedback' can help explain discrepancies between the provision, reception and acceptance of feedback.

METHODS

This is a qualitative inductive, reflective thematic analysis study. The authors interviewed 22 health professions educators participating in an online workshop to develop their feedback giving skills on their initial perspectives of the word 'feedback'.

RESULTS

We found four major themes: (1) Can I tell you a little story about my feedback experience? (2) It is probably going to be negative. (3) There is always something to learn if you are willing to hear the message. (4) It is like getting a report card. From the data, we suggest one key antecedent and two practical approaches one could take when giving feedback.

CONCLUSION

In this article, the authors highlight barriers during the feedback process due to the mere perception of the nature of feedback and the connotations associated with the term itself and suggest approaches that can refocus conversations towards a shared meaning and purpose of improvement, despite the preconceptions of the word 'feedback'.

Feedback practices in undergraduate clinical teaching in Sri Lanka - a qualitative study

BACKGROUND

Feedback is integral to medical education, enabling students to improve their knowledge, skills, and attitudes. Feedback practices may vary according to prevalent cultural and contextual factors. This study aimed to explore how feedback is conceptualized and practised in the clinical education of medical students in Sri Lanka.

METHODS

The study was conducted in three medical schools and affiliated hospitals that represent the cultural diversity of Sri Lanka. Purposive sampling was utilized to recruit clinical teachers and students who would provide rich information for the study. The study had three components: an observation study, interviews with clinical teachers and focus group discussions with clinical students. During the observation study, video recording was used as a data collection tool to observe feedback in real-life clinical teaching/learning settings. A constructivist grounded theory approach was adapted for analysis to explore current practices and perceptions inductively.

RESULTS

Feedback was conceptualised as spontaneous unidirectional provision of information for the improvement of students. It was often provided in public settings and in student groups. Error correction was the primary focus of feedback, but both teachers and students desired a balanced approach with reinforcement and reflection. Although the direct approach to corrective feedback was found beneficial for student learning, participants agreed that harsh feedback was to be avoided. The hierarchical culture and lack of programmed feedback in the curricula influenced feedback practices, suggesting the need for modification.

CONCLUSIONS

This study highlighted feedback practices in the local context, emphasizing the need to address the hierarchical gap in clinical settings, balance reinforcement and correction, and promote dialogue and reflection in the feedback processes. The findings will help clinical teachers from both the global south as well as the global north to recognize cultural and contextual differences in providing feedback.

Coaching in Competence by Design: A New Model of Coaching in the Moment and Coaching Over Time to Support Large Scale Implementation

Coaching is an increasingly popular means to provide individualized, learner-centered, developmental guidance to trainees in competency based medical education (CBME) curricula. Aligned with CBME's core components, coaching can assist in leveraging the full potential of this educational approach. With its focus on growth and improvement, coaching helps trainees develop clinical acumen and self-regulated learning skills. Developing a shared mental model for coaching in the medical education context is crucial to facilitate integration and subsequent evaluation of success. This paper describes the Royal College of Physicians and Surgeons of Canada's coaching model, one that is theory based, evidence informed, principle driven and iteratively and developed by a multidisciplinary team. The coaching model was specifically designed, fit for purpose to the postgraduate medical education (PGME) context and implemented as part of Competence by Design (CBD), a new competency based PGME program. This coaching model differentiates two coaching roles, which reflect different contexts in which postgraduate trainees learn and develop skills. Both roles are supported by the RX-OCR process: developing Relationship/Rapport, setting eXpectations, Observing, a Coaching conversation, and Recording/Reflecting. The CBD Coaching Model and its associated RX-OCR faculty development tool support the implementation of coaching in CBME. Coaching in the moment and coaching over time offer important mechanisms by which CBD brings value to trainees. For sustained change to occur and for learners and coaches to experience the model's intended benefits, ongoing professional development efforts are needed. Early post implementation reflections and lessons learned are provided.

"Maybe it's the first time somebody's been honest with you": exploring how residents reconcile feedback variability

BACKGROUND

Supervisors in postgraduate medical education may deliver different feedback for the same quality of performance. Residents may struggle to make sense of inconsistent and sometimes contradictory information. We sought to explore how residents experience feedback from different supervisors, how they process inconsistent information, and what factors influence their experiences.

METHODS

Eighteen residents participated in semi-structured interviews to explore their perspectives on feedback. Using a constructivist grounded theory approach, we engaged in iterative cycles of data collection and analysis, sampling until theoretical sufficiency was reached. Constant comparative analysis was used to identify and define themes.

RESULTS

We identified a central theme of reconciliation, which we defined as the act of processing inconsistent feedback and determining how to engage with it. This reconciliation was informed by the credibility of, and residents' relationship with, supervisors and was achieved through conversations with peers and mentors, observation of other supervisors' behavior toward their performance, and reflection on their own performance. Participants expressed a reluctance to discard feedback, even if they felt it was incongruent with previous feedback or their own self-concept and self-assessment.

CONCLUSION

The findings of this study show that while residents are regular consumers of feedback, not all feedback is used equally. Residents actively reconcile sometimes-contradictory feedback and must work to balance a general reluctance to discard feedback, while developing an understanding of its credibility. This work reinforces the importance of pedagogical relationships and identifies that facilitated reflection that explicitly acknowledges feedback inconsistencies may be important in the reconciliation process.

Students' Experiences of Peer Observed Teaching: A Qualitative Interview Study

PHENOMENON

Development of teaching skills is an important aspect of medical student training. One method of developing teaching skills is participation in peer teaching with observation and feedback from peers. This study aims to explore student teachers' experiences of peer observation of teaching and how they intend to utilize this feedback.

APPROACH

We conducted individual semi-structured interviews with peer tutors who had experienced peer observation of their small group teaching and subsequent feedback. The interviews were conducted by a medical student peer not involved in the peer observation of teaching scheme. They were audio recorded and transcribed. The pseudonymised transcripts were coded independently by two researchers using thematic analysis.

FINDINGS

Nine students participated in interviews lasting a mean of 42 minutes. We identified three main themes: motivations for observation, experiences of observation, and responses to feedback. Students were motivated to have their teaching observed by both intrinsic and extrinsic factors: to develop their skills and competence as a teacher, in recognition of the important role this plays in their career, to provide reassurance that they are providing good quality teaching, to ensure the content of their teaching is appropriate and accurate, and to provide evidence of engagement in, and development of, teaching. Students described feeling nervous before the observations and preparing more for their teaching than they might normally, however, during the observations they felt more comfortable which they attributed to the peer-peer relationship. Students described finding the narrative feedback more useful than the quantitative elements as it provided more detail as to how they might improve. Several students described how they have used the feedback they have received on their teaching to improve subsequent sessions.

INSIGHTS

Peer observation of teaching is a useful and acceptable method of providing feedback on student teaching and recipients intend to use this feedback to improve their teaching.

Predictors of faculty narrative evaluation quality in medical school clerkships

INTRODUCTION

Narrative approaches to assessment provide meaningful and valid representations of trainee performance. Yet, narratives are frequently perceived as vague, nonspecific and low quality. To date, there is little research examining factors associated with narrative evaluation quality, particularly in undergraduate medical education. The purpose of this study was to examine associations of faculty- and student-level characteristics with the quality of faculty member's narrative evaluations of clerkship students.

METHODS

The authors reviewed faculty narrative evaluations of 50 students' clinical performance in their inpatient medicine and neurology clerkships, resulting in 165 and 87 unique evaluations in the respective clerkships. The authors evaluated narrative quality using the Narrative Evaluation Quality Instrument (NEQI). The authors used linear mixed effects modelling to predict total NEQI score. Explanatory covariates included the following: time to evaluation completion, number of weeks spent with student, faculty total weeks on service per year, total faculty years in clinical education, student gender, faculty gender, and an interaction term between student and faculty gender.

RESULTS

Significantly higher narrative evaluation quality was associated with a shorter time to evaluation completion, with NEQI scores decreasing by approximately 0.3 points every 10 days following students' rotations (p = .004). Additionally, women faculty had statistically higher quality narrative evaluations with NEQI scores 1.92 points greater than men faculty (p = .012). All other covariates were not significant.

CONCLUSIONS

The quality of faculty members' narrative evaluations of medical students was associated with time to evaluation completion and faculty gender but not faculty experience in clinical education, faculty weeks on service, or the amount of time spent with students. Findings advance understanding on ways to improve the quality of narrative evaluations which are imperative given assessment models that will increase the volume and reliance on narratives.

Peer feedback in graduate training: A phenomenological investigation of possibilities

BACKGROUND

Whilst feedback is an essential component of clinical education, it is often lacking in clinical workplaces due to competing priorities. Peer feedback has been proposed as a potential solution but remains underexplored in terms of practicality and effects. We aimed to examine the experiences of peer feedback among paediatric trainees, and the associated feedback culture.

METHODS

Following an Interpretative Phenomenological Analysis approach, the personal experiences of 12 paediatric trainees were explored using semi-structured interviews. Interpretive themes were developed between the transcripts using processes of abstraction, subsumption, contextualisation, and cross-case analysis. Themes were subsequently subjected to member checking and peer debriefing processes.

RESULTS

We found that peer feedback was influenced by three contextual factors, namely, prevalent feedback culture, interpersonal consent, and the quality of relationships. Peer feedback culture was lacking in clinical workplaces. Feedback between peers was constrained by avoiding criticism and maintaining work relationships. Social and cultural norms inhibited constructive peer feedback without explicit consent.

CONCLUSIONS

Enabling peer feedback in clinical settings requires attention to cultural, relational, and consent barriers. Potential approaches should include helping clinicians to develop greater cultural reflexivity, resident training in how to be peer educators, and enhancing institutional supports for peer feedback.

Facilitating Residents' Understanding of Electronic Health Record Report Card Data Using Faculty Feedback and Coaching

PURPOSE

Feedback continues to present a challenge for competency-based medical education. Clear, consistent, and credible feedback is vital to supporting one's ongoing development, yet it can be difficult to gather clinical performance data about residents. This study sought to determine whether providing residents with electronic health record (EHR)-based report cards, as well as an opportunity to discuss these data with faculty trained using the R2C2 model, can help residents understand and interpret their clinical performance metrics.

METHOD

Using action research methodology, the author team collected EHR data from July 2017 to February 2020, for all residents (n = 21) in one 5-year Emergency Medicine program and created personalized report cards for each resident. During October 6-17, 2020, 8 out of 17 eligible residents agreed to have their feedback conversations recorded and participate in a subsequent interview with a nonphysician member of the research team. Data were analyzed using thematic analysis, and the authors used inductive analysis to identify themes in the data.

RESULTS

In analyzing both the feedback conversations as well as the individual interviews with faculty and residents, the authors identified 2 main themes: (1) Reactions and responses to receiving personalized EHR data and (2) The value of EHR data for assessment and feedback purposes. All participants believed that EHR data metrics are useful for prompting self-reflection, and many pointed to their utility in providing suggestions for actionable changes in their clinical practice. For faculty, having a tool through which underperforming residents can be shown "objective" data about their clinical performance helps underscore the need for improvement, particularly when residents are resistant.

CONCLUSIONS

The EHR is a valuable source of educational data, and this study demonstrates one of the many thoughtful ways it can be used for assessment and feedback purposes.

Beyond Checking Boxes: Exploring Tensions With Use of a Workplace-Based Assessment Tool for Formative Assessment in Clerkships

PURPOSE

To understand the role of a workplace-based assessment (WBA) tool in facilitating feedback for medical students, this study explored changes and tensions in a clerkship feedback activity system through the lens of cultural historical activity theory (CHAT) over 2 years of tool implementation.

METHOD

This qualitative study uses CHAT to explore WBA use in core clerkships by identifying feedback activity system elements (e.g., community, tools, rules, objects) and tensions among these elements. University of California, San Francisco core clerkship students were invited to participate in semistructured interviews eliciting experience with a WBA tool intended to enhance direct observation and feedback in year 1 (2019) and year 2 (2020) of implementation. In year 1, the WBA tool required supervisor completion in the school's evaluation system on a computer. In year 2, both students and supervisors had WBA completion abilities and could access the form via a smartphone separate from the school's evaluation system.

RESULTS

Thirty-five students participated in interviews. The authors identified tensions that shifted with time and tool iterations. Year 1 students described tensions related to cumbersome tool design, fear of burdening supervisors, confusion over WBA purpose, WBA as checking boxes, and WBA usefulness depending on clerkship context and culture. Students perceived dissatisfaction with the year 1 tool version among peers and supervisors. The year 2 mobile-based tool and student completion capabilities helped to reduce many of the tensions noted in year 1. Students expressed wider WBA acceptance among peers and supervisors in year 2 and reported understanding WBA to be for low-stakes feedback, thereby supporting formative assessment for learning.

CONCLUSIONS

Using CHAT to explore changes in a feedback activity system with WBA tool iterations revealed elements important to WBA implementation, including designing technology for tool efficiency and affording students autonomy to document feedback with WBAs.

Affecting Change: Enhancing Feedback Interactions with Radiology Trainees

Feedback is a critical part of the learning process and is a valuable tool to empower adult learners. Modern feedback theory places the learner at the center of the feedback encounter. Individual and institutional barriers to effective giving and receiving of feedback can be overcome through education and attention to the form and content of feedback. We review the elements of effective feedback and address issues of framing, environmental, and social factors which aid in providing psychological safety and trust, as necessary elements to create a culture of feedback in radiology training programs. We provide practical strategies to empower learners with the necessary skills to solicit, receive, and reflect on feedback.

The pursuit of fairness in assessment: Looking beyond the objective

Health professions education has undergone significant changes over the last few decades, including the rise of competency-based medical education, a shift to authentic workplace-based assessments, and increased emphasis on programmes of assessment. Despite these changes, there is still a commonly held assumption that objectivity always leads to and is the only way to achieve fairness in assessment. However, there are well-documented limitations to using objectivity as the 'gold standard' to which assessments are judged. Fairness, on the other hand, is a fundamental quality of assessment and a principle that almost no one contests. Taking a step back and changing perspectives to focus on fairness in assessment may help re-set a traditional objective approach and identify an equal role for subjective human judgement in assessment alongside objective methods. This paper explores fairness as a fundamental quality of assessments. This approach legitimises human judgement and shared subjectivity in assessment decisions alongside objective methods. Widening the answer to the question: 'What is fair assessment' to include not only objectivity but also expert human judgement and shared subjectivity can add significant value in ensuring learners are better equipped to be the health professionals required of the 21st century.

Culture and understanding the role of feedback for health professions students: realist synthesis protocol

INTRODUCTION

Clinical education has moved to a 'competency-based' model with an emphasis on workplace-based learning and assessment which, in turn, depends on feedback to be effective. Further, the understanding of feedback has changed from information about a performance directed to the learner performing the task, to a dialogue, which enables the learner to act and develop.In health professional education, feedback is a complex interaction between trainee, supervisor and the healthcare system. Most published research on feedback in health professional education originates in Europe and North America. Our interest is on the impact of Culture on this process, particularly in the context of Asian cultures.The (scientific) realist approach of Pawson and Tilley provides a means to examine complex interventions in social situations, and thus is an appropriate lens to use for this study. This is a protocol for a realist synthesis which asks how, why and in what circumstances do Asian Cultures influence health professional trainees to seek, respond to and use feedback given in the clinical environment, if at all.

METHODS AND ANALYSIS

An initial search was performed to help define the scope of the review question and develop our initial programme theory. The formal electronic search was carried out in February 2020 and included: CINAHL, ERIC, Medline and PsycInfo, and repeated in October 2020. Retrieved articles were imported into Covidence for screening and data extraction, after which components of the Context-Mechanisms-Outcomes configurations will be sought to refine the initial programme theory.

ETHICS AND DISSEMINATION

As this study is a literature review, ethics approval is not required.The findings will be documented in line with the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) publications standards for Realist syntheses, and we plan to disseminate the findings by means of a peer-reviewed journal article and conference presentation(s).

Making it fair: Learners' and assessors' perspectives of the attributes of fair judgement

INTRODUCTION

Optimising the use of subjective human judgement in assessment requires understanding what makes judgement fair. Whilst fairness cannot be simplistically defined, the underpinnings of fair judgement within the literature have been previously combined to create a theoretically-constructed conceptual model. However understanding assessors' and learners' perceptions of what is fair human judgement is also necessary. The aim of this study is to explore assessors' and learners' perceptions of fair human judgement, and to compare these to the conceptual model.

METHODS

A thematic analysis approach was used. A purposive sample of twelve assessors and eight post-graduate trainees undertook semi-structured interviews using vignettes. Themes were identified using the process of constant comparison. Collection, analysis and coding of the data occurred simultaneously in an iterative manner until saturation was reached.

RESULTS

This study supported the literature-derived conceptual model suggesting fairness is a multi-dimensional construct with components at individual, system and environmental levels. At an individual level, contextual, longitudinally-collected evidence, which is supported by narrative, and falls within ill-defined boundaries is essential for fair judgement. Assessor agility and expertise are needed to interpret and interrogate evidence, identify boundaries and provide narrative feedback to allow for improvement. At a system level, factors such as multiple opportunities to demonstrate competence and improvement, multiple assessors to allow for different perspectives to be triangulated, and documentation are needed for fair judgement. These system features can be optimized through procedural fairness. Finally, appropriate learning and working environments which considers patient needs and learners personal circumstances are needed for fair judgments.

DISCUSSION

This study builds on the theory-derived conceptual model demonstrating the components of fair judgement can be explicitly articulated whilst embracing the complexity and contextual nature of health-professions assessment. Thus it provides a narrative to support dialogue between learner, assessor and institutions about ensuring fair judgements in assessment.

Novel tool for assessing the quality of feedback in the emergency room (FEED-ER)

BACKGROUND

The Accreditation Council for Graduate Medical Education (ACGME) emphasizes constructive feedback as a critical component of residency training. Despite over a decade of using competency-based milestone evaluations, emergency medicine (EM) residency programs lack a standardized method for assessing the quality of feedback. We developed two novel EM-specific feedback surveys to assess the quality of feedback in the ER (FEED-ER) from both the resident and the faculty perspectives. This study aimed to evaluate the surveys' psychometric properties.

METHODS

We developed FEED-ER using a Likert scale with faculty and resident versions based on the ACGME framework and a literature review. The preliminary survey consisted of 25 questions involving the feedback domains of timeliness, respect/communication, specificity, action plan, and feedback culture. We conducted two modified Delphi rounds involving 17 content experts to ensure respondent understanding of the items, item coherence to corresponding feedback domains, thematic saturation of domain content, and time duration. A multicenter study was conducted at five university-based EDs in the United States and one in Thailand in 2019. We evaluated the descriptive statistics of the frequency of responses, validity evidence, and reliability of FEED-ER.

RESULTS

A total of 147 EM faculty and 126 EM residents completed the survey. Internal consistency was adequate (Cronbach's alpha > 0.70) and test-retest reliability showed adequate temporal stability (ICC > 0.80) for all dimensions. Content validity was deemed acceptable (CVC > 0.80) for all items. From the 25 items of FEED-ER, 23 loaded into the originally theorized dimensions (with factor loadings > 0.50). Additionally, the five feedback domains were found to be statistically distinct, with correlations between 0.40 and 0.60. The final survey has 23 items.

CONCLUSIONS

This is the first study to develop and provide validity evidence for an EM-specific feedback tool that has strong psychometric properties, is reproducible and reliable, and provides an objective measure for assessing the quality of feedback in the ED.

Development of the Feedback Quality Instrument: a guide for health professional educators in fostering learner-centred discussions

BACKGROUND

Face-to-face feedback plays an important role in health professionals' workplace learning. The literature describes guiding principles regarding effective feedback but it is not clear how to enact these. We aimed to create a Feedback Quality Instrument (FQI), underpinned by a social constructivist perspective, to assist educators in collaborating with learners to support learner-centred feedback interactions. In earlier research, we developed a set of observable educator behaviours designed to promote beneficial learner outcomes, supported by published research and expert consensus. This research focused on analysing and refining this provisional instrument, to create the FQI ready-to-use.

METHODS

We collected videos of authentic face-to-face feedback discussions, involving educators (senior clinicians) and learners (clinicians or students), during routine clinical practice across a major metropolitan hospital network. Quantitative and qualitative analyses of the video data were used to refine the provisional instrument. Raters administered the provisional instrument to systematically analyse educators' feedback practice seen in the videos. This enabled usability testing and resulted in ratings data for psychometric analysis involving multifaceted Rasch model analysis and exploratory factor analysis. Parallel qualitative research of the video transcripts focused on two under-researched areas, psychological safety and evaluative judgement, to provide practical insights for item refinement. The provisional instrument was revised, using an iterative process, incorporating findings from usability testing, psychometric testing and parallel qualitative research and foundational research.

RESULTS

Thirty-six videos involved diverse health professionals across medicine, nursing and physiotherapy. Administering the provisional instrument generated 174 data sets. Following refinements, the FQI contained 25 items, clustered into five domains characterising core concepts underpinning quality feedback: set the scene, analyse performance, plan improvements, foster learner agency, and foster psychological safety.

CONCLUSIONS

The FQI describes practical, empirically-informed ways for educators to foster quality, learner-centred feedback discussions. The explicit descriptions offer guidance for educators and provide a foundation for the systematic analysis of the influence of specific educator behaviours on learner outcomes.

Medical student self-assessment as emergency medicine residency applicants

BACKGROUND

Emergency medicine (EM) applicants are encouraged to consider their own "competitiveness" when deciding on the number of applications to submit. Program directors rank the Standardized Letter of Evaluation (SLOE) as the most important factor when reviewing an applicant. Accurate insight into how clinical performance is reflected on the SLOE could improve medical students' ability to gauge their own competitiveness.

OBJECTIVE

This study aims to determine the accuracy of students' self-assessment by SLOE evaluation measures when compared to the SLOE completed by faculty after their EM clerkship.

METHODS

Participants of this multicenter study included fourth-year medical students who had completed their EM clerkship and were applying to EM residency. Students completed a modified SLOE to reflect rankings they believed they would receive on their official SLOE. Additionally, students completed a survey assessing their knowledge of the SLOE, their perception of feedback during the clerkship, and their self-perceived competitiveness as an EM applicant. Correlation between the rankings on the student-completed SLOE and the official SLOE was analyzed using the Kendall correlation.

RESULTS

Of the 49 eligible students, 42 (85.7%) completed the study. The correlation between scores on the student-completed and official SLOE were significantly low (r < 0.68) for each item. The majority of students agreed that they were satisfied by the quantity and quality of feedback they received (31/42, 73.8%). Few students agreed that they knew how many applications to submit to ensure a match in EM (7/42, 16.7%).

CONCLUSION

This study demonstrates that students did not accurately predict their rankings on the official SLOE at the end of an EM rotation and had little insight into their competitiveness as an applicant. These findings highlight opportunities to mitigate the burden on students and programs caused by the increasing number of applications per applicant. Further research is needed as to whether strategies to increase insight into competitiveness are effective.

Numbers Encapsulate, Words Elaborate: Toward the Best Use of Comments for Assessment and Feedback on Entrustment Ratings

The adoption of entrustment ratings in medical education is based on a seemingly simple premise: to align workplace-based supervision with resident assessment. Yet it has been difficult to operationalize this concept. Entrustment rating forms combine numeric scales with comments and are embedded in a programmatic assessment framework, which encourages the collection of a large quantity of data. The implicit assumption that more is better has led to an untamable volume of data that competency committees must grapple with. In this article, the authors explore the roles of numbers and words on entrustment rating forms, focusing on the intended and optimal use(s) of each, with a focus on the words. They also unpack the problematic issue of dual-purposing words for both assessment and feedback. Words have enormous potential to elaborate, to contextualize, and to instruct; to realize this potential, educators must be crystal clear about their use. The authors set forth a number of possible ways to reconcile these tensions by more explicitly aligning words to purpose. For example, educators could focus written comments solely on assessment; create assessment encounters distinct from feedback encounters; or use different words collected from the same encounter to serve distinct feedback and assessment purposes. Finally, the authors address the tyranny of documentation created by programmatic assessment and urge caution in yielding to the temptation to reduce words to numbers to make them manageable. Instead, they encourage educators to preserve some educational encounters purely for feedback, and to consider that not all words need to become data.

Hierarchy, "Kreng Jai" and Feedback: A Grounded Theory Study Exploring Perspectives of Clinical Faculty and Medical Students in Thailand

Phenomenon: Institutional learning culture influences how feedback is provided, accepted, and acted upon. The Thai societal culture, characterized by strict hierarchy and collectivism, may have a significant impact on the learning culture and, therefore, feedback conversations between teachers and learners. This study explored: common approaches used by faculty to provide feedback to students; and faculty and student perspectives regarding sociocultural factors that impact feedback seeking, provision, and acceptance. Approach: Using a constructivist paradigm, we explored perspectives of clinical faculty and medical students at an academic medical center in Thailand using focus groups (students) and a focus group and individual interviews (faculty). Sessions were audiotaped, transcribed, and de-identified prior to analysis. Constant comparative analysis was performed on transcripts, focusing on perceived cultural factors that impacted feedback conversations. Findings: Thirty faculty participated in the study, four participated in a focus group, and 27 participated in individual interviews. Twenty-two medical students participated in four focus groups. We identified the following key themes, which could be grouped under three categories: (1) Faculty approaches to providing feedback (1.1) Feedback should be initiated by faculty. (1.2) Feedback is initiated primarily for deficit identification and correction. (2) Factors impacting students' feedback seeking and acceptance. (2.1) Students are willing to accept harsh feedback when it provides suggestions for improvement. (2.2) Feedback is most credible when faculty have direct knowledge of the student's effort. (2.3) Feedback seeking is considered a burden on teachers. (3) Cultural factors that influence feedback (3.1) Societal hierarchy perpetuates unidirectional top-down feedback. (3.2) Kreng jai (the balance between consideration for others and self-interests) affects feedback seeking and provision. Insights: Though the value of feedback on learning was emphasized by all participants, the hierarchical culture of Thai society was perceived to have a significant influence on feedback seeking, provision, and acceptance. Identifying and addressing societal as well as institutional cultural factors would be key in designing growth-enhancing feedback initiatives relevant to the local context. One size feedback training does not fit all.

Fairness in human judgement in assessment: a hermeneutic literature review and conceptual framework

Human judgement is widely used in workplace-based assessment despite criticism that it does not meet standards of objectivity. There is an ongoing push within the literature to better embrace subjective human judgement in assessment not as a 'problem' to be corrected psychometrically but as legitimate perceptions of performance. Taking a step back and changing perspectives to focus on the fundamental underlying value of fairness in assessment may help re-set the traditional objective approach and provide a more relevant way to determine the appropriateness of subjective human judgements. Changing focus to look at what is 'fair' human judgement in assessment, rather than what is 'objective' human judgement in assessment allows for the embracing of many different perspectives, and the legitimising of human judgement in assessment. However, this requires addressing the question: what makes human judgements fair in health professions assessment? This is not a straightforward question with a single unambiguously 'correct' answer. In this hermeneutic literature review we aimed to produce a scholarly knowledge synthesis and understanding of the factors, definitions and key questions associated with fairness in human judgement in assessment and a resulting conceptual framework, with a view to informing ongoing further research. The complex construct of fair human judgement could be conceptualised through values (credibility, fitness for purpose, transparency and defensibility) which are upheld at an individual level by characteristics of fair human judgement (narrative, boundaries, expertise, agility and evidence) and at a systems level by procedures (procedural fairness, documentation, multiple opportunities, multiple assessors, validity evidence) which help translate fairness in human judgement from concepts into practical components.

National inter-rater agreement of standardised simulated-patient-based assessments

PURPOSE

The forthcoming UK Medical Licensing Assessment will require all medical schools in the UK to ensure that their students pass an appropriately designed Clinical and Professional Skills Assessment (CPSA) prior to graduation and registration with a licence to practice medicine. The requirements for the CPSA will be set by the General Medical Council, but individual medical schools will be responsible for implementing their own assessments. It is therefore important that assessors from different medical schools across the UK agree on what standard of performance constitutes a fail, pass or good grade.

METHODS

We used an experimental video-based, single-blinded, randomised, internet-based design. We created videos of simulated student performances of a clinical examination at four scripted standards: clear fail (CF), borderline (BD), clear pass (CPX) and good (GD). Assessors from ten regions across the UK were randomly assigned to watch five videos in 12 different combinations and asked to give competence domain scores and an overall global grade for each simulated candidate. The inter-rater agreement as measured by the intraclass correlation coefficient (ICC) based on a two-way random-effects model for absolute agreement was calculated for the total domain scores.

RESULTS

120 assessors enrolled in the study, with 98 eligible for analysis. The ICC was 0.93 (95% CI 0.81-0.99). The mean percentage agreement with the scripted global grade was 74.4% (range 40.8-96.9%).

CONCLUSIONS

The inter-rater agreement amongst assessors across the UK when rating simulated candidates performing at scripted levels is excellent. The level of agreement for the overall global performance level for simulated candidates is also high. These findings suggest that assessors from across the UK viewing the same simulated performances show high levels of agreement of the standards expected of students at a 'clear fail,' 'borderline,' 'clear pass' and 'good' level.

Driving lesson or driving test? : A metaphor to help faculty separate feedback from assessment

Although there is consensus in the medical education world that feedback is an important and effective tool to support experiential workplace-based learning, learners tend to avoid the feedback associated with direct observation because they perceive it as a high-stakes evaluation with significant consequences for their future. The perceived dominance of the summative assessment paradigm throughout medical education reduces learners' willingness to seek feedback, and encourages supervisors to mix up feedback with provision of 'objective' grades or pass/fail marks. This eye-opener article argues that the provision and reception of effective feedback by clinical supervisors and their learners is dependent on both parties' awareness of the important distinction between feedback used in coaching towards growth and development (assessment for learning) and reaching a high-stakes judgement on the learner's competence and fitness for practice (assessment of learning). Using driving lessons and the driving test as a metaphor for feedback and assessment helps supervisors and learners to understand this crucial difference and to act upon it. It is the supervisor's responsibility to ensure that supervisor and learner achieve a clear mutual understanding of the purpose of each interaction (i.e. feedback or assessment). To allow supervisors to use the driving lesson-driving test metaphor for this purpose in their interactions with learners, it should be included in faculty development initiatives, along with a discussion of the key importance of separating feedback from assessment, to promote a feedback culture of growth and support programmatic assessment of competence.

Necessary but not sufficient: identifying conditions for effective feedback during internal medicine residents' clinical education

Competency-based medical education and programmatic assessment intend to increase the opportunities for meaningful feedback, yet these conversations remain elusive. By comparing resident and faculty perceptions of feedback opportunities within one internal medicine residency training program, we sought to understand whether and how principles underlying meaningful feedback could be supported or constrained across a variety of feedback opportunities. Using case-study qualitative methodology, interviews and focus groups were conducted to explore 19 internal medicine residents' and 7 faculty members' perceptions of feedback across a variety of feedback opportunities: coaching, mini-CEXs, in-training evaluation reports and routine clinical supervision. Our data analysis moved iteratively between developing conceptual understandings and fine-grained analyses, while attending to both deductive and inductive analysis. Our results suggest that all feedback opportunities, including those created through formalized assessments, can foster meaningful feedback if faculty establish a trusting relationship with the resident, base their feedback on direct observation and support resident learning. However, formalized assessments were often perceived as inhibiting the conditions for meaningful feedback. A coaching program provided a context in which meaningful feedback could arise, in part because faculty were supported in shifting their focus from patient to resident. Meaningful feedback in clinical education may be fostered across a variety of feedback opportunities, however, it is often constrained by assessment. We must consider whether increasing the frequency of formative assessments may inhibit efforts to improve our feedback cultures while, in contrast, freeing up faculty to focus on supporting resident learning could improve these cultures.

Relationships as the Backbone of Feedback: Exploring Preceptor and Resident Perceptions of Their Behaviors During Feedback Conversations

PURPOSE

Newer definitions of feedback emphasize learner engagement throughout the conversation, yet teacher and learner perceptions of each other's behaviors during feedback exchanges have been less well studied. This study explored perceptions of residents and faculty regarding effective behaviors and strategies during feedback conversations and factors that affected provision and acceptance of constructive feedback.

METHOD

Six outpatient internal medicine preceptors and 12 residents at Brigham and Women's Hospital participated (2 dyads per preceptor) between September 2017 and May 2018. Their scheduled feedback conversations were observed by the lead investigator, and one-on-one interviews were conducted with each member of the dyad to explore their perceptions of the conversation. Interviews were transcribed and analyzed for key themes. Because participants repeatedly emphasized teacher-learner relationships as key to meaningful feedback, a framework method of analysis was performed using the 3-step relationship-centered communication model REDE (relationship establishment, development, and engagement).

RESULTS

After participant narratives were mapped onto the REDE model, key themes were identified and categorized under the major steps of the model. First, establishment: revisit and renew established relationships, preparation allows deeper reflection on goals, set a collaborative agenda. Second, development: provide a safe space to invite self-reflection, make it about a skill or action. Third, engagement: enhance self-efficacy at the close, establish action plans for growth.

CONCLUSIONS

Feedback conversations between longitudinal teacher-learner dyads could be mapped onto a relationship-centered communication framework. Our study suggests that behaviors that enable trusting and supportive teacher-learner relationships can form the foundation of meaningful feedback.

Psychological safety in feedback: What does it look like and how can educators work with learners to foster it?

CONTEXT

Feedback conversations play a central role in health professions workplace learning. However, learners face a dilemma: if they engage in productive learning behaviours (such as asking questions, raising difficulties, offering opinions or contesting ideas), they risk exposing their limitations or offending the educator. This highlights the importance of psychological safety in encouraging learners to candidly engage in interactive dialogue and the co-construction of knowledge. Previous research has recommended that building safety, trust or an educational alliance is key to productive feedback encounters. Yet it is unclear how to translate this into practice. Hence our research question was: What does psychological safety look like in workplace feedback and how can educators work with learners to foster it?

METHODS

We analysed 36 videos of routine formal feedback episodes in clinical practice involving diverse health professionals. A psychologically safe learning environment was inferred when learners progressively disclosed information and engaged in productive learning behaviours during the conversation. We used thematic analysis to identify associated educator strategies, which seemed to promote psychological safety.

RESULTS

Four themes were identified: (a) setting the scene for dialogue and candour; (b) educator as ally; (c) a continuing improvement orientation, and (d) encouraging interactive dialogue. Educators approaches captured within these themes, seemed to foster a psychologically safe environment by conveying a focus on learning, and demonstrating respect and support to learners.

CONCLUSIONS

This study builds on claims regarding the importance of psychological safety in feedback by clarifying what psychological safety in workplace feedback conversations might look like and identifying associated educator approaches. The results may offer educators practical ways they could work with learners to encourage candid dialogue focused on improving performance.

Meaningful feedback through a sociocultural lens

This AMEE guide provides a framework and practical strategies for teachers, learners and institutions to promote meaningful feedback conversations that emphasise performance improvement and professional growth. Recommended strategies are based on recent feedback research and literature, which emphasise the sociocultural nature of these complex interactions. We use key concepts from three theories as the underpinnings of the recommended strategies: sociocultural, politeness and self-determination theories. We view the content and impact of feedback conversations through the perspective of learners, teachers and institutions, always focussing on learner growth. The guide emphasises the role of teachers in forming educational alliances with their learners, setting a safe learning climate, fostering self-awareness about their performance, engaging with learners in informed self-assessment and reflection, and co-creating the learning environment and learning opportunities with their learners. We highlight the role of institutions in enhancing the feedback culture by encouraging a growth mind-set and a learning goal-orientation. Practical advice is provided on techniques and strategies that can be used and applied by learners, teachers and institutions to effectively foster all these elements. Finally, we highlight throughout the critical importance of congruence between the three levels of culture: unwritten values, espoused values and day to day behaviours.

Students' Perspectives on Their Experience in Clinical Placements: Using a Modified Delphi Methodology to Engage Physiotherapy Stakeholders in Revising the National Form

Purpose: We developed an evidence-informed Student Evaluation of the Clinical Placement form. This form gives students the opportunity to share their feedback and perceptions of their clinical placement experiences and provides meaningful data to all stakeholders. Method: We used a modified Delphi process to engage a sample of national stakeholders: physiotherapy clinical education leads of academic departments, centre coordinators of clinical education, clinical instructors, and students. An expert consultant panel, in addition to the investigators, reviewed the responses from each round and helped develop the questionnaire for the subsequent round and finalize the evaluation form. Results: The response rate was 65.3% (47 of 72) for Round 1, 76.6% (36 of 47) for Round 2, and 100% (36 of 36) for Round 3. After three rounds of questionnaires, 89% of participants thought that the evaluation form met their needs. Conclusions: We developed a revised Student Evaluation of the Clinical Placement form that is informed by the literature and meaningful to all stakeholders. This form is being implemented in physiotherapy university programmes across Canada to enable students to share their experiences at clinical sites.

Resuscitation Education Science: Educational Strategies to Improve Outcomes From Cardiac Arrest: A Scientific Statement From the American Heart Association

The formula for survival in resuscitation describes educational efficiency and local implementation as key determinants in survival after cardiac arrest. Current educational offerings in the form of standardized online and face-to-face courses are falling short, with providers demonstrating a decay of skills over time. This translates to suboptimal clinical care and poor survival outcomes from cardiac arrest. In many institutions, guidelines taught in courses are not thoughtfully implemented in the clinical environment. A current synthesis of the evidence supporting best educational and knowledge translation strategies in resuscitation is lacking. In this American Heart Association scientific statement, we provide a review of the literature describing key elements of educational efficiency and local implementation, including mastery learning and deliberate practice, spaced practice, contextual learning, feedback and debriefing, assessment, innovative educational strategies, faculty development, and knowledge translation and implementation. For each topic, we provide suggestions for improving provider performance that may ultimately optimize patient outcomes from cardiac arrest.

Twelve tips to promote a feedback culture with a growth mind-set: Swinging the feedback pendulum from recipes to relationships

Feedback in medical education has traditionally showcased techniques and skills of giving feedback, and models used in staff development have focused on feedback providers (teachers) not receivers (learners). More recent definitions have questioned this approach, arguing that the impact of feedback lies in learner acceptance and assimilation of feedback with improvement in practice and professional growth. Over the last decade, research findings have emphasized that feedback conversations are complex interpersonal interactions influenced by a multitude of sociocultural factors. However, feedback culture is a concept that is challenging to define, thus strategies to enhance culture are difficult to pin down. In this twelve tips paper, we have attempted to define elements that constitute a feedback culture from four different perspectives and describe distinct strategies that can be used to foster a learning culture with a growth mind-set.

Reflection in medical education: intellectual humility, discovery, and know-how

Reflection has been proclaimed as a means to help physicians deal with medicine's inherent complexity and remedy many of the shortcomings of medical education. Yet, there is little agreement on the nature of reflection nor on how it should be taught and practiced. Emerging neuroscientific concepts suggest that human thought processes are largely nonconscious, in part inaccessible to introspection. Our knowledge of the world is fraught with uncertainty, ignorance and indeterminacy, and influenced by emotion, biases and illusions, including the illusion of not having illusions. Neuroscience also documents that lifelong learning processes may hone nonconscious cognition to high levels of sophistication, allowing rapid and precise perceptions, judgments and actions in complex situations. We argue that knowledge of mechanisms underlying human thought may be useful in designing educational programs to foster desired attributes such as curiosity, critical self-awareness and intuitive acumen in medical professionals. The juxtaposition of neuroscientific insights with ideas from Kant on reflective judgement, van Manen on tact, and Aristotle on phronésis, supports a concept of reflection that manifests as wise practice. We suggest that reflection in medical education should be (a) an imperative for educators seeking to guide learners to manage the complexity and "messiness" of medical practice, and (b) a role-modelling mode of medical practice characterized by self-correcting behaviors that culminate in good and right professional actions. An example illustrates reflective practice in the teaching and learning of physicianship.

Where philosophy meets culture: exploring how coaches conceptualise their roles

CONTEXT

Although conceptually attractive, coaching in medicine remains ill-defined, with little examination of the transferability of coaching principles from other fields. Here we explore how coaching is enacted both within and outside of medicine; we aim to understand both the elements required for coaching to be useful and the factors that may influence its translation to the medical education context.

METHODS

In this constructivist grounded theory study, we interviewed 24 individuals across three groups: physicians who consider themselves coaches in clinical learning settings (n = 8), physicians with experience as sports, arts or business coaches (n = 10), and sports coaches without medical backgrounds (n = 6). Data collection and analysis were conducted iteratively using constant comparison to identify themes and explore their relationships.

RESULTS

We identified a shared philosophy of coaching, comprising three core elements that our participants endorsed regardless of the coaching context: (i) mutual engagement, with a shared orientation towards growth and development; (ii) ongoing reflection involving both learners and coaches, and (iii) an embrace of failure as a catalyst for learning. Enacting these features appeared to be influenced by culture, which affected how coaching was defined and developed, how the coaching role was positioned within the learning context, and how comfortably vulnerability could be expressed. Participants struggled to clearly define the coaching role in medicine, instead acknowledging that the lines between educational roles were often blurred. Further, the embrace of failure appeared challenging in medicine, where showing vulnerability was perceived as difficult for both learners and teachers.

CONCLUSIONS

Medical education's embrace of coaching should be informed by an understanding of both coach and learner behaviours that need to be encouraged and trained, and the cultural and organisational supports that are required to foster success.

Identifying coaching skills to improve feedback use in postgraduate medical education

OBJECTIVES

Coaching in medical education has recently gained prominence, but minimal attention has been given to key skills and determining how they work to effectively ensure residents are progressing and developing self-assessment skills. This study examined process-oriented and content-oriented coaching skills used in coaching sessions, with particular attention to how supervisors use them to enhance resident acceptance of feedback to enhance learning.

METHODS

This qualitative study analysed secondary audiotaped data from 15 supervisors: resident dyads during two feedback sessions, 4 months apart. The R2C2 model was used to engage the resident, build a relationship, explore reactions to feedback, explore resident perceptions of content, and coach for change. Framework analysis was used, including familiarisation with the data, identifying the thematic framework, indexing and charting the data and mapping and interpretation.

RESULTS

Process skills included preparation, relationship development, using micro communication skills and techniques to promote reflection and self-assessment by the resident and supervisor flexibility. Content skills related to the specific feedback content included engaging the resident in discussion, ensuring the discussion was collaborative and focused on goal setting, co-developing a Learning Change Plan, ensuring resident commitment and following up on the plan. Together, these skills foster agency in the resident learner. Three overarching themes emerged from the analysis: the interconnectedness of process and content; tensions between encouraging self-direction and ensuring progress and competence; and balancing a coaching dialogue and a teaching monologue.

CONCLUSIONS

Effective coaching by supervisors requires a combination of specific process and content skills that are chosen depending on the needs of the individual resident. Mastering these skills helps residents engage and develop agency in their own professional development. These outcomes depend on faculty maintaining a balance between coaching and teaching, encouraging resident self-direction and ensuring progression to competence.

Feedback Redefined: Principles and Practice

Feedback is defined as a regulatory mechanism where the effect of an action is fed back to modify and improve future action. In medical education, newer conceptualizations of feedback place the learner at the center of the feedback loop and emphasize learner engagement in the entire process. But, learners reject feedback if they doubt its credibility or it conflicts with their self-assessment. Therefore, attention has turned to sociocultural factors that influence feedback-seeking, acceptance, and incorporation into performance. Understanding and application of specific aspects of psychosocial theories could help in designing initiatives that enhance the effect of feedback on learning and growth. In the end, the quality and impact of feedback should be measured by its influence on recipient behavior change, professional growth, and quality of patient care and not the skills of the feedback provider. Our objective is to compare and contrast older and newer definitions of feedback, explore existing feedback models, and highlight principles of relevant psychosocial theories applicable to feedback initiatives. Finally, we aim to apply principles from patient safety initiatives to emphasize a safe and just culture within which feedback conversations occur so that weaknesses are as readily acknowledged and addressed as strengths.

Developing a video-based method to compare and adjust examiner effects in fully nested OSCEs

BACKGROUND

Although averaging across multiple examiners' judgements reduces unwanted overall score variability in objective structured clinical examinations (OSCE), designs involving several parallel circuits of the OSCE require that different examiner cohorts collectively judge performances to the same standard in order to avoid bias. Prior research suggests the potential for important examiner-cohort effects in distributed or national examinations that could compromise fairness or patient safety, but despite their importance, these effects are rarely investigated because fully nested assessment designs make them very difficult to study. We describe initial use of a new method to measure and adjust for examiner-cohort effects on students' scores.

METHODS

We developed video-based examiner score comparison and adjustment (VESCA): volunteer students were filmed 'live' on 10 out of 12 OSCE stations. Following the examination, examiners additionally scored station-specific common-comparator videos, producing partial crossing between examiner cohorts. Many-facet Rasch modelling and linear mixed modelling were used to estimate and adjust for examiner-cohort effects on students' scores.

RESULTS

After accounting for students' ability, examiner cohorts differed substantially in their stringency or leniency (maximal global score difference of 0.47 out of 7.0 [Cohen's d = 0.96]; maximal total percentage score difference of 5.7% [Cohen's d = 1.06] for the same student ability by different examiner cohorts). Corresponding adjustment of students' global and total percentage scores altered the theoretical classification of 6.0% of students for both measures (either pass to fail or fail to pass), whereas 8.6-9.5% students' scores were altered by at least 0.5 standard deviations of student ability.

CONCLUSIONS

Despite typical reliability, the examiner cohort that students encountered had a potentially important influence on their score, emphasising the need for adequate sampling and examiner training. Development and validation of VESCA may offer a means to measure and adjust for potential systematic differences in scoring patterns that could exist between locations in distributed or national OSCE examinations, thereby ensuring equivalence and fairness.

Assessment, feedback and the alchemy of learning

CONTEXT

Models of sound assessment practices increasingly emphasise assessment's formative role. As a result, assessment must not only support sound judgements about learner competence, but also generate meaningful feedback to guide learning. Reconciling the tension between assessment's focus on judgement and decision making and feedback's focus on growth and development represents a critical challenge for researchers and educators.

METHODS

We synthesise the literature related to this tension, framed around four trends in education research: (i) shifting perspectives on assessment; (ii) shifting perspectives on feedback; (iii) increasing attention on learners' perceptions of assessment and feedback, and (iv) increasing attention on the influence of culture on assessment and feedback. We describe factors that produce and sustain this tension.

RESULTS

The lines between assessment and feedback frequently blur in medical education. Models of programmatic assessment deliberately use the same data for both purposes: low-stakes individual data points are used formatively, but then are added together to support summative judgements. However, the translation of theory to practice is not straightforward. Efforts to embed meaningful feedback in programmes of learning face a multitude of threats. Learners may perceive assessment with formative intent as summative, restricting their engagement with it as feedback, and thus diminishing its learning value. A learning culture focused on assessment may limit learners' sense of safety to explore, to experiment, and sometimes to fail.

CONCLUSIONS

Successfully blending assessment and feedback demands clarity of purpose, support for learners, and a system and organisational commitment to a culture of improvement rather than a culture of performance.

Beyond hands-on and hands-off: supervisory approaches and entrustment on the inpatient ward

CONTEXT

The concept of entrustment has garnered significant attention in medical specialties, despite variability in supervision styles and entrustment decisions. There is a need to further study the enactment of supervision on inpatient wards to inform competency-based assessment design.

METHODS

Attending physicians, while supervising on clinical teaching inpatient wards, were invited to describe a recent moment of enacting supervision with an internal medicine resident. Constructivist grounded theory guided data collection and analysis. Interview transcripts were analysed in iterative cycles to inform data collection. Constant comparison was used to build a theory of supervision from the identified themes.

RESULTS

In 2016-2017, 23 supervisors from two Canadian universities with supervision reputations ranging from very involved to less involved participated in one or two interviews (total: 28). Supervisors were not easily dichotomised into styles based on behaviour because all used similar oversight strategies. Supervisors described adjusting between 'hands-on' (e.g. detail oriented) and 'hands-off' (e.g. less visible on ward) styles depending on the context. All also contended with the competing roles of clinical teacher and care provider. Supervisors made a distinction between the terms `entrust' and `trust', and did not grant complete entrustment to senior residents.

CONCLUSIONS

We propose that a supervisor's perceived responsibility for the ward underlies adjustments between 'hands-on' (i.e. personal ward responsibility) and 'hands-off' (i.e. shared ward responsibility) styles. Our approaches to clinical supervision model combines this responsibility tension with the tension between patient care and teaching to illustrate four supervisory approaches, each with unique priorities influencing entrustment. Given the fluidity in supervision, documenting changes in oversight strategies, rather than absolute levels of entrustment, may be more informative for assessment purposes. Research is needed to determine if there is sufficient association between the supervision provided, the entrustment decision made and the supervisor's trust in a trainee to use these as proxies in assessing a trainee's competence.

Barriers and Facilitators to Self-Directed Learning in Continuing Professional Development for Physicians in Canada: A Scoping Review

PURPOSE

This scoping review explored the barriers and facilitators that influence engagement in and implementation of self-directed learning (SDL) in continuing professional development (CPD) for physicians in Canada.

METHOD

This review followed the six-stage scoping review framework of Arksey and O'Malley and of Daudt et al. In 2015, the authors searched eight online databases for English-language Canadian articles published January 2005-December 2015. To chart and analyze data from the 17 included studies, they employed a two-step analysis process composed of conventional content analysis followed by directed coding applying the Theoretical Domains Framework (TDF).

RESULTS

Conventional content analysis generated five categories of barriers and facilitators: individual, program, technological, environmental, and workplace/organizational. Directed coding guided by the TDF allowed analysis of barriers and facilitators to behavior change according to two key groups: physicians engaging in SDL, and SDL developers designing and implementing SDL programs. Of the 318 total barriers and facilitators coded, 290 (91.2%) were coded for physicians and 28 (8.8%) for SDL developers. The majority (209; 65.7%) were coded in four key TDF domains: environmental context and resources, social influences, beliefs about consequences, and behavioral regulation.

CONCLUSIONS

This scoping review identified five categories of barriers and facilitators in the literature and four key TDF domains where most factors related to behavior change of physicians and SDL developers regarding SDL programs in CPD were coded. There was a significant gap in the literature about factors that may contribute to SDL developers' capacity to design and implement SDL programs in CPD.

The Art (and Artifice) of Seeking Feedback: Clerkship Students' Approaches to Asking for Feedback

PURPOSE

As attention has shifted to learners as significant partners in feedback interactions, it is important to explore what feedback-seeking behaviors medical students use and how the faculty-student relationship affects feedback-seeking behaviors.

METHOD

This qualitative study was inspired by the organizational psychology literature. Third-year medical students were interviewed at Maine Medical Center in April-May 2017 after completing a traditional block rotation clerkship or a nine-month longitudinal integrated clerkship (LIC). A constructivist grounded theory approach was used to analyze transcripts and develop themes.

RESULTS

Fourteen students participated (eight LIC, six block rotation). Themes associated with why students sought feedback included goal orientations, perceived benefits and costs, and student and feedback provider characteristics. Factors influencing the way students sought feedback included busy environments, timing, and cues students were attuned to. Students described more inquiry than monitoring approaches and used various indirect and noninquiry techniques (artifice) in asking for feedback. Students did not find summative feedback as helpful as seeking feedback themselves, and they suggested training in seeking feedback would be beneficial. Faculty-student relationship dynamics included several aspects affecting feedback-seeking behaviors, and relationship differences in the LIC and block models affected feedback-seeking behaviors.

CONCLUSIONS

Medical students have many motives to seek feedback and adapt their feedback-seeking behaviors to actively participate in an intricate dialogic interaction with feedback providers. Students gradually refine the art (and artifice) of obtaining the specific feedback information that meets their needs. The authors offer a prototype curriculum that may facilitate students' development of feedback-seeking skills.

Does it matter who writes down the feedback? A comparison of teacher- vs. student-completed clinical encounter cards during clinical rotations in veterinary studies

Objective: Despite the fact that feedback (FB) provided by teachers to students is a recognised, effective teaching tool, successful use of feedback during clinical training depends on many factors. In addition to appropriate training and attitude of teachers, sustainable feedback requires an appropriate teaching culture and active commitment on the part of the students to receive, accept and use FB. This study examines the use of two different clinical encounter cards (CECs) during clinical rotation and investigates whether students take a more active part in the feedback process when using these cards. The objective of this study is to test whether it has a positive effect if students write down FB themselves and to verify this positive effect. Methodology: 161 students in their 9th semester of veterinary studies each had to use two clinical encounter cards (types 1 and 2) during their rotations on 10 wards. For this, students had to ask teachers for FB before starting a clinical activity. The oral FB given by the teachers was either written down on the CEC by the teachers (CEC type 1) or by the students (CEC type 2). Furthermore, the students were asked to assess their own performance by means of anchor criteria and to evaluate the quality of the FB provided by the teachers. Based on the entries in the CECs submitted, the following indicators for both CEC types could be calculated: (1) FB quantity and quality (length and specificity), (2) differentiation of self-assessment, as well as (3) level of satisfaction with the FB provided by the teachers. Results: With 2,377 CECs submitted, the mean CEC return rate was 74%. 99% of the cards showed positive FB, 69% contained constructive FB with suggestions for improvement, and 87% suggested specific next steps. On average, the FB written down by teachers was longer (12.4 versus 9.7 words) and more specific (1.9 versus 1.7 out of 3) than FB written down by students. Length and specificity decreased in the course of the semester. Neither the differentiation of self-assessment (proportion of differentiated entering of self-assessment) nor the students' level of satisfaction with the FB differed between the two examined CEC variants. Conclusion: The use of CECs across the cohort was successfully possible; however, the fact that students formulated and wrote down the FB themselves did not result in more comprehensive or more specific FB. Self-assessment and level of satisfaction with the teachers' FB remained unchanged.

Positive Change in Feedback Perceptions and Behavior: A 10-Year Follow-up Study

BACKGROUND

Providing and learning from feedback are essential components of medical education, and typically described as resistant to change. But given a decade of change in the clinical context in which feedback occurs, the authors asked if, and how, perceptions of feedback and feedback behaviors might have changed in response to contextual affordances.

METHODS

In 2017, the authors conducted a follow-up, ethnographic study on 2 general pediatric floors at the same children's hospital where another ethnographic study on a general pediatric floor was conducted in 2007. Data sources included (1) 21 and 34 hours of observation in 2007 and 2017, respectively, (2) 35 and 25 interviews with general pediatric attending physicians and residents in 2007 and 2017, respectively, and (3) a review of 120 program documents spanning 2007 to 2017. Data were coded and organized around 3 recommendations for feedback that were derived from 2007 data and served as standards for assessing change in 2017.

RESULTS

Data revealed progress in achieving each recommendation. Compared with 2007, participants in 2017 more clearly distinguished between feedback and evaluation; residents were more aware of in-the-moment feedback, and they had shifted their orientation from evaluation and grades to feedback and learning. Explanations for progress in achieving recommendations, which were derived from the data, pointed to institutional and national influences, namely, the pediatric milestones.

CONCLUSIONS

On the basis of follow-up, ethnographic data, changes in the clinical context of pediatric education may afford positive change in perceptions of feedback and feedback behavior and point to influences within and beyond the institution.

Challenges of feedback provision in the workplace: A qualitative study of emergency medicine residents and teachers

BACKGROUND

Feedback is an effective pedagogical tool in clinical teaching and learning, but is often perceived as unsatisfactory. Little is known about the effect of a busy clinical environment on feedback-giving and -seeking behaviors. This study aims to determine the perceptions and challenges of feedback provision in a busy clinical setting, exemplified by an emergency department (ED).

METHODS

A qualitative semi-structured interview study design was employed. Thirty-six participants (18 attending physicians, 18 residents) were purposively sampled from three EDs in northern Taiwan between August 2015 and July 2016. Interviews were recorded, transcribed, and analyzed thematically.

RESULTS

Three major themes were identified with illustrative quotes: (1) the balance between patient safety and providing feedback, (2) variability in feedback, and (3) influential factors, barriers and enablers.

CONCLUSIONS

In real practice, clinical duties competed with the impulse to provide feedback. The variety and complexity of feedback extended beyond style and content. Clinical and contextual factors - some of which may be presented as barriers - influenced how, when and whether a teacher or learner decided to give or seek feedback.

Uncovering the unknown: A grounded theory study exploring the impact of self-awareness on the culture of feedback in residency education

AIM

Self-assessment and reflection are essential for meaningful feedback. We aimed to explore whether the well-known Johari window model of self-awareness could guide feedback conversations between faculty and residents and enhance the institutional feedback culture.

METHODS

We had previously explored perceptions of residents and faculty regarding sociocultural factors impacting feedback. We re-analyzed data targeting themes related to self-assessment, reflection, feedback seeking and acceptance, aiming to generate individual and institutional feedback strategies applicable to each quadrant of the window.

RESULTS

We identified the following themes for each quadrant: (1) Behaviors known to self and others - Validating the known; (2) Behaviors unknown to self but known to others - Accepting the blind; (3) Behaviors known to self and unknown to others - Disclosure of hidden; and (4) Behaviors unknown to self and others - Uncovering the unknown. Normalizing self-disclosure of limitations, encouraging feedback seeking, training in nonjudgmental feedback and providing opportunities for longitudinal relationships could promote self-awareness, ultimately expanding the "open" quadrant of the Johari window.

CONCLUSIONS

The Johari window, a model of self-awareness in interpersonal communications, could provide a robust framework for individuals to improve their feedback conversations and institutions to design feedback initiatives that enhance its quality and impact.

Frontline learning of medical teaching: "you pick up as you go through work and practice"

BACKGROUND

Few medical teachers have received formal teaching education. Along with individual and organizational barriers to participation in teacher training programs, increasing numbers and altered distribution of physicians away from major teaching centers have increased the difficulty of attendance. Furthermore, it is not known if traditional faculty development formats are the optimal learning options given findings from existing studies document both positive and negative outcomes. There is a gap in research that explores how medical teachers learn to teach and also limited research regarding how medical teachers actually teach. The purpose of this study was to provide insight into how physicians describe their teaching of trainees, and the nature of their teaching development and improvement to inform faculty development programs.

METHODS

Semi-structured interviews were conducted with 36 physicians, with a broad range of teaching experience, purposefully selected from five disciplines: Internal Medicine, Pediatrics, Psychiatry, Surgery, and Family Medicine. A qualitative, inductive approach was used to analyse the data.

RESULTS

Teaching was described as being centered on the needs of individual trainees, but was dependent on patient presentation and environmental context. For this group of physicians learning to teach was perceived as a dynamic and evolving process influenced by multiple life experiences. The physicians had not learnt to teach through formal education and then put that learning into practice, but had learnt to teach and improve their teaching through their trial and errors teaching. Life experiences unconnected with the medical environment contributed to their knowledge of teaching along with limited formal learning to teach experiences. Teaching practice was influenced by peers and trainees, feedback, and observation. The findings suggest these medical teachers learn to teach along a continuum largely through their teaching practice.

CONCLUSION

The findings suggested that the participants' major resource for learning how to teach was informal experiential learning, both in and out of the workplace. This may have implications for faculty development strategies for medical teaching education.

Student-Initiated Feedback Using Clinical Encounter Cards during Clinical Rotations in Veterinary Medicine: A Feasibility Study

Despite its critical role for successful student learning, providing adequate teacher feedback is still a major issue during clinical education. In human medical education, the implementation of clinical encounter cards (CECs) has led to more frequent, timely, and structured teacher feedback. The present study aimed to introduce student-initiated CECs in a veterinary medical setting (clinical rotations). A total of 24 students were randomly assigned to a control group (standard rotations) and an intervention group where they had to ask for teacher feedback using CECs. The feasibility of implementing CECs was evaluated by examining the content of the completed CECs and by using anonymous student and teacher questionnaires. In addition, acceptance of the intervention and changes in feedback behavior were examined from both the teachers' and students' perspectives. Overall, it was shown that using CECs is not only feasible in a veterinary clinical setting but also conducive to more frequent and constructive teacher feedback. However, some teachers postponed completing the CECs due to time pressure, leading to less direct and timely feedback. Moreover, students felt more comfortable asking for feedback from less experienced, younger teachers, and teachers' quantitative ratings and open commentaries seemed to be affected by leniency bias. Finally, a focus group including teachers and students discussed these results in light of their practical experiences. This led to a streamlining of the implementation process and optimizations to facilitate future large-scale implementation. The study has implications for veterinary educators wishing to improve feedback in their institution.

Enhancing Feedback on Professionalism and Communication Skills in Anesthesia Residency Programs

BACKGROUND

Despite its importance, training faculty to provide feedback to residents remains challenging. We hypothesized that, overall, at 4 institutions, a faculty development program on providing feedback on professionalism and communication skills would lead to (1) an improvement in the quantity, quality, and utility of feedback and (2) an increase in feedback containing negative/constructive feedback and pertaining to professionalism/communication. As secondary analyses, we explored these outcomes at the individual institutions.

METHODS

In this prospective cohort study (October 2013 to July 2014), we implemented a video-based educational program on feedback at 4 institutions. Feedback records from 3 months before to 3 months after the intervention were rated for quality (0-5), utility (0-5), and whether they had negative/constructive feedback and/or were related to professionalism/communication. Feedback records during the preintervention, intervention, and postintervention periods were compared using the Kruskal-Wallis and χ tests. Data are reported as median (interquartile range) or proportion/percentage.

RESULTS

A total of 1926 feedback records were rated. The institutions overall did not have a significant difference in feedback quantity (preintervention: 855/3046 [28.1%]; postintervention: 896/3327 [26.9%]; odds ratio: 1.06; 95% confidence interval, 0.95-1.18; P = .31), feedback quality (preintervention: 2 [1-4]; intervention: 2 [1-4]; postintervention: 2 [1-4]; P = .90), feedback utility (preintervention: 1 [1-3]; intervention: 2 [1-3]; postintervention: 1 [1-2]; P = .61), or percentage of feedback records containing negative/constructive feedback (preintervention: 27%; intervention: 32%; postintervention: 25%; P = .12) or related to professionalism/communication (preintervention: 23%; intervention: 33%; postintervention: 24%; P = .03). Institution 1 had a significant difference in feedback quality (preintervention: 2 [1-3]; intervention: 3 [2-4]; postintervention: 3 [2-4]; P = .001) and utility (preintervention: 1 [1-3]; intervention: 2 [1-3]; postintervention: 2 [1-4]; P = .008). Institution 3 had a significant difference in the percentage of feedback records containing negative/constructive feedback (preintervention: 16%; intervention: 28%; postintervention: 17%; P = .02). Institution 2 had a significant difference in the percentage of feedback records related to professionalism/communication (preintervention: 26%; intervention: 57%; postintervention: 31%; P < .001).

CONCLUSIONS

We detected no overall changes but did detect different changes at each institution despite the identical intervention. The intervention may be more effective with new faculty and/or smaller discussion sessions. Future steps include refining the rating system, exploring ways to sustain changes, and investigating other factors contributing to feedback quality and utility.