Reflection as a component of formative assessment appears to be instrumental in promoting the use of feedback; an observational study

Reflecting on insight and insights into reflection: a systematic review of insight and reflection in post graduate medical education

BACKGROUND

Clinicians recognize insight as important for safe independent medical practice. Clinical education literature focuses on self-reflection. The aim of this review is to describe how clinical educators conceptualize reflection and ask is it analogous to how clinicians conceptualize insight?

METHODS

Using PRISMA guidelines, a systematic review of the literature around insight and reflection in postgraduate medical education was undertaken. A thematic analysis of the concepts of insight and reflection was performed.

RESULTS

A total of 75 reports were included in the analysis. The literature focussed predominantly on reflection with little discussion of insight. Three main themes were generated: episodic reflection; cyclic reflection; reflection as a state. Reflection as a state seemed to be the professional quality most often aspired to but was less well defined in terms of educational interventions. When more than one model was described, it was often with a reflective state being the ideal that episodic or cyclic reflection may approximate. It is not clear that it is possible to progress up the hierarchy.

CONCLUSION

We present a novel description of a hierarchy from discrete episodes of reflection, to cyclic processes that involve reflection, through to a state in which the practitioner is reflective. There is no unified understanding of how an individual ascends this hierarchy, or a cohesive description of what insight is for an independent medical practitioner. This review highlights the need for research into how practicing clinicians conceptualize and characterize insight in their training and practice.

New approach to learning medical procedures using a smartphone and the Moodle platform to facilitate assessments and written feedback

Background: To overcome communication obstacles between medical students and trainers, we designed serial learning activities utilizing a smartphone and web-based instruction (WBI) on the Moodle platform to provide clear and retrievable trainer feedback to students on an objective structured clinical examination (OSCE) item. Methods: We evaluated students’ learning achievement and satisfaction with the new learning tool. A total of 80 fourth-year medical students participated. They installed the Moodle app (the WBI platform) on their smartphones and practiced an endotracheal suction procedure on a medical simulation mannequin while being evaluated by a trainer regarding competence in clinical skills on the smartphone app. Students’ competency was evaluated by comparing the scores between the formative assessment and the summative assessment. The degree of satisfaction and usefulness for the smartphone and WBI system were analyzed. Results: The means (standard deviations, SDs) of the formative and summative assessments were 8.80 (2.53) and 14.24 (1.97) out of a total of 17 points, respectively, reflecting a statistically significant difference (P<0.05). The degree of satisfaction and perceived usefulness of the smartphone app and WBI system were excellent, with means (SDs) of 4.60 (0.58), and 4.60 (0.65), respectively. Conclusion: We believe that the learning process using a smartphone and the Moodle platform offers good guidance for OSCE skill development because trainers’ written feedback is recorded online and is retrievable at all times, enabling students to build and maintain competency through frequent feedback review.

A video- and feedback-based approach to teaching communication skills in undergraduate clinical dental education: The student perspective

OBJECTIVE

Student evaluation of a communication-education programme that combined patient consultation videos with peer- or expert-based feedback.

METHODS

A self-perception questionnaire was given to undergraduate dental students who had undertaken a curricular communication training and feedback programme, in which each participant was videoed three times during a patient interview or consultation, subsequently receiving either peer (PG) or expert feedback (EG). The questionnaire used feedback programme content to evaluate student perspectives, ascertaining whether the students experienced a gain in knowledge and professional conversational skills, whether videos or feedback helped them improve more and general questions about the structure and content of the curriculum. Differences between feedback groups were analysed (Mann-Whitney U test).

RESULTS

45 participants (peer group: n = 23, expert group: n = 22) took part in the study. The peer group experienced watching their video (38%) and receiving feedback (33%) as their major beneficial aspect to improve communication skills (watching fellow student´s video: 17%, providing feedback: 12%). The expert group experienced the major effect in receiving expert feedback (73%, video watching: 27%). The students said that communication skills should be a core competency in dental education learning outcomes (PG: 4.48 ± 0.67, EG: 4.41 ± 0.67; P > 0.05) and that communication curricula should include video-based feedback (PG: 3.91 ± 0.73, EG: 4.00 ± 0.93; P > 0.05).

CONCLUSION

Watching videos and both receiving and providing peer group feedback were experienced as helpful in improving students' communication skills. These findings suggest that a longitudinal communication curriculum, which includes both video-based peer feedback and correlated self-reflection, is a promising learning approach for dental education.

Sharing Reflections on Multisource Feedback in a Peer Group Setting: Stimulating Physicians' Professional Performance and Development

PURPOSE

Reflecting on and using feedback are important for physicians' continuous professional development (CPD). A common format is the discussion of multisource feedback (MSF) in a one-on-one session with a trusted peer or coach. A new approach is to discuss MSF during a peer group session moderated by a professional facilitator. This qualitative study explored how physicians experience participation in these peer group sessions in the context of their CPD.

METHOD

Between March and July 2018, 26 physicians were interviewed about their experiences in a peer group session. These physicians represented 13 monospecialty physician groups from 5 general hospitals in the Netherlands. Interviews were transcribed verbatim and analyzed iteratively, following the interpretative phenomenological approach.

RESULTS

Participation was experienced as a process of disclosing and sharing personal reflections with peers while striking a balance between interpersonal proximity to and distance from peers. Sharing reflections with peers rendered the feedback more meaningful, deepened collegial relationships, and created a sense of urgency for improvement. Improvement goals were mostly related to relational fine-tuning in collaboration; goals related to individual career management remained in the background. Influential factors for the perceived effectiveness of the group sessions were related to the facilitator's expertise, group size, continuity and quality of collegial relationships, personal vulnerabilities, and the context of CPD policy.

CONCLUSIONS

Peer group sessions offered interactivity and established a clear link between individual physicians and their work environments. Sharing reflections on MSF in a peer group setting provided physicians with nuanced insight into their professional performance and fostered a community spirit that supported the implementation of intended changes. Future research should focus on the role of group dynamics and communication strategies and the application of coaching principles, such as drawing up a detailed plan of action and monitoring the follow-up process.

Study protocol: content and perceived educational utility of different modalities of clinical teaching visit (CTV) workplace-based assessments within Australian general practice vocational training: a cross-sectional study

INTRODUCTION

Clinical teaching visits (CTVs) are formative workplace-based assessments that involve a senior general practitioner (GP) observing a clinical practice session of a general practice registrar (specialist vocational GP trainee). These visits constitute a key part of Australian GP training. Despite being mandatory and resource-intensive, there is a paucity of evidence regarding the content and educational utility of CTVs. This study aims to establish the content and educational utility of CTVs across varying practice settings within Australia, as perceived by registrars and their assessors ('CT visitors'). In addition, this study aims to establish registrar, CT visitor and practice factors associated with CTV content and perceived CTV utility ratings.

METHODS AND ANALYSIS

This study will collect data prospectively using online questionnaires completed soon after incident CTVs. Participants will be registrars and CT visitors of CTVs conducted from March 2020 to January 2021. The setting is three Regional Training Organisations across four Australian states and territories (encompassing 37% of Australian GP registrars).Outcome factors will be a number of specified CTV content elements occurring during the CTV as well as participants' perceptions of CTV utility, which will be analysed using univariate and multivariable regression.

ETHICS AND DISSEMINATION

Ethics approval has been granted by the University of Newcastle Human Research Ethics Committee, approval number H-2020-0037. Study findings are planned to be disseminated via conference presentation, peer-reviewed journals, educational practice translational workshops and the GP Synergy research subwebsite.

Using self and peer video annotations of simulated patient encounters in communication training to facilitate the reflection of communication skills: an implementation study

Background: The mandatory communication skills course for fourth-year medical students at the University of Bern Medical School aims to prepare students for challenging communication situations. Students role-play four different scenarios with simulated patients (SPs) and receive feedback from the patient's perspective. The scenarios are video-recorded and uploaded onto the University's virtual learning environment. Students can watch and annotate their own videos and give others access to view them. Project description: Although the course is well liked by students, we identified three areas for improvement: lack of faculty feedback; little active use of the video-recordings;lack of opportunity for students to discuss their experiences with each other. We aimed to address these shortcomings by introducing an additional learning task: students are asked to annotate a section of the video in which they had performed well, and one in which they thought they could have done better, in both their own and a colleague's videos. These video clips and annotations served as the basis of a subsequent two-hour small-group seminar with a physician tutor. The course was evaluated by a mandatory online questionnaire. Results: All 247 students completed the questionnaire. The annotation tool and task were deemed to be comprehensible. Students believed they had learnt more from annotating a peers' video than from their own and most thought being assessed by peers was acceptable. The physician tutors' comments were largely deemed as helpful. The mean mark for the course given by students was 4.6 (median 5) (1=very poor, 6=very good). Conclusion: A communication skills course expanded by video-annotations and group discussions with a physician tutor was shown to be feasible and was well received by students and faculty.

Guidance for providing effective feedback in clinical supervision in postgraduate medical education: a systematic review

The primary aim was to systematically review the empirical evidence relating to models and guidance for providing effective feedback in clinical supervision occurring in postgraduate medical education contexts. A secondary aim was to identify the common and differentiating components of models and guidance for providing effective feedback in this context. A systematic review was conducted. Fifty-one records met the inclusion criteria, including 12 empirical studies. Empirical records meeting inclusion criteria were critically appraised. Qualitative content analysis was applied to the guidance on effective and ineffective feedback provision to identify key principles. A composite model was created synthesising the guidance identified for providing effective feedback. The evidence supporting specific models and guidance in postgraduate medical education was limited. However, there is evidence to support all of the commonly identified principles. In addition, a consensus about the principles of effective feedback in clinical supervision in postgraduate medical education was found.

Use of feedback on medium-term blood pressure measurement skills in medical students: a randomized controlled trial

BACKGROUND

The objective of this study was to assess the effect of feedback on medium-term blood pressure (BP) measurement skills in medical students versus a control group.

METHODS

A randomized controlled trial in first-year medical students was conducted. After theoretical-practical training on measuring BP, the students were randomized into a control group or intervention group. The skill was assessed immediately after the training (T1) by evaluating students with a simulated standardized patient using a checklist. Students in the intervention group received feedback immediately after assessment, whereas the control group received no feedback. After 3 months (T2), each student was reassessed in the same way as for (T1).

RESULTS

Ninety-two first-year medical students took part in the study (45 in control group and 47 in intervention group). At T1, there were no differences in the skill measured. At T2 (after three months), there was a significant difference in the intervention group (score = 23.97 ± 3.82) compared with the control group (score = 20.91 ± 4.87), P < 0.001, d = 0.69. In the intervention group, the scores were maintained at 3 months (T1 = 23.23 and T2 = 23.97, P = 0.335), whereas in the control group, scores declined significantly (T1 = 23.44 and T2 = 20.91; P = 0.002).

CONCLUSION

Receiving feedback promotes retention of learning of BP measurement skills over the medium term. Further studies applying feedback to other skills should be conducted.

A case for feedback and monitoring assessment in competency-based medical education

PURPOSE

Within competency-based medical education, self-regulated learning (SRL) requires residents to leverage self-assessment and faculty feedback. We sought to investigate the potential for competency-based assessments to foster SRL by quantifying the relationship between faculty feedback and entrustment ratings as well as the congruence between faculty assessment and resident self-assessment.

MATERIALS AND METHODS

We collected comments in (a) an emergency medicine objective structured clinical examination group (objective structured clinical examinations [OSCE] and emergency medicine OSCE group [EMOG]) and (b) a first-year resident multidisciplinary resuscitation "Nightmares" course assessment group (NCAG) and OSCE group (NOG). We assessed comments across five domains including Initial Assessment (IA), Diagnostic Action (DA), Therapeutic Action (TA), Communication (COM), and entrustment. Analyses included structured qualitative coding and (non)parametric and descriptive analyses.

RESULTS

In the EMOG, faculty's positive comments in the entrustment domain corresponded to lower entrustment score Mean Ranks (MRs) for IA (<11.1), DA (<11.2), and entrustment (<11.6). In NOG, faculty's negative comments resulted in lower entrustment score MRs for TA (<11.8 and <10) and DA (<12.4), and positive comments resulted in higher entrustment score MRs for IA (>15.4) and COM (>17.6). In the NCAG, faculty's positive IA comments were negatively correlated with entrustment scores (ρ = -.27, P = .04). Across programs, faculty and residents made similar domain-specific comments 13% of the time.

CONCLUSIONS

Minimal and inconsistent associations were found between narrative and numerical feedback. Performance monitoring accuracy and feedback should be included in assessment validation.

Quality of Feedback in Higher Education: A Review of Literature

In raising the standards for professional educators, higher educators must be prepared to provide the highest quality feedback on student performance and work products toward improved outcomes. This review of the literature examined the major findings of 70 quantitative, mixed methods, or qualitative studies found in higher education journals across a range of disciplines. Multiple recommendations and results for feedback emerged which fall into the categories described by Susan Brookhart. This review found research for each of Brookhart’s categories, with results indicating differences between the perceptions of adherence to sound feedback practices versus the reality of implementation, the potential for innovative tool use, and a disagreement about the effectiveness of peers for providing effective feedback. Indicators for quality within the research confirmed the importance of commonly accepted standards such as positivity, specificity, timeliness, and encouraging active student participation. Additionally, trends and themes indicated a need for the consistent implementation of the feedback exchange process and flexibility to account for student input/preferences. Greater consistency toward the application of these quality indicators should be undertaken when determining the quality of higher education feedback for preservice teachers prior to undertaking summative licensure assessments.

Video-based feedback using real consultations for a formative assessment in communication skills

BACKGROUND

Pre-recorded videotapes have become the standard approach when teaching clinical communication skills (CCS). Furthermore, video-based feedback (VF) has proven to be beneficial in formative assessments. However, VF in CCS with the use of pre-recorded videos from real-life settings is less commonly studied than the use of simulated patients. To explore: 1) perceptions about the potential benefits and challenges in this kind of VF; 2) differences in the CCC scores in first-year medical residents in primary care, before and after a communication program using VF in a curricular formative assessment.

METHOD

We conducted a pre/post study with a control group. The intervention consisted of VF sessions regarding CCS, performed in a small group with peers and a facilitator. They reviewed clinical consultations pre-recorded in a primary care setting with real patients. Before and after the intervention, 54 medical residents performed two clinical examinations with simulated patients (SP), answered quantitative scales (Perception of Patient-Centeredness and Jefferson Empathy Scale), and semi-structured qualitative questionnaires. The performances were scored by SP (Perception of Patient-Centeredness and CARE scale) and by two blind raters (SPIKES protocol-based and CCOG-based scale). The quantitative data analysis employed repeated-measures ANOVA. The qualitative analysis used the Braun and Clarke framework for thematic analysis.

RESULTS

The quantitative analyses did not reveal any significant differences in the sum scores of the questionnaires, except for the Jefferson Empathy Scale. In the qualitative questionnaires, the main potential benefits that emerged from the thematic analysis of the VF method were self-perception, peer-feedback, patient-centered approach, and incorporation of reflective practices. A challenging aspect that emerged from facilitators was the struggle to relate the VF with theoretical references and the resident's initial stress to record and watch oneself on video.

CONCLUSION

VF taken from real-life settings seems to be associated with a significant increase in self-perceived empathy. The study of other quantitative outcomes related to this VF intervention needs larger sample sizes. VF with clinical patients from real healthcare settings appears to be an opportunity for a deeper level of self-assessment, peer-feedback, and reflective practices.

Feedback in medical education - a workshop report with practical examples and recommendations

Background: As a teaching method, feedback is an integral part of medical education. However, there is a lack of a uniform theoretical basis or generally recognized guidelines for its specific design. Against this background, the aim of this article is to discuss conceptual considerations and empirical findings regarding feedback using various practical examples. Procedure and conceptual considerations: Building on the results of a workshop of the Committee for Communicative and Social Competences of the Society for Medical Education (GMA), this article first explains central conceptual considerations and empirical results on the topic of feedback. A particular focus is on various variables that influence the effect of feedback. This includes the feedback source, the frequency of feedback, starting points of feedback, the connection between feedback and reflection as well as the motivation and meta-cognitive skills of the feedback recipient. Practical examples: The implementation of feedback in practice is illustrated using eight examples from the field of medical and dental education. They stem from various settings and the focus is on formative oral feedback. It will become evident that the focus is more on the givers of feedback than the recipients of feedback. Instructions for recipients of feedback on how to reflect on it is still the exception. Discussion: Many of the relevant aspects for the effect of feedback described in the literature are already taken into account in the practical examples discussed. In conclusion, seven recommendations are made for implementing feedback in practice.

Using conversation analysis to explore feedback on resident performance

Feedback on clinical performance of residents is seen as a fundamental element in postgraduate medical education. Although literature on feedback in medical education is abundant, many supervisors struggle with providing this feedback and residents experience feedback as insufficiently constructive. With a detailed analysis of real-world feedback conversations, this study aims to contribute to the current literature by deepening the understanding of how feedback on residents' performance is provided, and to formulate recommendations for improvement of feedback practice. Eight evaluation meetings between program directors and residents were recorded in 2015-2016. These meetings were analyzed using conversation analysis. This is an ethno-methodological approach that uses a data-driven, iterative procedure to uncover interactional patterns that structure naturally occurring, spoken interaction. Feedback in our data took two forms: feedback as a unidirectional activity and feedback as a dialogic activity. The unidirectional feedback activities prevailed over the dialogic activities. The two different formats elicit different types of resident responses and have different implications for the progress of the interaction. Both feedback formats concerned positive as well as negative feedback and both were often mitigated by the participants. Unidirectional feedback and mitigating or downplaying feedback is at odds with the aim of feedback in medical education. Dialogic feedback avoids the pitfall of a program director-dominated conversation and gives residents the opportunity to take ownership of their strengths and weaknesses, which increases chances to change resident behavior. On the basis of linguistic analysis of our real-life data we suggest implications for feedback conversations.

Theory, a lost character? As presented in general practice education research papers

OBJECTIVES

The use of theory in research is reflected in its presence in research writing. Theory is often an ineffective presence in medical education research papers. To progress the effective use of theory in medical education, we need to understand how theory is presented in research papers. This study aims to elicit how theory is being written into general practice (GP) vocational education research papers in order to elucidate how theory might be more effectively used. This has relevance for the field of GP and for medical education more broadly.

METHODS

This is a scoping review of the presentation of theory in GP vocational education research published between 2013 and 2017. An interpretive approach is taken. We frame research papers as a form of narrative and draw on the theories of Aristotle's poetics and Campbell's monomyth. We seek parallels between the roles of theory in a research story and theories of characterisation.

RESULTS

A total of 23 papers were selected. Theories of 'reflective learning', 'communities of practice' and 'adult learning' were most used. Six tasks were assigned to theory: to align with a position; to identify a research problem; to serve as a vehicle for an idea; to provide a methodological tool; to interpret findings, and to represent an object of examination. The prominence of theory in the papers ranged from cameo to major roles. Depending on the way theory was used and the audience, theory had different impacts. There were parallels between the tasks assigned to theory and the roles of four of Campbell's archetypal characters. Campbell's typology offers guidance on how theory can be used in research paper 'stories'.

CONCLUSIONS

Theory can be meaningfully present in the story of a research paper if it is assigned a role in a deliberate way and this is articulated. Attention to the character development of theory and its positioning in the research story is important.

Using deliberate practice framework to assess the quality of feedback in undergraduate clinical skills training

BACKGROUND

In this research paper we report on the quality of feedback provided in the logbooks of pre-clinical undergraduate students based on a model of 'actionable feedback'. Feedback to clinical learners about their performance is crucial to their learning, which ultimately impacts on their development into competent clinicians. Due to students' concerns regarding the inconsistency and quality of feedback provided by clinicians, a structured feedback improvement strategy to move feedback forward was added to the clinical skills logbook. The instrument was also extended for peer assessment. This study aims to assess the quality of feedback using the deliberate practice framework.

METHODS

A feedback scoring system was used to retrospectively assess the quality of tutor and peer logbook feedback provided to second and third year medical students to identify deliberate practice components i.e. task, performance gap and action plan. The sample consisted of 425 second year and 600 third year feedback responses over a year.

RESULTS

All three deliberate practice components were observed in the majority of the written feedback for both classes. The frequency was higher in peer (83%, 89%) than tutor logbook assessments (51%, 67%) in both classes respectively. Average tutor and peer task, gap and action feedback scores ranged from 1.84-2.07 and 1.93-2.21 respectively. The overall quality of feedback provided by the tutor and peer was moderate and less specific (average score < or = 2). The absence of the three components was noted in only 1% of the feedback responses in both 2nd and 3rd year.

CONCLUSION

This study found that adding in a feed-forward strategy to the logbooks increased the overall quality of tutor and peer feedback as the task, gap and action plans were described. Deliberate practice framework provides an objective assessment of tutor and peer feedback quality and can be used for faculty development and training. The findings from our study suggest that the ratings from the tool can also be used as guidelines to provide feedback providers with feedback on the quality of feedback they provided. This includes specifically describing a task, performance gap and providing a learning plan as feed-forward to enhance feedback given.

Twelve tips on how to provide self-regulated learning (SRL) enhanced feedback on clinical performance

The provision of self-regulated learning (SRL) enhanced feedback on performing clinical skills and making a clinical diagnosis recognizes the importance of feedback on the use of key SRL processes. In contrast to the broader concept of self-directed learning, SRL has a specific focus on the individual learner's approach to achieve a task, including their planning, self-monitoring, and future adaptations. The key SRL processes can be identified using structured microanalysis during the clinical task and feedback to the learner using the tips outlined in this article. It is essential that SRL enhanced feedback is integrated with best practice on providing feedback to ensure that its potential is achieved.

Communicating medication risk to cardiovascular patients in Qatar

Purpose

Patient safety is gaining prominence in health professional curricula. Patient safety must be complemented by teaching and skill development in practice settings. The purpose of this paper is to explore how experienced pharmacists identify, prioritize and communicate adverse drug effects to patients.

Design/methodology/approach

A focus group discussion was conducted with cardiology pharmacy specialists working in a Doha hospital, Qatar. The topic guide sought to explore participants’ views, experiences and approaches to educating patients regarding specific cardiovascular therapy safety and tolerability. Discussions were audio-recorded and transcribed verbatim. Data were coded and organized around identified themes and sub-themes. Working theories were developed by the three authors based on relevant topic characteristics associated with the means in which pharmacists prioritize and choose adverse effect information to communicate to patients.

Findings

Nine pharmacists participated in the discussion. The specific adverse effects prioritized were consistent with the reported highest prevalence. Concepts and connections to three main themes described how pharmacists further tailored patient counseling: potential adverse effects and their perceived importance; patient encounter; and cultural factors. Pharmacists relied on initial patient dialogue to judge an individual’s needs and capabilities to digest safety information, and drew heavily upon experience with other counseling encounters to further prioritize this information, processes dependent upon development and accessing exemplar cases.

Originality/value

The findings underscore practical experience as a critical instructional element of undergraduate health professional patient safety curricula and for developing associated clinical reasoning.

The Feedback Tango: An Integrative Review and Analysis of the Content of the Teacher-Learner Feedback Exchange

PURPOSE

To conduct an integrative review and analysis of the literature on the content of feedback to learners in medical education.

METHOD

Following completion of a scoping review in 2016, the authors analyzed a subset of articles published through 2015 describing the analysis of feedback exchange content in various contexts: audiotapes, clinical examination, feedback cards, multisource feedback, videotapes, and written feedback. Two reviewers extracted data from these articles and identified common themes.

RESULTS

Of the 51 included articles, about half (49%) were published since 2011. Most involved medical students (43%) or residents (43%). A leniency bias was noted in many (37%), as there was frequently reluctance to provide constructive feedback. More than one-quarter (29%) indicated the feedback was low in quality (e.g., too general, limited amount, no action plans). Some (16%) indicated faculty dominated conversations, did not use feedback forms appropriately, or provided inadequate feedback, even after training. Multiple feedback tools were used, with some articles (14%) describing varying degrees of use, completion, or legibility. Some articles (14%) noted the impact of the gender of the feedback provider or learner.

CONCLUSIONS

The findings reveal that the exchange of feedback is troubled by low-quality feedback, leniency bias, faculty deficient in feedback competencies, challenges with multiple feedback tools, and gender impacts. Using the tango dance form as a metaphor for this dynamic partnership, the authors recommend ways to improve feedback for teachers and learners willing to partner with each other and engage in the complexities of the feedback exchange.

Peer-assisted debriefing of multisource feedback: an exploratory qualitative study

BACKGROUND

The Manitoba Physician Achievement Review (MPAR) is a 360-degree feedback assessment that physicians undergo every 7 years to retain licensure. Deliberate reflection on feedback has been demonstrated to encourage practice change. The MPAR Reflection Exercise (RE), a peer-assisted debriefing tool, was developed whereby the physician selects a peer with whom to review and reflect on feedback, committing to change. This qualitative study explores how physicians who had undergone the MPAR used the RE, what areas of change are identified and committed to, and what they perceived as the role of reflection in the MPAR process.

METHODS

The MPAR RE was piloted out to a cohort of MPAR-reviewed physicians. Thematic analysis was conducted on completed exercises (n = 61). Semi-structured interviews were conducted with individuals (n = 6) who completed the MPAR RE until saturation was reached.

RESULTS

Physicians reviewed feedback with a range of peers, including colleagues, staff, and spouses. Many physicians were surprised by feedback, both positive and negative, but interviewees found the RE useful in processing feedback. Areas where physicians committed to change were diverse, covering all CanMEDS roles. Most physicians identified themselves as being successful in implementing change, though time, habit, and structures were cited as barriers.

CONCLUSIONS

Peer-assisted debriefing can assist reflection of multisource feedback. It is easy to implement, is not resource-intensive, and feedback implies that it is effective at promoting change. Participants, with the aid of peers, identified areas for change, developed approaches for change, and largely thought themselves successful at implementing changes. Areas of change included all seven CanMEDS roles.

The impact of expert- and peer feedback on communication skills of undergraduate dental students - a single-blinded, randomized, controlled clinical trial

OBJECTIVE

To evaluate the effect of peer- and expert feedback on communication skills of undergraduate dental students.

METHODS

All students of the first clinical treatment course (n=46) were randomly assigned into two groups. For three times a medical-dental interview/consultation of each student with a real patient was videotaped. After every consultation the videos were assessed either by a person experienced in communication (expert group) or by a fellow student (peer group), giving the students feedback regarding their chairside performed communication skills. Before and after the feedback-interventions all students conducted an interview with simulated patients, which was rated using a validated global rating and analyzed statistically.

RESULTS

Global ratings mean scores after feedback-intervention were significantly improved (p<0.05). Thereby, no significant differences in the overall assessment could be observed between expert and peer feedback (p>0.05).

CONCLUSION

During this study students improved their communication skills in dentist-patient interactions. The communication experience of the feedback provider seems not to have any impact on the communication skills in undergraduate dental students.

PRACTICE IMPLICATIONS

The clinical courses in dentistry offer the opportunity to implement peer-feedback interventions in real treatment situation as part of communication training to longitudinally improve communication skills.

High-Quality Feedback Regarding Professionalism and Communication Skills in Otolaryngology Resident Education

Objective

The Accreditation Council for Graduate Medical Education (ACGME) requires competency-based education for residents and recommends 5 basic features of high-quality feedback. Our aim was to examine the incorporation of feedback in articles regarding professionalism and interpersonal/communication skills for otolaryngology residency training curriculum.

Data Sources

PubMed, Embase, ERIC, Cochrane Library, Web of Science, Scopus, and ClinicalTrials.gov .

Methods

We used studies identified during a systematic review of all indexed years through October 4, 2016.

Results

Eighteen studies were included in this review. Professionalism was discussed in 16, of which 15 (94%) examined aspects of feedback. Interpersonal/communication skills were the focus of 16 articles, of which 14 16 (88%) discussed aspects of feedback. Our assessment demonstrated that timeliness was addressed in 8 (44%) articles, specificity in 4 (22%), learner reaction and reflection in 4 (22%), action plans in 3 (20%), and balancing reinforcing/corrective feedback in 2 (13%). Two articles did not address feedback, and 6 did not address aspects of high-quality feedback. The ACGME-recommended feedback systems of ADAPT (ask, discuss, ask, plan together) and R2C2 (relationship, reactions, content, and coach) were not reported in any of the studies.

Conclusion

Feedback is an essential component of graduate medical education and is required by the ACGME milestones assessment system. However, the core feedback components recommended by the ACGME are rarely included in the otolaryngology resident education literature.

Self-entrustment: how trainees' self-regulated learning supports participation in the workplace

Clinical workplaces offer postgraduate trainees a wealth of opportunities to learn from experience. To promote deliberate and meaningful learning self-regulated learning skills are foundational. We explored trainees' learning activities related to patient encounters to better understand what aspects of self-regulated learning contribute to trainees' development, and to explore supervisor's role herein. We conducted a qualitative non-participant observational study in seven general practices. During two days we observed trainee's patient encounters, daily debriefing sessions and educational meetings between trainee and supervisor and interviewed them separately afterwards. Data collection and analysis were iterative and inspired by a phenomenological approach. To organise data we used networks, time-ordered matrices and codebooks. Self-regulated learning supported trainees to increasingly perform independently. They engaged in self-regulated learning before, during and after encounters. Trainees' activities depended on the type of medical problem presented and on patient, trainee and supervisor characteristics. Trainees used their sense of confidence to decide if they could manage the encounter alone or if they should consult their supervisor. They deliberately used feedback on their performance and engaged in reflection. Supervisors appeared vital in trainees' learning by reassuring trainees, discussing experience, knowledge and professional issues, identifying possible unawareness of incompetence, assessing performance and securing patient safety. Self-confidence, reflection and feedback, and support from the supervisor are important aspects of self-regulated learning in practice. The results reflect how self-regulated learning and self-entrustment promote trainees' increased participation in the workplace. Securing organized moments of interaction with supervisors is beneficial to trainees' self-regulated learning.

Self-Observation and Peer Feedback as a Faculty Development Approach for Problem-Based Learning Tutors: A Program Evaluation

PROBLEM

Good teaching requires spontaneous, immediate, and appropriate action in response to various situations. It is even more crucial in problem-based learning (PBL) tutorials, as the tutors, while directing students toward the identification and attainment of learning objectives, must stimulate them to contribute to the process and provide them with constructive feedback. PBL tutors in medicine lack opportunities to receive feedback from their peers on their teaching strategies. Moreover, as tutorials provide little or no time to stop and think, more could be learned by reflecting on the experience than from the experience itself. We designed and evaluated a faculty development approach to developing PBL tutors that combined self-reflection and peer feedback processes, both powerful techniques for improving performance in education.

INTERVENTION

We developed an observation instrument for PBL facilitation to be used both by tutors to self-observe and reflect on own teaching strategies and by peers to observe and provide feedback to tutors. Twenty PBL sessions were video-recorded. Tutors completed the instrument immediately after their PBL session and again while watching their video-recorded session (self-observation). A group of three observers completed the instrument while watching each recorded session and provided feedback to each tutor (peer observation and feedback). We investigated tutors' perceptions of the feasibility and acceptability of the approach and gathered data on its effectiveness in enhancing tutors' facilitation skills.

CONTEXT

The preclinical medical curriculum at the University of Geneva is essentially taught by PBL. A new program of faculty development based on self-observation and peer feedback was offered to voluntary tutors and evaluated.

OUTCOME

Our results suggest that self-observation and peer feedback, supported by an instrument, can be effective in enhancing tutors' facilitation skills. Reflection on self-observation raised teachers' awareness of the effectiveness of the strategies they used to foster student learning. This motivated a need to change their teaching practice. However, for the changes to become operative, peer feedback was required, providing the cues and strategies needed to improve the facilitation skills.

LESSONS LEARNED

Peer coaching was considered feasible and useful to improve tutors' facilitation skills. Evaluating the program made it possible to assess tutors' needs and the reasons underlying their difficulties, and this in turn provided the basis for advanced workshops. Nonetheless, aspects related to logistics and the time constraints of such an individualized approach, as well as the cultural appropriation of peer coaching, might be obstacles that need to be addressed.

How Entrustment Is Informed by Holistic Judgments Across Time in a Family Medicine Residency Program: An Ethnographic Nonparticipant Observational Study

PURPOSE

Entrustment has mainly been conceptualized as delegating discrete professional tasks. Because residents provide most of their patient care independently, not all resident performance is visible to supervisors; the entrustment process involves more than granting discrete tasks. This study explored how supervisors made entrustment decisions based on residents' performance in a long-term family medicine training program.

METHOD

A qualitative nonparticipant observational study was conducted in 2014-2015 at competency-based family medicine residency programs in the Netherlands. Seven supervisor-resident pairs participated. During two days, one researcher observed first-year residents' patient encounters, debriefing sessions, and supervisor-resident educational meetings and interviewed them separately afterwards. Data were collected and analyzed using iterative, phenomenological inductive research methodology.

RESULTS

The entrustment process developed over three phases. Supervisors based their initial entrustment on prior knowledge about the resident. In the ensuing two weeks, entrustment decisions regarding independent patient care were derived from residents' observed general competencies necessary for a range of health problems (clinical reasoning, decision making, relating to patients); medical knowledge and skills; and supervisors' intuition. Supervisors provided supervision during and after encounters. Once residents performed independently, supervisors kept reevaluating their decisions, informed by residents' overall growth in competencies rather than by adhering to a predefined set of tasks.

CONCLUSIONS

Supervisors in family medicine residency training took a holistic approach to trust, based on general competencies, knowledge, skills, and intuition. Entrustment started before training and developed over time. Building trust is a mutual process between supervisor and resident, requiring a good working relationship.

Core principles of assessment in competency-based medical education

The meaningful assessment of competence is critical for the implementation of effective competency-based medical education (CBME). Timely ongoing assessments are needed along with comprehensive periodic reviews to ensure that trainees continue to progress. New approaches are needed to optimize the use of multiple assessors and assessments; to synthesize the data collected from multiple assessors and multiple types of assessments; to develop faculty competence in assessment; and to ensure that relationships between the givers and receivers of feedback are appropriate. This paper describes the core principles of assessment for learning and assessment of learning. It addresses several ways to ensure the effectiveness of assessment programs, including using the right combination of assessment methods and conducting careful assessor selection and training. It provides a reconceptualization of the role of psychometrics and articulates the importance of a group process in determining trainees' progress. In addition, it notes that, to reach its potential as a driver in trainee development, quality care, and patient safety, CBME requires effective information management and documentation as well as ongoing consideration of ways to improve the assessment system.

The quality of feedback during formative OSCEs depends on the tutors' profile

BACKGROUND

During their pre-clinical years, medical students are given the opportunity to practice clinical skills with simulated patients. During these formative objective structured clinical encounters (OSCEs), tutors from various backgrounds give feedback on students' history taking, physical exam, and communication skills. The aim of the study was to evaluate whether the content and process of feedback varied according to the tutors' profile.

METHODS

During 2013, all 2^nd and 3^rd year medical students and tutors involved in three formative OSCEs were asked to fill in questionnaires, and their feedback sessions were audiotaped. Tutors were divided into two groups: 1) generalists: primary care, general internist and educationalist physicians 2) specialists involved in the OSCE related to their field of expertise. Outcome measures included the students' perceptions of feedback quality and utility and objective assessment of feedback quality.

RESULTS

Participants included 251 medical students and 38 tutors (22 generalists and 16 specialists). Students self-reported that feedback was useful to improve history taking, physical exam and communication skills. Objective assessment showed that feedback content essentially focused on history taking and physical exam skills, and that elaboration on clinical reasoning or communication/professionalism issues was uncommon. Multivariate analyses showed that generalist tutors used more learner-centered feedback skills than specialist tutors (stimulating student's self-assessment (p < .001; making the student active in finding solutions, p < .001; checking student's understanding, p < .001) and elaborated more on communication and professionalism issues (p < 0.001). Specialists reported less training in how to provide feedback than generalists.

CONCLUSION

These findings suggest that generalist tutors are more learner-centered and pay more attention to communication and professionalism during feedback than specialist tutors. Such differences may be explained by differences in feedback training but also by differences in practice styles and frames of references that should be further explored.

Reclaiming a theoretical orientation to reflection in medical education research: a critical narrative review

CONTEXT

Reflection and reflective practice have become popular topics of scholarly dialogue in medical education. This popularity has given rise to checklists, portfolios and other tools to inspire and document reflection. We argue that some of the common ways in which reflection has been applied are influenced by broader discourses of assessment and evidence, and divorced from original theories of reflection and reflective practice.

METHODS

This paper was developed using a critical narrative approach. First we present two theoretical lenses provided by theories of reflection. Next we present a summary of relevant literature, indexed in PubMed from 2004 to 2014, relating to the application of reflection or reflective practice to undergraduate and postgraduate medical education. We categorise these articles broadly by trends and problematise the trends relative to the two theoretical lenses of reflection.

RESULTS

Two relevant theoretical orientations of reflection for medical education are: (i) reflection as epistemology of practice, and (ii) reflection as critical social inquiry. Three prevalent trends in the application of reflection to medical education are: (i) utilitarian applications of reflection; (ii) a focus on the self as the object of reflection, and (iii) reflection and assessment. These trends align with dominant epistemological positions in medicine, but not with those that underpin reflection.

CONCLUSIONS

We argue for continued theorising of and theoretically informed applications of reflection, drawing upon epistemologies of practice and critical reflection as critical social inquiry. These directions offer medical education research broad and deep potential in theories of reflection, particularly in relation to knowledge creation within uncertain and complex situations, and challenging of dominant discourses and structures. Future work could explore how dominant epistemological positions and discourses in medicine influence theories from other disciplines when these theories are deployed in medical education.

Self-evaluation and peer-feedback of medical students' communication skills using a web-based video annotation system. Exploring content and specificity

OBJECTIVE

Self-evaluation and peer-feedback are important strategies within the reflective practice paradigm for the development and maintenance of professional competencies like medical communication. Characteristics of the self-evaluation and peer-feedback annotations of medical students' video recorded communication skills were analyzed.

METHOD

Twenty-five year 4 medical students recorded history-taking consultations with a simulated patient, uploaded the video to a web-based platform, marked and annotated positive and negative events. Peers reviewed the video and self-evaluations and provided feedback. Analyzed were the number of marked positive and negative annotations and the amount of text entered. Topics and specificity of the annotations were coded and analyzed qualitatively.

RESULTS

Students annotated on average more negative than positive events. Additional peer-feedback was more often positive. Topics most often related to structuring the consultation. Students were most critical about their biomedical topics. Negative annotations were more specific than positive annotations. Self-evaluations were more specific than peer-feedback and both show a significant correlation. Four response patterns were detected that negatively bias specificity assessment ratings.

CONCLUSION

Teaching students to be more specific in their self-evaluations may be effective for receiving more specific peer-feedback.

PRACTICE IMPLICATIONS

Videofragmentrating is a convenient tool to implement reflective practice activities like self-evaluation and peer-feedback to the classroom in the teaching of clinical skills.

Factors influencing trainers' feedback-giving behavior: a cross-sectional survey

BACKGROUND

The literature provides some insight into the role of feedback givers, but little information about within-trainer factors influencing 'feedback-giving behaviours'. We looked for relationships between characteristics of feedback givers (self-efficacy, task perception, neuroticism, extraversion, agreeableness and conscientiousness) and elements of observation and feedback (frequency, quality of content and consequential impact).

METHODS

We developed and tested several hypotheses regarding the characteristics and elements in a cross-sectional digital survey among GP trainers and their trainees in 2011 and 2012. We conducted bivariate analysis using Pearson correlations and performed multiple regression analysis.

RESULTS

Sixty-two trainer-trainee couples from three Dutch institutions for postgraduate GP training participated in the study. Trainer scores on 'task perception' and on a scale of the trait 'neuroticism' correlated positively with frequency of feedback and quality of feedback content. Multiple regression analysis supported positive correlations between task perception and frequency of feedback and between neuroticism and quality of feedback content. No other correlations were found.

CONCLUSION

This study contributes to the literature on feedback giving by revealing factors that influence feedback-giving behaviour, namely neuroticism and task perception. Trainers whose task perception included facilitation of observation and feedback (task perception) and trainers who were concerned about the safety of their patients during consultations with trainees (neuroticism) engaged more frequently in observation and feedback and gave feedback of higher quality.