Feedback-giving behaviour in performance evaluations during clinical clerkships

Peer assessment in medical communication skills training in programmatic assessment: A qualitative study examining faculty and student perceptions

INTRODUCTION

Current literature recommends assessment of communication skills in medical education combining different settings and multiple observers. There is still a gap in understanding about whether and how peers assessment facilitates learning in communication skills training.

METHODS

We designed a qualitative study using focus group interviews and thematic analysis, in a medical course in the Netherlands. We aimed to explore medical students' and teachers' experiences, perceptions, and perspectives about challenges and facilitating factors in PACST (Peer assessment in medical communication skills training).

RESULTS

Most of the participants reported that peer feedback was a valuable experience when learning communication skills. The major challenges for the quality and credibility of PACST reported by the participants are the question whether peer feedback is critical enough for learning and the difficulty of actually engaging students in the assessment process.

CONCLUSION

Teachers reviewing students' peer assessments may improve the quality and their credibility and the reviewed assessments can best be used for learning purposes. We suggest to pay sufficient attention to teachers' roles in PACST, ensuring a safe and trustworthy environment and additionally helping students to internalize the value of being vulnerable during the evaluation process.

Feedback-based learning from viewpoints of surgical nursing students: A mixed-method study

BACKGROUND

Feedback-based learning (FBL) focuses on guiding the learning process according to educational objectives and the student's needs. This study aimed to investigate surgical nursing students' perceptions and explore their experiences of FBL.

METHOD

The present study used a mixed-methods sequential explanatory design that was conducted in the quantitative and qualitative phases. Surgical nursing students participated in the quantitative phase (n = 105). In the first phase, students completed two questionnaires about FBL and clinical feedback. Semi-structured face-to-face interviews were used to collect qualitative data in the second phase. Graneheim and Lundman's inductive approaches were used to analyse the qualitative data.

RESULTS

The mean (SD) score for students' perception of FBL was 3.99 ± 0.70. The qualitative results were explored in two themes, "motivational support for improvement" and "unpleasant learning".

CONCLUSION

In this study, the positive and negative aspects of FBL were explained. FBL is perceived as a motivational support mechanism to improve students' capabilities during their academic courses and also prepare them for future careers. Conversely, FBL may experience unpleasant learning due to negative feedback and negative emotions.

Learning in the workplace: Development of a simple language statement assessment tool that supports second-level nurse practice

AIM

To focus learning through clarity of the enrolled nurse (EN) role (a second tier nurse position) through development of a user-friendly workplace performance assessment tool commensurate with EN standards for practice.

BACKGROUND

Internationally, the nursing workforce comprises regulated and unregulated staff. In Australia, similar to other western countries, there are two tiers of regulated workforce, namely Registered Nurses (RNs) and Enrolled Nurses (ENs). Differences in RN and EN standards based on the education preparation are not always clearly differentiated in workplace practice. Roles are often seen as interchangeable: Improved clarity of both regulated and unregulated roles, when numbers of healthcare workers are burgeoning, assists performance assessment that guides further learning and safe practice.

DESIGN

Two phase sequential, non-experimental design.

METHODS

Phase one used focus groups (n=48), expert reference panel (n=8) and end-users (n=16) to develop simple language statements. Phase two involved field testing of the statements.

FINDINGS

A 30-item, criterion-based workplace performance tool was developed. Principal component analysis of completed tools indicated work could be organised around three key areas of practice, namely, higher order thinking and problem solving, routine daily activities of care and personal and social attributes.

DISCUSSION

Participants reported the statement items assisted in determining suitable activities and accompanying cues in discussing learning needs. Analysis assisted with discriminating broader elements of EN workplace performance.

CONCLUSIONS

Workplace learning is important for nurses to continue to build their capacity to deliver optimum care. Assessment tools that describe professional capability in plain language statements and provide examples of supportive behavioural cues help guide on-going learning through improving the validity and thereby consistency of assessment processes. Furthermore, comprehensible and meaningful statements and cues can readily be adopted by students and educators to target learning and feedback thereby enhancing clarity of the EN role, to distinguish from other nursing roles.

Designing feedback processes in the workplace-based learning of undergraduate health professions education: a scoping review

BACKGROUND

Feedback processes are crucial for learning, guiding improvement, and enhancing performance. In workplace-based learning settings, diverse teaching and assessment activities are advocated to be designed and implemented, generating feedback that students use, with proper guidance, to close the gap between current and desired performance levels. Since productive feedback processes rely on observed information regarding a student's performance, it is imperative to establish structured feedback activities within undergraduate workplace-based learning settings. However, these settings are characterized by their unpredictable nature, which can either promote learning or present challenges in offering structured learning opportunities for students. This scoping review maps literature on how feedback processes are organised in undergraduate clinical workplace-based learning settings, providing insight into the design and use of feedback.

METHODS

A scoping review was conducted. Studies were identified from seven databases and ten relevant journals in medical education. The screening process was performed independently in duplicate with the support of the StArt program. Data were organized in a data chart and analyzed using thematic analysis. The feedback loop with a sociocultural perspective was used as a theoretical framework.

RESULTS

The search yielded 4,877 papers, and 61 were included in the review. Two themes were identified in the qualitative analysis: (1) The organization of the feedback processes in workplace-based learning settings, and (2) Sociocultural factors influencing the organization of feedback processes. The literature describes multiple teaching and assessment activities that generate feedback information. Most papers described experiences and perceptions of diverse teaching and assessment feedback activities. Few studies described how feedback processes improve performance. Sociocultural factors such as establishing a feedback culture, enabling stable and trustworthy relationships, and enhancing student feedback agency are crucial for productive feedback processes.

CONCLUSIONS

This review identified concrete ideas regarding how feedback could be organized within the clinical workplace to promote feedback processes. The feedback encounter should be organized to allow follow-up of the feedback, i.e., working on required learning and performance goals at the next occasion. The educational programs should design feedback processes by appropriately planning subsequent tasks and activities. More insight is needed in designing a full-loop feedback process, in which specific attention is needed in effective feedforward practices.

The Interactive Relationship Between Instructor Perceptions and Learner Personality on Surgical Skills Performance

OBJECTIVE

The objective of this study was to examine the association between learner personality and capacity to be trained (i.e., performance improvement) on a surgical task, and how instructor perceptions of the learners' capacity to be trained interact with learner personality and performance during training and feedback. There is meaningful heterogeneity in the degree of learner surgical skills acquisition despite receiving the same amount of training. While learner personality may independently contribute to skill acquisition, the instructor-learner feedback process is also important to consider. To better understand this interpersonal relationship, it is necessary to also consider instructor factors (i.e., perceptions), and how this may contribute to learner variability in skills training.

DESIGN

This exploratory study employed a prospective two-phase design. Medical and non-medical undergraduate students (N = 62) completed measures of personality and participated in two 20-minute training sessions with expert feedback 2 weeks apart, performing an end-to-side anastomosis on a low-fidelity model. Learner performance and instructors' perceptions of a learner's capacity to be trained were assessed.

PARTICIPANTS

Sixty-two medical and non-medical undergraduate students.

RESULTS

There was a significant interaction between learner Extraversion and instructor's perceptions of learner capacity to be trained. Higher learner Extraversion was only associated with an increase in performance improvement for those who were considered trainable (OR = 4.83, p = 0.017). Post hoc analysis revealed a significant difference in the amount of feedback provided to participants who were considered trainable (M = 9.45) versus not trainable (M = 16.48).

CONCLUSIONS

This study highlights the importance of both individual learner factors and instructor perceptions on surgical skill acquisition.

Exploring complexities in the reform of assessment practice: a critical realist perspective

Although the principles behind assessment for and as learning are well-established, there can be a struggle when reforming traditional assessment of learning to a program which encompasses assessment for and as learning. When introducing and reporting reforms, tensions in faculty may arise because of differing beliefs about the relationship between assessment and learning and the rules for the validity of assessments. Traditional systems of assessment of learning privilege objective, structured quantification of learners' performances, and are done to the students. Newer systems of assessment promote assessment for learning, emphasise subjectivity, collate data from multiple sources, emphasise narrative-rich feedback to promote learner agency, and are done with the students. This contrast has implications for implementation and evaluative research. Research of assessment which is done to students typically asks, "what works", whereas assessment that is done with the students focuses on more complex questions such as "what works, for whom, in which context, and why?" We applied such a critical realist perspective drawing on the interplay between structure and agency, and a systems approach to explore what theory says about introducing programmatic assessment in the context of pre-existing traditional approaches. Using a reflective technique, the internal conversation, we developed four factors that can assist educators considering major change to assessment practice in their own contexts. These include enabling positive learner agency and engagement; establishing argument-based validity frameworks; designing purposeful and eclectic evidence-based assessment tasks; and developing a shared narrative that promotes reflexivity in appreciating the complex relationships between assessment and learning.

Midwives' experiences of completing written feedback: The emotions, challenges and solutions

AIM

Written feedback is a valued learning tool for midwifery students, providing information on clinical performance with the aim to improve future practice. One aim of this study was to explore the experiences of midwives in completing written feedback in the clinical setting.

DESIGN

This qualitative study is situated within a hermeneutic phenomenological framework.

METHODS

Data were collected through focus groups and individual interviews, then transcribed and subjected to thematic content analysis.

RESULTS

Three interconnected themes of Emotions, Challenges and Solutions were identified. Midwifery participants experienced strong emotional reactions (anxiety, guilt, frustration) around the completion of written feedback in the clinical setting due to four challenges (lack of time, continuity, clarity of feedback content and direct supervision), which resulted in solutions being employed to offset or minimise problematic written feedback.

CONCLUSIONS

Completing written feedback in the clinical setting was a challenging experience for participants in this study, affecting their ability to do so in some cases. This is concerning as the literature is supportive of the positive impact written feedback has on the growth and potential of students.

Analysis of Supervisors' Feedback to Residents on Communicator, Collaborator, and Professional Roles During Case Discussions

BACKGROUND

Literature examining the feedback supervisors give to residents during case discussions in the realms of communication, collaboration, and professional roles (intrinsic roles) focuses on analyses of written feedback and self-reporting.

OBJECTIVES

We quantified how much of the supervisors' verbal feedback time targeted residents' intrinsic roles and how well feedback time was aligned with the role targeted by each case. We analyzed the educational goals of this feedback. We assessed whether feedback content differed depending on whether the residents implied or explicitly expressed a need for particular feedback.

METHODS

This was a mixed-methods study conducted from 2017 to 2019. We created scripted cases for radiology and internal medicine residents to present to supervisors, then analyzed the feedback given both qualitatively and quantitatively. The cases were designed to highlight the CanMEDS intrinsic roles of communicator, collaborator, and professional.

RESULTS

Radiologists (n = 15) spent 22% of case discussions providing feedback on intrinsic roles (48% aligned): 28% when the case targeted the communicator role, 14% for collaborator, and 27% for professional. Internists (n = 15) spent 70% of discussions on intrinsic roles (56% aligned): 66% for communicator, 73% for collaborator, and 72% for professional. Radiologists' goals were to offer advice (66%), reflections (21%), and agreements (7%). Internists offered advice (41%), reflections (40%), and clarifying questions (10%). We saw no consistent effects when residents explicitly requested feedback on an intrinsic role.

CONCLUSIONS

Case discussions represent frequent opportunities for substantial feedback on intrinsic roles, largely aligned with the clinical case. Supervisors predominantly offered monologues of advice and agreements.

Where the rubber meets the road - An integrative review of programmatic assessment in health care professions education

INTRODUCTION

Programmatic assessment was introduced as an approach to design assessment programmes with the aim to simultaneously optimize the decision-making and learning function of assessment. An integrative review was conducted to review and synthesize results from studies investigating programmatic assessment in health care professions education in practice.

METHODS

The authors systematically searched PubMed, Web of Science, and ERIC to identify studies published since 2005 that reported empirical data on programmatic assessment. Characteristics of the included studies were extracted and synthesized, using descriptive statistics and thematic analysis.

RESULTS

Twenty-seven studies were included, which used quantitative methods (n = 10), qualitative methods (n = 12) or mixed methods (n = 5). Most studies were conducted in clinical settings (77.8%). Programmatic assessment was found to enable meaningful triangulation for robust decision-making and used as a catalyst for learning. However, several problems were identified, including overload in assessment information and the associated workload, counterproductive impact of using strict requirements and summative signals, lack of a shared understanding of the nature and purpose of programmatic assessment, and lack of supportive interpersonal relationships. Thematic analysis revealed that the success and challenges of programmatic assessment were best understood by the interplay between quantity and quality of assessment information, and the influence of social and personal aspects on assessment perceptions.

CONCLUSION

Although some of the evidence may seem compelling to support the effectiveness of programmatic assessment in practice, tensions will emerge when simultaneously stimulating the development of competencies and assessing its result. The identified factors and inferred strategies provide guidance for navigating these tensions.

Performance in practice; exploring trainer and trainee experiences of user-designed formative assessment tools

INTRODUCTION

General Practice training in Ireland currently has various methods of formative assessment and feedback delivered to trainees. In 2018 the Irish College of General Practitioners commissioned the generation of two new user-designed formative feedback tools that would allow trainee feedback to drive learning. These tools became known as the Performance in Practice (PiP) tools.

AIMS

To explore the experiences of General Practice (GP) trainers and trainees having completed a pilot of using the PiP tools for 4 months.

METHODS

An explorative phenomenological approach was taken to understand the experiences of trainers and trainees. One to one interviews were conducted, and the transcripts analysed for themes and sub-theme via Template analysis.

RESULTS

User experiences focused on two main areas; educational value and acceptability. In relation to educational value, the PiP tools were seen as an improvement over established forms of formative feedback, as they were centred around the curriculum and therefore reflected the unique multifaceted requirements of an independently practising GP. Acceptability primarily focused around data governance and structures, as well as practical issues such as ease of software use.

CONCLUSIONS

Overall, the experience of using the PiP tools was positive for both trainers and trainees. Future plans to further explore implementation of the PiP tools have been significantly informed by this research.

Student perspectives on competency-based portfolios: Does a portfolio reflect their competence development?

INTRODUCTION

Portfolio-based assessments require that learners' competence development is adequately reflected in portfolio documentation. This study explored how students select and document performance data in their portfolios and how they perceive these data to be representative for their competence development.

METHODS

Students uploaded performance data in a competency-based portfolio. During one clerkship period, twelve students also recorded an audio diary in which they reflected on experiences and feedback that they perceived to be indicants of their competence development. Afterwards, these students were interviewed to explore the extent to which the performance documentation in the portfolio corresponded with what they considered illustrative evidence of their development. The interviews were analyzed using thematic analysis.

RESULTS

Portfolios provide an accurate but fragmented picture of student development. Portfolio documentation was influenced by tensions between learning and assessment, student beliefs about the goal of portfolios, student performance evaluation strategies, the learning environment and portfolio structure.

DISCUSSION

This study confirms the importance of taking student perceptions into account when implementing a competency-based portfolio. Students would benefit from coaching on how to select meaningful experiences and performance data for documentation in their portfolios. Flexibility in portfolio structure and requirements is essential to ensure optimal fit between students' experienced competence development and portfolio content.

A view from the sender side of feedback: anticipated receptivity to clinical feedback when changing prior physicians' clinical decisions-a mixed methods study

When physicians transition patients, the physician taking over may change the diagnosis. Such a change could serve as an important source of clinical feedback to the prior physician. However, this feedback may not transpire if the current physician doubts the prior physician's receptivity to the information. This study explored facilitators of and barriers to feedback communication in the context of patient care transitions using an exploratory sequential, qualitative to quantitative, mixed methods design. Twenty-two internal medicine residents and hospitalist physicians from two teaching hospitals were interviewed and data were analyzed thematically. A prominent theme was participants' reluctance to communicate diagnostic changes. Participants perceived case complexity and physical proximity to facilitate, and hierarchy, unfamiliarity with the prior physician, and lack of relationship to inhibit communication. In the subsequent quantitative portion of the study, forty-one hospitalists completed surveys resulting in 923 total survey responses. Multivariable analyses and a mixed-effects model were applied to survey data with anticipated receptivity as the outcome variable. In the mixed-effects model, four factors had significant positive associations with receivers' perceived receptivity: (1) feedback senders' time spent on teaching services (β = 0.52, p = 0.02), (2) receivers' trustworthiness and clinical credibility (β = 0.49, p < 0.001), (3) preference of both for shared work rooms (β = 0.15, p = 0.006), and (4) receivers being peers (β = 0.24, p < 0.001) or junior colleagues (β = 0.39, p < 0.001). This study suggests that anticipated receptivity to feedback about changed clinical decisions affects clinical communication loops. Without trusting relationships and opportunities for low risk, casual conversations, hospitalists may avoid such conversations.

Influences on the implementation of Mini-CEX and DOPS for postgraduate medical trainees' learning: A grounded theory study

Introduction: In order for Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) to actually have a positive effect on trainees' learning, the way in which the tools are implemented is of key importance. However, there are many factors influencing their implementation. In this study, we aim to develop a comprehensive model of such factors. Methods: Using a constructivist grounded theory approach, we performed eight focus groups. Participants were postgraduate trainees and supervisors from three different specialties; all were experienced with Mini-CEX and/or DOPS. Data were analyzed for recurring themes, underlying concepts and their interactions using constant comparison. Results: We developed a model demonstrating how the implementation of Mini-CEX and DOPS for trainees' learning is influenced by 13 factors relating to four categories: organizational culture (e.g. value of teaching and feedback), work structure (e.g. time for Mini-CEX and DOPS, faculty development), instruments (e.g. content of assessment), and users (e.g. relationship between trainees and supervisors), and their interaction. Conclusions: We developed a complex model of influencing factors relating to four categories. Consideration of this model might support successful implementation and trainees' learning with Mini-CEX and DOPS.

A review of eye tracking for understanding and improving diagnostic interpretation

Inspecting digital imaging for primary diagnosis introduces perceptual and cognitive demands for physicians tasked with interpreting visual medical information and arriving at appropriate diagnoses and treatment decisions. The process of medical interpretation and diagnosis involves a complex interplay between visual perception and multiple cognitive processes, including memory retrieval, problem-solving, and decision-making. Eye-tracking technologies are becoming increasingly available in the consumer and research markets and provide novel opportunities to learn more about the interpretive process, including differences between novices and experts, how heuristics and biases shape visual perception and decision-making, and the mechanisms underlying misinterpretation and misdiagnosis. The present review provides an overview of eye-tracking technology, the perceptual and cognitive processes involved in medical interpretation, how eye tracking has been employed to understand medical interpretation and promote medical education and training, and some of the promises and challenges for future applications of this technology.

Application of mini-clinical evaluation exercise for assessing the integrated-based learning during physical diagnostic course

Medical education paradigm has been questioned for the requirements of improving the quality and quantity of medical students. This study was to explore the efficiency of integrated-based learning (IBL) used mini-clinical evaluation exercise (mini-CEX) during physical diagnostics course. One hundred and eleven volunteered students were randomly divided into three groups: lecture-based learning (LBL), case-based learning (CBL), and IBL. Nephrotic syndrome was the teaching content. In the IBL group, students were provided the guideline and additional interpretation from the instructor about the basic knowledge related to disease as vertical integration curriculum. Their performance was evaluated by mini-CEX and theoretical examination, respectively. All subjects have completed the study. The difference of five factors (medical interview, physical examination, clinical judgment, organizational effectiveness, and competence) in mini-CEX between IBL, CBL, and LBL were statistically significant (p <0.05). Sample sizes of below, meets, and above the expectations of mini-CEX in different instructional groups were statistically significant (X² =17.842, p =0.001). The final exam scores in IBL group and the CBL group were significantly higher than that of LBL group (F =41.553, p =0.000). And the relationship of final exam score only in the IBL group was positive existed with medical interview (R =0.466, p =0.004), physical examination (R =0.328, p =0.048), professional attitude (R =0.366, p =0.026), and communication skill (R =0.412, p =0.011). Therefore, our study revealed the effect of IBL on the medical students' skills. It highlights IBL could improve the physical examination, organizational effectiveness, and competence and the application of basic knowledge. © 2018 International Union of Biochemistry and Molecular Biology, 46(5):417-423, 2018.

Effect of individual structured and qualified feedback on improving clinical performance of dental students in clinical courses-randomised controlled study

AIM

Analysis of the effect of individual structured and qualified feedback (FB) on practical skills development of dental students during clinical courses.

METHODS

Fifty-three final-year dental students at Jena University Hospital participated in this prospective randomised controlled interventional study. Two calibrated assessors evaluated 128 pre- and post-assessments of 4 different dental treatment steps performed by dental students during the integrated clinical course in restorative dentistry and prosthodontics and the clinical course paediatric dentistry. The assessment included direct observation, graded and non-grading evaluation and was documented with a specific FB assessment tool. Dental students in the intervention group (IG) received an elaborated, structured and qualified FB after the pre-assessment that focussed on individual strengths and weaknesses, providing specific suggestions for improvement and establishing a personal learning goal. Participants were randomly allocated to the IG and the control group (CG).

RESULTS

In both groups, dental students significantly enhanced their performance, but the difference was higher in the IG than in the CG. Large effect sizes (ES) were observed in all observed items, but FB had largest effect size in improving technical skills (ES = 1.6), followed by management (ES = 1.3) and communication skills (ES = 0.8). Factors with the highest influence on FB in enhancing dental students' clinical performance were their insight into their own mistakes or omissions, the observed dental treatment step and the duration of FB.

CONCLUSION

Individual structured and qualified FB is an effective method to enhance dental students' professional performances and to individually guide the learning process.

Students perception towards feedback in clinical sciences in an outcome-based integrated curriculum

Background & Objective:

Feedback has been identified as one of the key strategies for learning in the outcome-based curriculum. Students are more interested in their grades paying little attention to the feedback, may not understand the importance of feedback and its effect on their performance because of their perception, and beliefs. Non-constructive feedback will not result in the improvement of the students’ performance. This study aims to explore; student’s perception of useful feedback; the purpose of feedback and believes about written feedback.

Methods:

This analytical cross-sectional study was conducted from November 2017 to January 2018 at Majmaah University. Students studying in clinical phase were recruited. Data were collected from 121 students by self-structured questionnaire using complete enumeration sampling method.

Results:

Majority of the students (45.5%) disagreed that the feedback should always contain marks; (49.6%) commented that the tutor did not provide enough constructive feedback. While we ask the purpose of feedback (62.8%), agree with two-way nature of feedback, and it is helpful to find there expected performance. Almost two third (67.8%) of the students believe that limited feedback is the reason for frustration and they did not receive comments for improvement.

Conclusions:

Students are aware of the purpose of feedback. Senior students give more value to feedback and in the opinion that feedback provides useful suggestions for future improvement and limited feedback is the reason for frustration. The results highlight the need for more structured feedback mechanism, and there is a need for faculty engagement in training to fill the existing gapes to create an effective educational alliance.

Does it matter who writes down the feedback? A comparison of teacher- vs. student-completed clinical encounter cards during clinical rotations in veterinary studies

Objective: Despite the fact that feedback (FB) provided by teachers to students is a recognised, effective teaching tool, successful use of feedback during clinical training depends on many factors. In addition to appropriate training and attitude of teachers, sustainable feedback requires an appropriate teaching culture and active commitment on the part of the students to receive, accept and use FB. This study examines the use of two different clinical encounter cards (CECs) during clinical rotation and investigates whether students take a more active part in the feedback process when using these cards. The objective of this study is to test whether it has a positive effect if students write down FB themselves and to verify this positive effect. Methodology: 161 students in their 9th semester of veterinary studies each had to use two clinical encounter cards (types 1 and 2) during their rotations on 10 wards. For this, students had to ask teachers for FB before starting a clinical activity. The oral FB given by the teachers was either written down on the CEC by the teachers (CEC type 1) or by the students (CEC type 2). Furthermore, the students were asked to assess their own performance by means of anchor criteria and to evaluate the quality of the FB provided by the teachers. Based on the entries in the CECs submitted, the following indicators for both CEC types could be calculated: (1) FB quantity and quality (length and specificity), (2) differentiation of self-assessment, as well as (3) level of satisfaction with the FB provided by the teachers. Results: With 2,377 CECs submitted, the mean CEC return rate was 74%. 99% of the cards showed positive FB, 69% contained constructive FB with suggestions for improvement, and 87% suggested specific next steps. On average, the FB written down by teachers was longer (12.4 versus 9.7 words) and more specific (1.9 versus 1.7 out of 3) than FB written down by students. Length and specificity decreased in the course of the semester. Neither the differentiation of self-assessment (proportion of differentiated entering of self-assessment) nor the students' level of satisfaction with the FB differed between the two examined CEC variants. Conclusion: The use of CECs across the cohort was successfully possible; however, the fact that students formulated and wrote down the FB themselves did not result in more comprehensive or more specific FB. Self-assessment and level of satisfaction with the teachers' FB remained unchanged.

The Feedback Tango: An Integrative Review and Analysis of the Content of the Teacher-Learner Feedback Exchange

PURPOSE

To conduct an integrative review and analysis of the literature on the content of feedback to learners in medical education.

METHOD

Following completion of a scoping review in 2016, the authors analyzed a subset of articles published through 2015 describing the analysis of feedback exchange content in various contexts: audiotapes, clinical examination, feedback cards, multisource feedback, videotapes, and written feedback. Two reviewers extracted data from these articles and identified common themes.

RESULTS

Of the 51 included articles, about half (49%) were published since 2011. Most involved medical students (43%) or residents (43%). A leniency bias was noted in many (37%), as there was frequently reluctance to provide constructive feedback. More than one-quarter (29%) indicated the feedback was low in quality (e.g., too general, limited amount, no action plans). Some (16%) indicated faculty dominated conversations, did not use feedback forms appropriately, or provided inadequate feedback, even after training. Multiple feedback tools were used, with some articles (14%) describing varying degrees of use, completion, or legibility. Some articles (14%) noted the impact of the gender of the feedback provider or learner.

CONCLUSIONS

The findings reveal that the exchange of feedback is troubled by low-quality feedback, leniency bias, faculty deficient in feedback competencies, challenges with multiple feedback tools, and gender impacts. Using the tango dance form as a metaphor for this dynamic partnership, the authors recommend ways to improve feedback for teachers and learners willing to partner with each other and engage in the complexities of the feedback exchange.

How the Start into the Clinical Elective Year Could be Improved: Qualitative Results and Recommendations from Student Interviews

Background: Entering the Clinical Elective Year (CEY) is a challenging transition phase for undergraduate medical students. Students become members of a professional team, thereby taking over certain tasks, which are executed more or less independently. Factors which facilitate (or impede) this transition in the perception of students are not well described. We therefore wanted to explore, what students perceived to be helpful during the first phase of the CEY and possibly derive respective recommendations. Methods: We conducted semi-structured interviews with 5th year medical students after they had completed the first two months of their CEY. Students were asked which problems they had faced and how they felt prepared for the CEY. Interviews were audio-recorded, transcribed, and analysed by qualitative content analysis. Results: From 34 interviews, we included 28 into analysis. Overall, 24 students were satisfied or very satisfied with their start into the CEY. Satisfaction was expressed with respect to workplace experiences, learning progress, responsibilities and team integration. Especially, students appreciated if they were integrated as active members of the team, were given responsibility for certain units of work, and received well-structured formal teaching and supervision. Students had divergent opinions about the quality of teaching and supervision, about their own achievements, and the recognition they received. Students recommended improvements in respect to formal teaching and supervision by clinical supervisors, preparation of the CEY by university, and supporting structures in the hosting institution. Conclusion: Students in this study were generally satisfied with the first two months of their CEY. Facilitating factors were active and responsible involvement into routine patient care, and high quality formal teaching and supervision. Findings may inform universities, teaching hospitals, and students how to better shape the first phase of the CEY.

Guidelines: The do's, don'ts and don't knows of direct observation of clinical skills in medical education

INTRODUCTION

Direct observation of clinical skills is a key assessment strategy in competency-based medical education. The guidelines presented in this paper synthesize the literature on direct observation of clinical skills. The goal is to provide a practical list of Do's, Don'ts and Don't Knows about direct observation for supervisors who teach learners in the clinical setting and for educational leaders who are responsible for clinical training programs.

METHODS

We built consensus through an iterative approach in which each author, based on their medical education and research knowledge and expertise, independently developed a list of Do's, Don'ts, and Don't Knows about direct observation of clinical skills. Lists were compiled, discussed and revised. We then sought and compiled evidence to support each guideline and determine the strength of each guideline.

RESULTS

A final set of 33 Do's, Don'ts and Don't Knows is presented along with a summary of evidence for each guideline. Guidelines focus on two groups: individual supervisors and the educational leaders responsible for clinical training programs. Guidelines address recommendations for how to focus direct observation, select an assessment tool, promote high quality assessments, conduct rater training, and create a learning culture conducive to direct observation.

CONCLUSIONS

High frequency, high quality direct observation of clinical skills can be challenging. These guidelines offer important evidence-based Do's and Don'ts that can help improve the frequency and quality of direct observation. Improving direct observation requires focus not just on individual supervisors and their learners, but also on the organizations and cultures in which they work and train. Additional research to address the Don't Knows can help educators realize the full potential of direct observation in competency-based education.

Feedback for Learners in Medical Education: What Is Known? A Scoping Review

PURPOSE

To conduct a scoping review of the literature on feedback for learners in medical education.

METHOD

In 2015-2016, the authors searched the Ovid MEDLINE, ERIC, CINAHL, ProQuest Dissertations and Theses Global, Web of Science, and Scopus databases and seven medical education journals (via OvidSP) for articles published January 1980-December 2015. Two reviewers screened articles for eligibility with inclusion criteria. All authors extracted key data and analyzed data descriptively.

RESULTS

The authors included 650 articles in the review. More than half (n = 341) were published during 2010-2015. Many centered on medical students (n = 274) or residents (n = 192); some included learners from other disciplines (n = 57). Most (n = 633) described methods used for giving feedback; some (n = 95) described opinions and recommendations regarding feedback. Few studies assessed approaches to feedback with randomized, educational trials (n = 49) or described changes in learner behavior after feedback (n = 49). Even fewer assessed the impact of feedback on patient outcomes (n = 28).

CONCLUSIONS

Feedback is considered an important means of improving learner performance, as evidenced by the number of articles outlining recommendations for feedback approaches. The literature on feedback for learners in medical education is broad, fairly recent, and generally describes new or altered curricular approaches that involve feedback for learners. High-quality, evidence-based recommendations for feedback are lacking. In addition to highlighting calls to reassess the concepts and complex nature of feedback interactions, the authors identify several areas that require further investigation.

Am I ready for it? Students' perceptions of meaningful feedback on entrustable professional activities

BACKGROUND

Receiving feedback while in the clinical workplace is probably the most frequently voiced desire of students. In clinical learning environments, providing and seeking performance-relevant information is often difficult for both supervisors and students. The use of entrustable professional activities (EPAs) can help to improve student assessment within competency-based education. This study aimed to illustrate what students' perceptions are of meaningful feedback viewed as conducive in preparing for performing EPA unsupervised.

METHODS

In a qualitative multicentre study we explored students' perceptions on meaningful feedback related to EPAs in the clinical workplace. Focus groups were conducted in three different healthcare institutes. Based on concepts from the literature, the transcripts were coded, iteratively reduced and displayed.

RESULTS

Participants' preferences regarding meaningful feedback on EPAs were quite similar, irrespective of their institution or type of clerkship. Participants explicitly mentioned that feedback on EPAs could come from a variety of sources. Feedback must come from a credible, trustworthy supervisor who knows the student well, be delivered in a safe environment and stress both strengths and points for improvement. The feedback should be provided immediately after the observed activity and include instructions for follow-up. Students would appreciate feedback that refers to their ability to act unsupervised.

CONCLUSION

There is abundant literature on how feedback should be provided, and what factors influence how feedback is sought by students. This study showed that students who are training to perform an EPA unsupervised have clear ideas about how, when and from whom feedback should be delivered.

Twelve tips for the implementation of EPAs for assessment and entrustment decisions

The concept of entrustable professional activities (EPAs) reframes the approach to assessment in competency-based medical education. Key to this concept is the linking of assessment to decision making about entrusting learners with clinical responsibilities. Based on recent literature and the authors' experiences with implementing EPAs, this article provides practical recommendations for how to implement EPAs for assessment and entrustment decisions in the workplace. Tips for supervising clinicians include talking to learners about trust, using EPA descriptions to guide learning and teaching, providing learners with greater ad hoc responsibilities, using EPAs to identify/create opportunities for assessment and feedback, including case-based discussions and acknowledging gut feelings about learner readiness for more autonomy. Tips for curriculum leaders entail enabling the trust development, applying trust decisions at all levels of the supervision scale, employing all available information sources for entrustment, empowering learner ownership of the assessment process and using technology for learner tracking and program evaluation.

"It's Just Not the Culture": A Qualitative Study Exploring Residents' Perceptions of the Impact of Institutional Culture on Feedback

Phenomenon: Competency-based medical education requires ongoing performance-based feedback for professional growth. In several studies, medical trainees report that the quality of faculty feedback is inadequate. Sociocultural barriers to feedback exchanges are further amplified in graduate and postgraduate medical education settings, where trainees serve as frontline providers of patient care. Factors that affect institutional feedback culture, enhance feedback seeking, acceptance, and bidirectional feedback warrant further exploration in these settings.

APPROACH

Using a constructivist grounded theory approach, we sought to examine residents' perspectives on institutional factors that affect the quality of feedback, factors that influence receptivity to feedback, and quality and impact of faculty feedback. Four focus group discussions were conducted, with two investigators present at each. One facilitated the discussion, and the other observed the interactions and took field notes. We audiotaped and transcribed the discussions, and performed a thematic analysis. Measures to ensure rigor included thick descriptions, independent coding by two investigators, and attention to reflexivity.

FINDINGS

We identified five key themes, dominated by resident perceptions regarding the influence of institutional feedback culture. The theme labels are taken from direct participant quotes: (a) the cultural norm lacks clear expectations and messages around feedback, (b) the prevailing culture of niceness does not facilitate honest feedback, (c) bidirectional feedback is not part of the culture, (d) faculty-resident relationships impact credibility and receptivity to feedback, and (e) there is a need to establish a culture of longitudinal professional growth. Insights: Institutional culture could play a key role in influencing the quality, credibility, and acceptability of feedback. A polite culture promotes a positive learning environment but can be a barrier to honest feedback. Feedback initiatives focusing solely on techniques of feedback giving may not enhance meaningful feedback. Further research on factors that promote feedback seeking, receptivity to constructive feedback, and bidirectional feedback would provide valuable insights.

'Sometimes the work just needs to be done': socio-cultural influences on direct observation in medical training

CONTEXT

Direct observation promises to strengthen both coaching and assessment, and calls for its increased use in medical training abound. Despite its apparent potential, the uptake of direct observation in medical training remains surprisingly limited outside the formal assessment setting. The limited uptake of observation raises questions about cultural barriers to its use. In this study, we explore the influence of professional culture on the use of direct observation within medical training.

METHODS

Using a constructivist grounded theory approach, we interviewed 22 residents or fellows (10 male, 12 female) about their experiences of being observed during training. Participants represented a range of specialties and training levels. Data collection and analysis were conducted iteratively. Themes were identified using constant comparative analysis.

RESULTS

Observation was used selectively; specialties tended to observe the clinical acts that they valued most. Despite these differences, we found two cultural values that consistently challenged the ready implementation of direct observation across specialties: (i) autonomy in learning and (ii) efficiency in health care provision. Furthermore, we found that direct observation was a primarily learner-driven activity, which left learners caught in the middle, wanting observation but also wanting to appear independent and efficient.

CONCLUSIONS

The cultural values of autonomy in learning and practice and efficiency in health care provision challenge the integration of direct observation into clinical training. Medical learners are often expected to ask for observation, but such requests are socially and culturally fraught, and likely to constrain the wider uptake of direct observation.

Nursing Students' Perceptions of Anecdotal Notes as Formative Feedback

Anecdotal notes are a method of providing formative feedback to nursing students following clinical experiences. The extant literature on anecdotal notes is written only from the educator perspective, focusing on rationale for and methods of production, rather than on evaluation of effectiveness. A retrospective descriptive study was carried out with a cohort of 283 third year baccalaureate nursing students to explore their perceptions of anecdotal notes as effective formative feedback. The majority of students valued verbal as well as anecdotal note feedback. They preferred to receive feedback before the next learning experience. Students found the quality of feedback varied by instructor. The anecdotal note process was found to meet identified formative feedback requirements as well as the nursing program's requirement for transparency of evaluation and due process. It is necessary to provide professional development to clinical nurse educators to assist them develop high quality formative feedback using anecdotal notes.

Profiling postgraduate workplace-based assessment implementation in Ireland: a retrospective cohort study

In 2010, workplace-based assessment (WBA) was formally integrated as a method of formative trainee assessment into 29 basic and higher specialist medical training (BST/HST) programmes in six postgraduate training bodies in Ireland. The aim of this study is to explore how WBA is being implemented and to examine if WBA is being used formatively as originally intended. A retrospective cohort study was conducted and approved by the institution’s Research Ethics Committee. A profile of WBA requirements was obtained from 29 training programme curricula. A data extraction tool was developed to extract anonymous data, including written feedback and timing of assessments, from Year 1 and 2 trainee ePortfolios in 2012–2013. Data were independently quality assessed and compared to the reference standard number of assessments mandated annually where relevant. All 29 training programmes mandated the inclusion of at least one case-based discussion (max = 5; range 1–5). All except two non-clinical programmes (93 %) required at least two mini-Clinical Evaluation Exercise assessments per year and Direct Observation of Procedural Skills assessments were mandated in 27 training programmes over the course of the programme. WBA data were extracted from 50 % of randomly selected BST ePortfolios in four programmes (n = 142) and 70 % of HST ePortfolios (n = 115) in 21 programmes registered for 2012–2013. Four programmes did not have an eligible trainee for that academic year. In total, 1142 WBAs were analysed. A total of 164 trainees (63.8 %) had completed at least one WBA. The average number of WBAs completed by HST trainees was 7.75 (SD 5.8; 95 % CI 6.5–8.9; range 1–34). BST trainees completed an average of 6.1 assessments (SD 9.3; 95 % CI 4.01–8.19; range 1–76). Feedback—of varied length and quality—was provided on 44.9 % of assessments. The majority of WBAs were completed in the second half of the year. There is significant heterogeneity with respect to the frequency and quality of feedback provided during WBAs. The completion of WBAs later in the year may limit available time for feedback, performance improvement and re-evaluation. This study sets the scene for further work to explore the value of formative assessment in postgraduate medical education.

Guidelines: the do's, don'ts and don't knows of feedback for clinical education

INTRODUCTION

The guidelines offered in this paper aim to amalgamate the literature on formative feedback into practical Do's, Don'ts and Don't Knows for individual clinical supervisors and for the institutions that support clinical learning.

METHODS

The authors built consensus by an iterative process. Do's and Don'ts were proposed based on authors' individual teaching experience and awareness of the literature, and the amalgamated set of guidelines were then refined by all authors and the evidence was summarized for each guideline. Don't Knows were identified as being important questions to this international group of educators which if answered would change practice. The criteria for inclusion of evidence for these guidelines were not those of a systematic review, so indicators of strength of these recommendations were developed which combine the evidence with the authors' consensus.

RESULTS

A set of 32 Do and Don't guidelines with the important Don't Knows was compiled along with a summary of the evidence for each. These are divided into guidelines for the individual clinical supervisor giving feedback to their trainee (recommendations about both the process and the content of feedback) and guidelines for the learning culture (what elements of learning culture support the exchange of meaningful feedback, and what elements constrain it?)

CONCLUSION

Feedback is not easy to get right, but it is essential to learning in medicine, and there is a wealth of evidence supporting the Do's and warning against the Don'ts. Further research into the critical Don't Knows of feedback is required. A new definition is offered: Helpful feedback is a supportive conversation that clarifies the trainee's awareness of their developing competencies, enhances their self-efficacy for making progress, challenges them to set objectives for improvement, and facilitates their development of strategies to enable that improvement to occur.

Competency-based veterinary education: an integrative approach to learning and assessment in the clinical workplace

When graduating from veterinary school, veterinary professionals must be ready to enter the complex veterinary profession. Therefore, one of the major responsibilities of any veterinary school is to develop training programmes that support students' competency development on the trajectory from novice student to veterinary professional. The integration of learning and assessment in the clinical workplace to foster this competency development in undergraduate veterinary education was the central topic of this thesis.