What attributes guide best practice for effective feedback? A scoping review

Designing feedback processes in the workplace-based learning of undergraduate health professions education: a scoping review

BACKGROUND

Feedback processes are crucial for learning, guiding improvement, and enhancing performance. In workplace-based learning settings, diverse teaching and assessment activities are advocated to be designed and implemented, generating feedback that students use, with proper guidance, to close the gap between current and desired performance levels. Since productive feedback processes rely on observed information regarding a student's performance, it is imperative to establish structured feedback activities within undergraduate workplace-based learning settings. However, these settings are characterized by their unpredictable nature, which can either promote learning or present challenges in offering structured learning opportunities for students. This scoping review maps literature on how feedback processes are organised in undergraduate clinical workplace-based learning settings, providing insight into the design and use of feedback.

METHODS

A scoping review was conducted. Studies were identified from seven databases and ten relevant journals in medical education. The screening process was performed independently in duplicate with the support of the StArt program. Data were organized in a data chart and analyzed using thematic analysis. The feedback loop with a sociocultural perspective was used as a theoretical framework.

RESULTS

The search yielded 4,877 papers, and 61 were included in the review. Two themes were identified in the qualitative analysis: (1) The organization of the feedback processes in workplace-based learning settings, and (2) Sociocultural factors influencing the organization of feedback processes. The literature describes multiple teaching and assessment activities that generate feedback information. Most papers described experiences and perceptions of diverse teaching and assessment feedback activities. Few studies described how feedback processes improve performance. Sociocultural factors such as establishing a feedback culture, enabling stable and trustworthy relationships, and enhancing student feedback agency are crucial for productive feedback processes.

CONCLUSIONS

This review identified concrete ideas regarding how feedback could be organized within the clinical workplace to promote feedback processes. The feedback encounter should be organized to allow follow-up of the feedback, i.e., working on required learning and performance goals at the next occasion. The educational programs should design feedback processes by appropriately planning subsequent tasks and activities. More insight is needed in designing a full-loop feedback process, in which specific attention is needed in effective feedforward practices.

Evaluating governance in a clinical and translational research organization

Institutional Development Awards for Clinical and Translational Research (IDeA-CTR) networks, funded by NIH/NIGMS, aim to advance CTR infrastructure to address historically unmet state and regional health needs. Success depends on the response to actionable feedback to IDeA-CTR leadership from network partners and governance groups through annual surveys, interviews, and governance body recommendations. The Great Plains IDeA-CTR applied internal formative meta-evaluation to evaluate dispositions of 172 governance recommendations from 2017 to 2021. Results provided insights to improve the classification and quality of recommendations, credibility of evaluation processes, responsiveness to recommendations, and communications and governance in a complex CTR network comprising multiple coalitions.

The quality of verbal feedback given by nursing near-peer tutors: A qualitative study

BACKGROUND

Students' acceptance of peer feedback (PF) in health professions education has been mixed because of doubts about its quality and accuracy. Providing peer tutor (PT) training in giving effective feedback may increase the quality of PF.

OBJECTIVES

To analyse the (1) quality and (2) accuracy of near-PF provided to first-year nursing students after simulated practice and (3) evaluate the discrepancies between what faculty, PTs and clinical staff teach about certain nursing skills by analysing PT-student feedback discussions.

DESIGN

Qualitative study using content analysis.

SETTINGS

This study was conducted in a simulation centre in a Singapore university during first-year undergraduate nursing students' laboratory lessons.

PARTICIPANTS

Near-PT (second- to fourth-year undergraduate nursing students).

METHODS

Near-PTs received virtual training on providing structured feedback. They provided in-person PF to first-year nursing students after simulated practice of a technical nursing skill. PF was audio recorded. Quality was determined by PTs' adherence to the proposed PF structure and the Debriefing Assessment for Simulation in Healthcare-Rater Version Short Form (DASH-RV-Short) scores. Directed content analyses were also conducted to qualitatively evaluate PF quality based on DASH-RV-Short, PF accuracy and discrepancies in content taught for each nursing skill.

RESULTS

Most PTs evaluated the skill chronologically or focused on the mistakes made instead of following the PF structure. DASH-RV-Short scores were 'good' for most elements because PTs gave specific suggestions and justified them, ensured students' psychological safety, and used effective verbal communication. PF was mostly accurate, but expectations relating to each nursing skill differed among the PTs, faculty and clinical staff.

CONCLUSIONS

With training, senior nursing students can provide specific and detailed feedback to their juniors, which may fill the gap for timely and specific feedback in health professions education. Future PT training programmes should clarify common variations and mistakes in skills performance to ensure PTs and faculty tutors agree on what students should learn.

What does "Timely" Mean to Residents? Challenging Feedback Assumptions in Postgraduate Education

Introduction

Current orthodoxy states that feedback should be timely and face-to-face, yet the optimal timing and mode of delivery for feedback is unclear. We explored what "optimal timing" means from residents' points of view as feedback providers and receivers, to ultimately inform strategies to optimize feedback in training.

Methods

As near-peers who have dual roles in both providing and receiving feedback, 16 subspecialty (PGY4 and 5) internal medicine residents were interviewed about their perceptions of the optimal timing and format of feedback. Using constructivist grounded theory, interviews were conducted and analyzed iteratively.

Results

Drawing on their experiences as providers and recipients, residents described simultaneously considering and weighing multiple factors when deciding on when and how to provide feedback. These included their own readiness to engage in providing meaningful feedback, the perceived receptiveness of the learner and the apparent urgency of feedback delivery (e.g., if patient safety was at stake). Face-to-face verbal feedback was valued for encouraging dialogue but could be uncomfortable and limited by time constraints. Written feedback could be more honest and concise, and the possibility of asynchronous delivery had potential to overcome issues with timing and discomfort.

Discussion

Participants' perceptions of the optimal timing of feedback challenge current assumptions about the benefits of "immediate" versus "delayed". The concept of "optimal timing" for feedback was found to be complex and context-dependent, defying a formulaic approach. There may be a role for asynchronous and/or written feedback, which has potential to address unique issues identified issues in near-peer relationships.

Measuring changes in nursing students' workplace performance following feedback encounters: A quasi-experimental study

BACKGROUND

Work-integrated learning and performance is intimately linked with assessment and feedback; however, empirical evidence concerning the impact of feedback on student performance in workplace settings is limited. A novel practice-based feedback intervention-the REMARK (nuRse fEedback iMplementAtion frameworRK) program-was developed and implemented. Sustainable feedback processes that involve both learners and learning partners can ultimately maximise learning outcomes, optimise self-regulation, and impact performance.

OBJECTIVES

To determine the impact of implementing an educational intervention based on best practice principles of feedback on undergraduate nursing student performance during clinical placements in acute healthcare settings.

METHODS

A non-equivalent, quasi-experimental design was used. A pre- and post-assessment strategy for both control and intervention groups was employed. The REMARK program was based on known attributes of effective feedback critiqued in the literature that foster interactions between a learner and learning partner and, hence, support dialogic feedback.

RESULTS

Clinical assessment data of 214 final-year nursing students collected at three time points were analysed. Using multivariate modelling, the results indicated that students participating in the REMARK programme (intervention group) had statistically higher performance scores than students in the control group when time and placement setting were controlled.

CONCLUSIONS

This research provides evidence that engagement with an intervention based on attributes of effective feedback improves nursing students' workplace performance during clinical placement. To achieve the greatest advantage, incorporating strategies that encourage students to adopt an active role in feedback conversations that relate to the student's learning goals are recommended.

An analysis of glaucoma repeat measures assessment results: Are core competencies enough?

PURPOSE

The need to validate core competency skills in qualified optometrists wishing to take part in extended roles in glaucoma care has been questioned. This analysis examines the ability of qualified optometrists to perform relevant core competency skills under standardised objective assessment conditions to explore whether such validation is justified. It also investigates if there are associations between performance, gender and length of time since qualification.

METHODS

Anonymised data from the Cardiff University assessment programme for the Wales Optometry Postgraduate Education Centre (WOPEC) Local Optical Committee Support Unit glaucoma referral filtering and monitoring pathway delivered between January 2017 and March 2020 were analysed. Results were combined with demographic data from the General Optical Council register of optometrists in the UK to investigate associations between performance and practitioner characteristics, namely length of time since qualification and gender.

RESULTS

The assessment results of 2215 optometrists practising in England (approximately 15% of all UK registered optometrists and 30% of all optometrists registered in England) were analysed. Failure rates for first time assessment in each of five objective structured clinical examination style practical assessments were 8.5% (van Herick); 8.8% (slit lamp binocular indirect ophthalmoscopy); 10.1% (Goldmann applanation tonometry calibration); 21.9% (Goldmann applanation tonometry) and 23.3% (case scenario interpretation and management). There were either no associations or at most very weak associations between performance and practitioner characteristics.

CONCLUSIONS

Our results suggest that these competencies are not universally present in optometrists practising in England and that ongoing training and assessment of these competencies is justified for entry into extended roles. There are no meaningful associations between performance in these assessments and gender or time since qualification.

[Contribution and limits of "OSCE", "long-case" and "global end-of-placement marking" as end-of-rotation assessment methods. Experience from two internal medicine wards]

INTRODUCTION

During placements, there is an opportunity to learn clinical skills and to assess their application. However, it represents two different goals. The validity of an end-of-placement assessment is questionable, as the medical competency is contextual. We decided to evaluate the contribution and limits of different assessment modalities as an end-of-placement assessment.

MATERIAL AND METHODS

Internal medicine clerks were assessed using the Mini-Cex grid by a structured objective clinical examination (OSCE), a long-case clinical examination (LCE) and a global end-of-placement marking (GEPM). Following these evaluations, students and teachers fulfilled an open questionnaire.

RESULTS

In 2021, 41 students and 16 teachers participated in the study. Physical examination was evaluated in 0%, 97% et 76% of cases during OSCE, LCE and GEPM, respectively; teaching skills were assessed for 100, 42 et 49% of students in OSCE, LCE and GEPM, respectively. As compared to OSCE, there was a perceived superiority of LCE regarding its formative value (P=0.07 and P=0.03) and its summative value (P=0.0007 and P=0.02), for students and teachers, respectively. Qualitative analysis highlights the breadth of clinical skills that could be assessed during OSCE stations. Integration into a team was an additional skill that could specifically be assessed during GEPM. GEPM could also take into account the progress made during placement.

CONCLUSION

Despite its subjectivity, LCE seemed to be the preferred modality for an end-of-rotation assessment.

Consensus marking as a grading method for the development of evaluative judgement: Comparing assessor and students

AIM

This study explored postgraduate nursing students' perceptions, anxiety and satisfaction of an innovative and novel grading method for online vivas, consensus marking, compared with traditional assessor judgement.

BACKGROUND

Reflection, self-evaluation and feedback conversations have the potential to develop nursing students' evaluative judgement. Consensus marking is a novel method of grading students' performance that supports students to reflect, self-evaluate and grade their own work. Active engagement in a feedback dialogue supports students to calibrate their self-evaluation to the required standard in a grade negotiation. Through this approach, students are supported to develop evaluative judgement and lifelong learning skills.

DESIGN

A convergent mixed-methods parallel research design was used.

METHODS

Students enrolled in a postgraduate emergency nursing unit of study completed two online viva assessments. One viva was graded using traditional assessor judgement and the other used consensus marking, involving a two-way feedback dialogue, where students had an opportunity to actively engage in grading their own work with the assessor. Student perceptions of each grading method were explored through semi-structured interviews. Interview data were analysed thematically using a six-stage approach. Student anxiety and satisfaction were measured pre- and post each viva using valid and reliable questionnaires. Non-parametric analyses explored differences in anxiety and satisfaction between the two grading methods. Alpha was set at 0.05.

RESULTS

Forty-six participants had complete data for anxiety and satisfaction across both test occasions (82%) and were included in the analysis. Of these, 13 students participated in follow up interviews. Students perceived that the ability to self-evaluate performance and discuss their grade with the assessor using consensus marking was less hierarchical and similar to a collegial debrief. Student anxiety was significantly lower prior to consensus marking compared with the assessor judged viva (p < 0.001). Students were significantly more satisfied with consensus marking compared with assessor judgement (p < 0.01).

CONCLUSIONS

Consensus marking created an opportunity for students to identify knowledge deficits through reflection and self-evaluation of their own performance prior to external judgement. Students were more satisfied and less anxious with the consensus marking grading method compared with traditional assessor judgement. These findings have implications for the development and application of new grading methods in nursing education to facilitate the development of evaluative judgement.

Scoping reviews in medical education: A scoping review

OBJECTIVES

Over the last two decades, the number of scoping reviews in core medical education journals has increased by 4200%. Despite this growth, research on scoping reviews provides limited information about their nature, including how they are conducted or why medical educators undertake this knowledge synthesis type. This gap makes it difficult to know where the field stands and may hamper attempts to improve the conduct, reporting and utility of scoping reviews. Thus, this review characterises the nature of medical education scoping reviews to identify areas for improvement and highlight future research opportunities.

METHOD

The authors searched PubMed for scoping reviews published between 1/1999 and 4/2020 in 14 medical education journals. The authors extracted and summarised key bibliometric data, the rationales given for conducting a scoping review, the research questions and key reporting elements as described in the PRISMA-ScR. Rationales and research questions were mapped to Arksey and O'Malley's reasons for conducting a scoping review.

RESULTS

One hundred and one scoping reviews were included. On average, 10.1 scoping reviews (SD = 13.1, median = 4) were published annually with the most reviews published in 2019 (n = 42). Authors described multiple reasons for undertaking scoping reviews; the most prevalent being to summarise and disseminate research findings (n = 77). In 11 reviews, the rationales for the scoping review and the research questions aligned. No review addressed all elements of the PRISMA-ScR, with few authors publishing a protocol (n = 2) or including stakeholders (n = 20). Authors identified shortcomings of scoping reviews, including lack of critical appraisal.

CONCLUSIONS

Scoping reviews are increasingly conducted in medical education and published by most core journals. Scoping reviews aim to map the depth and breadth of emerging topics; as such, they have the potential to play a critical role in the practice, policy and research of medical education. However, these results suggest improvements are needed for this role to be fully realised.

Performance assessment: Consensus statement and recommendations from the 2020 Ottawa Conference

INTRODUCTION

In 2011 the Consensus Statement on Performance Assessment was published in Medical Teacher. That paper was commissioned by AMEE (Association for Medical Education in Europe) as part of the series of Consensus Statements following the 2010 Ottawa Conference. In 2019, it was recommended that a working group be reconvened to review and consider developments in performance assessment since the 2011 publication.

METHODS

Following review of the original recommendations in the 2011 paper and shifts in the field across the past 10 years, the group identified areas of consensus and yet to be resolved issues for performance assessment.

RESULTS AND DISCUSSION

This paper addresses developments in performance assessment since 2011, reiterates relevant aspects of the 2011 paper, and summarises contemporary best practice recommendations for OSCEs and WBAs, fit-for-purpose methods for performance assessment in the health professions.

Effect of Detailed OSCE Score Reporting on Learning and Anxiety in Medical School

INTRODUCTION

There is growing literature on increasing feedback from Objective Structured Clinical Examinations (OSCEs) and one approach is a score report. The purpose of this study was to implement and evaluate a score report for a second and fourth-year medical school OSCE.

METHODS

We developed an electronic OSCE score report that displayed comments and performance by domain within and across stations (checklist items and rating scales were tagged to each domain). Our initial pilot released the score report after pass/fail decisions but subsequent iterations released the score report the same day as the exam. Our evaluation approach included both student surveys and focus groups.

RESULTS

Students felt the OSCE score report was accurate, identified strengths and weaknesses, and would likely cause them to take future action, with second-year students more likely to act on the report than fourth year students. The thematic analysis revealed barriers and enablers to utilizing feedback as well as the power of the score report to reduce anxiety.

CONCLUSIONS

Our OSCE score report was simple to develop and implement the same day as an OSCE with an overall positive response from students with respect to accuracy and ability to use the information for future learning.

The use of factor analysis and abductive inference to explore students' and practitioners' perspectives of feedback: divergent or congruent understanding?

BACKGROUND

The importance of feedback in workplace-based settings cannot be underestimated. Approaches that evaluate feedback reflect either the sender's or receiver's viewpoint in isolation of each other. This study investigated prevailing student and practitioner views of feedback resulting from development and testing of a survey about feedback.

METHOD

This study used a cross-sectional design, incorporating use of expert consultation and factor analysis of surveys. Fifty-two items based on identified attributes for effective feedback from current research were developed and reviewed through expert consultation. Surveys developed from the items were completed by students (n = 209) and practitioners (n = 145). The juxtaposition of items based on students' and practitioners' responses to the surveys were examined through use of exploratory factor analysis.

RESULTS

Separate student and practitioner surveys resulted. Each survey contained 23 items that clustered into factors. The item statements were different across practitioner and student groups Only nine items were shared across factors identified for both groups. The resulting factors represented different notions of feedback-namely, practitioners had a process-oriented focus in comparison with students' outcome focus.

CONCLUSION

While students and practitioners view feedback differently this does not necessarily mean they are incongruous.

Role of Technology in Self-Assessment and Feedback Among Hospitalist Physicians: Semistructured Interviews and Thematic Analysis

Background

Lifelong learning is embedded in the culture of medicine, but there are limited tools currently available for many clinicians, including hospitalists, to help improve their own practice. Although there are requirements for continuing medical education, resources for learning new clinical guidelines, and developing fields aimed at facilitating peer-to-peer feedback, there is a gap in the availability of tools that enable clinicians to learn based on their own patients and clinical decisions.

Objective

The aim of this study was to explore the technologies or modifications to existing systems that could be used to benefit hospitalist physicians in pursuing self-assessment and improvement by understanding physicians’ current practices and their reactions to proposed possibilities.

Methods

Semistructured interviews were conducted in two separate stages with analysis performed after each stage. In the first stage, interviews (N=12) were conducted to understand the ways in which hospitalist physicians are currently gathering feedback and assessing their practice. A thematic analysis of these interviews informed the prototype used to elicit responses in the second stage.

Results

Clinicians actively look for feedback that they can apply to their practice, with the majority of the feedback obtained through self-assessment. The following three themes surrounding this aspect were identified in the first round of semistructured interviews: collaboration, self-reliance, and uncertainty, each with three related subthemes. Using a wireframe, the second round of interviews led to identifying the features that are currently challenging to use or could be made available with technology.

Conclusions

Based on each theme and subtheme, we provide targeted recommendations for use by relevant stakeholders such as institutions, clinicians, and technologists. Most hospitalist self-assessments occur on a rolling basis, specifically using data in electronic medical records as their primary source. Specific objective data points or subjective patient relationships lead clinicians to review their patient cases and to assess their own performance. However, current systems are not built for these analyses or for clinicians to perform self-assessment, making this a burdensome and incomplete process. Building a platform that focuses on providing and curating the information used for self-assessment could help physicians make more accurately informed changes to their own clinical practice and decision-making.

Scoping reviews in health professions education: challenges, considerations and lessons learned about epistemology and methodology

Scoping reviews are increasingly used in health professions education to synthesize research and scholarship, and to report on the depth and breadth of the literature on a given topic. In this Perspective, we argue that the philosophical stance scholars adopt during the execution of a scoping review, including the meaning they attribute to fundamental concepts such as knowledge and evidence, influences how they gather, analyze, and interpret information obtained from a heterogeneous body of literature. We highlight the principles informing scoping reviews and outline how epistemology-the aspect of philosophy that "deals with questions involving the nature of knowledge, the justification of beliefs, and rationality"-should guide methodological considerations, toward the aim of ensuring the production of a high-quality review with defensible and appropriate conclusions. To contextualize our claims, we illustrate some of the methodological challenges we have personally encountered while executing a scoping review on clinical reasoning and reflect on how these challenges could have been reconciled through a broader understanding of the methodology's philosophical foundation. We conclude with a description of lessons we have learned that might usefully inform other scholars who are considering undertaking a scoping review in their own domains of inquiry.

An optimal environment for placement learning: listening to the voices of speech and language therapy students

BACKGROUND

Placements are a core component of learning within speech and language therapy (SLT) undergraduate and postgraduate university programmes. They facilitate the development of students' professional and clinical competencies and socialization into the profession. It is indicated that an optimal environment for placement learning often pivots on the mentoring, supervision and feedback the student receives.

AIMS

To explore the perspectives of student SLTs in the Republic of Ireland in relation to factors that would best support their learning and competency development on placement.

METHODS & PROCEDURES

Qualitative data were collected from an anonymous online student survey that consisted of eight open-ended questions. Thematic analysis was applied to the data. Excerpts from the data were selected to illustrate the themes constructed.

OUTCOMES & RESULTS

A total of 117 students responded. Four salient themes were generated that capture the students' perspectives of an optimal environment for placement learning and competency development.

CONCLUSIONS & IMPLICATIONS

This study supports quality assurance within the practice education of student SLTs and highlights aspects of an optimal learning environment that practice educators can strive to develop. In parallel, this study points to the need for improved supports from university personnel and placement site managers, and an increased need for student preparedness and self-reflection. Implications for continuing professional development specific to the role of a practice educator is described, in addition to the recommendation of an expanded perspective of supervision within SLT. What this paper adds What is already known on the subject Placements are an integral component of SLT undergraduate and postgraduate university programmes that enable students to translate theory to practice. Placement involves a transition from structured and predictable learning of the classroom to more dynamic learning environment within the placement site. Assessment of placement is carried out by practice educators using competency assessment tools. An optimal environment for placement learning often pivots on the mentoring, supervision and feedback that the student receives from their practice educator. What this paper adds to existing knowledge This study explores student SLTs' voices in relation to what they consider an optimal environment for placement learning should be and what they believe may best support them in their journey to develop their clinical competencies. It supports quality assurance of the practice education of our future colleagues. What are the potential or actual clinical implications of this work? The findings of this enquiry emphasize the distinction between competence as a SLT and competence as a practice educator and have subsequent implications for the content of continuing professional development for practice educators. In particular, an expanded perspective of the models and frameworks of supervision to promote and implement within the practice education of SLT students is presented.

Learning Conversations: An Analysis of the Theoretical Roots and Their Manifestations of Feedback and Debriefing in Medical Education

Feedback and debriefing are experience-informed dialogues upon which experiential models of learning often depend. Efforts to understand each have largely been independent of each other, thus splitting them into potentially problematic and less productive factions. Given their shared purpose of improving future performance, the authors asked whether efforts to understand these dialogues are, for theoretical and pragmatic reasons, best advanced by keeping these concepts unique or whether some unifying conceptual framework could better support educational contributions and advancements in medical education.The authors identified seminal works and foundational concepts to formulate a purposeful review and analysis exploring these dialogues' theoretical roots and their manifestations. They considered conceptual and theoretical details within and across feedback and debriefing literatures and traced developmental paths to discover underlying and foundational conceptual approaches and theoretical similarities and differences.Findings suggest that each of these strategies was derived from distinct theoretical roots, leading to variations in how they have been studied, advanced, and enacted; both now draw on multiple (often similar) educational theories, also positioning themselves as ways of operationalizing similar educational frameworks. Considerable commonality now exists; those studying and advancing feedback and debriefing are leveraging similar cognitive and social theories to refine and structure their approaches. As such, there may be room to merge these educational strategies as learning conversations because of their conceptual and theoretical consistency. Future scholarly work should further delineate the theoretical, educational, and practical relevance of integrating feedback and debriefing.

Impact of a work-based feedback intervention on student performance during clinical placements in acute-care healthcare settings: a quasi-experimental protocol for the REMARK programme

INTRODUCTION

Current perspectives present feedback as a dynamic, dialogic process. It is widely accepted that feedback can have an impact on workplace performance, however, how dialogic feedback is enacted with the learner in authentic healthcare settings is less apparent. This paper seeks to describe the design and development of an implementation study to promote the learner voice in the feedback process and improve feedback encounters between learners and learning partners in healthcare settings.

METHODS AND ANALYSIS

A quasi-experimental study design will be used to evaluate whether implementation of a work-based intervention to improve feedback impacts student performance during clinical placements in healthcare settings. Student performance will be measured at three time points: baseline (pre), mid-placement (post-test 1) and end-placement (post-test 2) in keeping with standard assessment processes of the participating university. The intervention is underpinned by Normalisation Process Theory and involves a layered design that targets learners and learning partners using best-practice education strategies. Data regarding participants' engagement with feedback during clinical placements and participants' level of adoption of the intervention will be collected at the completion of the clinical placement period.

ETHICS AND DISSEMINATION

This study has ethics approval from both Griffith University and Metro South Health Human Research and Ethics committees. Dissemination of results will be local, national and international through forums, seminars, conferences and publications.