Exploring the Role of Self-Motives in Postgraduate Trainees' Feedback-Seeking Behavior in the Clinical Workplace: A Multicenter Study of Workplace-Based Assessments From the United Kingdom

Surgical Residents' Perception of Feedback on Their Education: Protocol for a Scoping Review

BACKGROUND

Feedback is an essential tool for learning and improving performance in any sphere of education, including training of resident physicians. The learner's perception of the feedback they receive is extremely relevant to their learning progress, which must aim at providing qualified care for patients. Studies pertinent to the matter differ substantially with respect to methodology, population, context, and objective, which makes it even more difficult to achieve a clear understanding of the topic. A scoping review on this theme will unequivocally enhance and organize what is already known.

OBJECTIVE

The aim of this study is to identify and map out data from studies that report surgical residents' perception of the feedback received during their education.

METHODS

The review will consider studies on the feedback perception of resident physicians of any surgical specialty and age group, attending any year of residency, regardless of the type of feedback given and the way the perceptions were measured. Primary studies published in English, Spanish, and Portuguese since 2017 will be considered. The search will be carried out in 6 databases and reference lists will also be searched for additional studies. Duplicates will be removed, and 2 independent reviewers will screen the selected studies' titles, abstracts, and full texts. Data extraction will be performed through a tool developed by the researchers. Descriptive statistics and qualitative analysis (content analysis) will be used to analyze the data. A summary of the results will be presented in the form of diagrams, narratives, and tables.

RESULTS

The findings of this scoping review were submitted to an indexed journal in July 2024, currently awaiting reviewer approval. The search was executed on March 15, 2024, and resulted in 588 articles. After the exclusion of the duplicate articles and those that did not meet the eligibility criteria as well as the inclusion of articles through a manual search, 13 articles were included in the review.

CONCLUSIONS

Conducting a scoping review is the best way to map what is known about a subject. By focusing on the feedback perception more than the feedback itself, the results of this study will surely contribute to gaining a deeper understanding of how to proceed to enhance internal feedback and surgical residents' learning progress.

TRIAL REGISTRATION

Open Science Framework yexb; https://osf.io/yexkb.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/56727.

Translation and psychometric evaluation of composite feedback-seeking behavior questionnaire among Iranian medical residents

BACKGROUND

Proactively seeking feedback from clinical supervisors, peers or other healthcare professionals is a valuable mechanism for residents to obtain useful information about and improve their performance in clinical settings. Given the scant studies investigating the limited aspects of psychometrics properties of the feedback-seeking instruments in medical education, this study aimed to translate the feedback-seeking behavior scales (frequency of feedback-seeking, motives of feedback-seeking, and promotion of feedback-seeking by supervisors) into Persian and evaluate the psychometric properties of the composite questionnaire among medical residents at Tehran University of Medical Sciences in Iran.

METHODS

In this cross-sectional study, feedback-seeking behavior scales were translated through the forward-backward method, and its face validity and content validity were assessed by 10 medical residents and 18 experts. The test-retest reliability was evaluated by administering the questionnaire to 20 medical residents on two testing occasions. A convenience sample of 548 residents completed the questionnaire. Construct validity was examined by exploratory factor analysis and confirmatory factor analysis and concurrent validity was determined by Pearson's correlation coefficient.

RESULTS

Content validity assessment showed that the CVR (0.66 to 0.99) and CVI (0.82 to 0.99) values for items and S-CVI values (0.88 to 0.99) for scales were satisfactory. The exploratory and confirmatory factor analysis demonstrated that the models were confirmed with eight items and two factors (explaining 70.98% of the total variance) for the frequency of feedback-seeking scale, with 16 items and four factors (explaining 73.22% of the total variance) for the motives of feedback seeking scale and with four items and one factor (explaining 69.46% of the total variance) for promotion of feedback-seeking by supervisors. AVE values greater than 0.5 and discriminant validity correlations significantly less than 1.0 demonstrated that the total scores of the composite feedback-seeking behavior questionnaire had a favorable fit and the questions could fit their respective factors, and the latent variables were distinct. We found positive and significant correlations between the three scales and their subscales.

CONCLUSION

The results of the present study supported the validity and reliability of the Persian composite feedback-seeking behavior questionnaire for assessing feedback-seeking behaviors in medical residents. Applying the questionnaire in residency programs may enhance the quality of clinical education.

Making assessment a team sport: a qualitative study of facilitated group feedback in internal medicine residency

Purpose

Competency-based medical education relies on feedback from workplace-based assessment (WBA) to direct learning. Unfortunately, WBAs often lack rich narrative feedback and show bias towards Medical Expert aspects of care. Building on research examining interactive assessment approaches, the Queen's University Internal Medicine residency program introduced a facilitated, team-based assessment initiative ("Feedback Fridays") in July 2017, aimed at improving holistic assessment of resident performance on the inpatient medicine teaching units. In this study, we aim to explore how Feedback Fridays contributed to formative assessment of Internal Medicine residents within our current model of competency-based training.

Method

A total of 53 residents participated in facilitated, biweekly group assessment sessions during the 2017 and 2018 academic year. Each session was a 30-minute facilitated assessment discussion done with one inpatient team, which included medical students, residents, and their supervising attending. Feedback from the discussion was collected, summarized, and documented in narrative form in electronic WBA forms by the program's assessment officer for the residents. For research purposes, verbatim transcripts of feedback sessions were analyzed thematically.

Results

The researchers identified four major themes for feedback: communication, intra- and inter-personal awareness, leadership and teamwork, and learning opportunities. Although feedback related to a broad range of activities, it showed strong emphasis on competencies within the intrinsic CanMEDS roles. Additionally, a clear formative focus in the feedback was another important finding.

Conclusions

The introduction of facilitated team-based assessment in the Queen's Internal Medicine program filled an important gap in WBA by providing learners with detailed feedback across all CanMEDS roles and by providing constructive recommendations for identified areas for improvement.

Utilisation of in-consultation supervisor assistance in general practice training and personal cost to trainees: a cross-sectional study

Aim The aim of the study was to establish whether two previously described barriers to effective in-consultation assistance-seeking by general practice (GP) vocational specialist trainees (ie concern about patient impressions of their competence, and discomfort presenting to supervisors in front of patients) influenced the frequency of trainee in-consultation assistance-seeking from their supervisor. Methods This was a cross-sectional study nested in the Registrar Clinical Encounters in Clinical Training ongoing cohort study of Australian GP trainees. Trainee participants completed contemporaneous records of 60 consecutive patient consultations, including whether supervisory assistance was sought. Trainees also completed a cross-sectional survey including items eliciting their beliefs about patient impressions and their own discomfort in seeking in-consultation supervisory assistance. These were factors of interest in multivariable logistic regression analyses; the outcome factor in both regression models was the seeking of in-consultation supervisory assistance. Results In 2018, 778 trainees (778/876, response rate 89%) completed the cross-sectional survey. No association was found between the odds of in-consultation help-seeking and perceived decrease in patient impressions of trainee competence (OR = 1.09; 95% CI: 0.91, 1.31; P  = 0.36) or higher comfort presenting outside patients' hearing (OR = 0.9; 95% CI: 0.77, 1.05; P  = 0.19). Discussion Contrary to expected utility models of help-seeking, trainees may not consider personal discomfort or impression management to be important enough, compared to patient safety and other considerations, to influence decisions regarding in-consultation help-seeking. Clinical supervisors should, nevertheless, consider the potential personal costs to trainees and maintain trainee self-esteem and confidence by providing in-consultation assistance in front of patients as comfortably and effectively as possible.

"Discovering shine through feedback seeking"---feedback seeking among new graduate nurses: a qualitative study

BACKGROUND

Feedback is critical to improving practitioners' clinical practice and professional growth. Although they are still considered junior practitioners, their feedback-seeking experiences have yet to be investigated. This study aimed to understand the fundamental thoughts and experiences of new graduate nurses regarding feedback-seeking and to identify the main factors that influence their feedback-seeking behaviors.

METHODS

Conducting a descriptive phenomenological study, semi-structured in-depth interviews with newly graduated nurses from four hospitals in Zhejiang Province, China, face-to-face or via video call in the hospital conference room through purposive and snowball sampling. Interview data were evaluated using Colaizzi's 7-step phenomenological data analysis. The COREQ checklist was followed.

RESULTS

A total of 15 new graduate nurses were interviewed as a sample, and 13 categories emerged from our data. They were categorized into four central elements: (1) perceptions and attitudes, (2) drivers, (3) dilemmas and needs, and (4) transformation and growth.

CONCLUSIONS

This study found that new graduate nurses have various needs but face dilemmas in the feedback-seeking process. Nursing managers should be proficient at providing positive leadership, collaborating with clinical mentors to foster an atmosphere where new graduate nurses may obtain honest, transparent, and fair feedback, and exercising caution when providing negative feedback.

Workplace-based assessments-Articulating the playbook

INTRODUCTION

A workplace-based assessment (WBA) is a learning recording device that is widely used in medical education globally. Although entrenched in medical curricula, and despite a substantial body of literature exploring them, it is not yet fully understood how WBAs play out in practice. Adopting a constructivist standpoint, we examine these assessments, in the workplace, using principles based upon naturalist inquiry, drawing from a theoretical framework based on Goffman's dramaturgical analogy for the presentation of self, and using qualitative research methods to articulate what is happening as learners complete them.

METHODS

Learners were voluntarily recruited to participate in the study from a single teaching hospital. Data were generated, in-situ, through observations with field notes and audiovisual recording of WBAs, along with accompanying interviews with learners.

RESULTS

Data from six learners was analysed to reveal a set of general principles-the WBA playbook. These four principles were tacit, unwritten, unofficial and learners applied them to complete their WBA proformas: (1) maintain the impression of progression, (2) manage the authenticity of the individual proforma, (3) avoid losing face with the assessor and (4) complete the proforma in an effort-efficient way. By adhering to these principles, learners expressed their understanding of their social position in their world at that time the documents were created.

DISCUSSION

This paper recognises the value of the WBA as a lived experience, and of the WBA document as a social space, where learners engage in a social performance before the readers of the proforma. Such an interpretation better represents what happens as learners undergo and record WBAs in the real-world, recognising WBAs as learner-centred, learner-driven, meaning-making phenomena. In this way, as a record of interpretation and meanings, the subjective nature of the WBA process is a strength to be harnessed, rather than a weakness to be glossed over.

A scoping review of self-monitoring in graduate medical education

BACKGROUND

Physicians and physicians-in-training have repeatedly demonstrated poor accuracy of global self-assessments, which are assessments removed from the context of a specific task, regardless of any intervention. Self-monitoring, an in-the-moment self-awareness of one's performance, offers a promising alternative to global self-assessment. The purpose of this scoping review is to better understand the state of self-monitoring in graduate medical education.

METHODS

We performed a scoping review following Arksey and O'Malley's six steps: identifying a research question, identifying relevant studies, selecting included studies, charting the data, collating and summarising the results and consulting experts. Our search queried Ovid Medline, Web of Science, PsychINFO, Eric and EMBASE databases from 1 January 1999 to 12 October 2022.

RESULTS

The literature search yielded 5363 unique articles. The authors identified 77 articles for inclusion. The search process helped create a framework to identify self-monitoring based on time and context dependence. More than 20 different terms were used to describe self-monitoring, and only 13 studies (17%) provided a definition for the equivalent term. Most research focused on post-performance self-judgements of a procedural skill (n = 31, 42%). Regardless of task, studies focused on self-judgement (n = 66, 86%) and measured the accuracy or impact on performance of self-monitoring (n = 41, 71%). Most self-monitoring was conducted post-task (n = 65, 84%).

CONCLUSION

Self-monitoring is a time- and context-dependent phenomenon that seems promising as a research focus to improve clinical performance of trainees in graduate medical education and beyond. The landscape of current literature on self-monitoring is sparse and heterogeneous, suffering from a lack of theoretical underpinning, inconsistent terminology and insufficiently clear definitions.

Evaluating the perceptions of workplace-based assessments in surgical training: a systematic review

INTRODUCTION

Workplace-based assessments (WBAs) are intended to maximise learning opportunities in surgical training. There is speculation as to whether mandatory assessments in this form contribute to a tick-box culture. The objective of this review was to investigate surgical trainees' attitudes towards WBAs.

METHODS

This systematic review of qualitative studies was performed in accordance with the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) statement. The literature was searched on the Medline®, Embase™, PubMed and Web of Science™ databases on 22 March 2022.

RESULTS

Sixteen studies were included in the review, mostly carried out on users of the Intercollegiate Surgical Curriculum Programme portfolio in the UK. Trainees felt that WBAs were educationally useful, providing opportunity for feedback, but this was overshadowed by a pressure to reach a set annual quota for WBAs and achieve high scores. Other themes included inaccurate recording of WBAs, the role of WBAs as formative or summative assessments, engagement and accessibility of trainers, and lack of time to complete WBAs.

CONCLUSIONS

Negative perceptions about WBAs were widespread among surgical trainees despite a recognition of their capacity to facilitate learning. This review supports the recent removal of the annual quota for WBAs in UK surgical training programmes.

Assessment-Seeking Strategies: Navigating the Decision to Initiate Workplace-Based Assessment

Phenomenon: Competency-based medical education (CBME) relies on workplace-based assessment (WBA) to generate formative feedback (assessment for learning-AfL) and make inferences about competence (assessment of learning-AoL). When approaches to CBME rely on residents to initiate WBA, learners experience tension between seeking WBA for learning and for establishing competence. How learners resolve this tension may lead to unintended consequences for both AfL and AoL. We sought to explore the factors that impact both decisions to seek and not to seek WBA and use the findings to build a model of assessment-seeking strategy used by residents. In building this model we consider how the link between WBA and promotion or progression within a program impacts an individual's assessment-seeking strategy. Approach: We conducted 20 semi-structured interviews with internal medicine residents at Queen's University about the factors that influence their decision to seek or avoid WBA. Using grounded theory methodology, we applied a constant comparative analysis to collect data iteratively and identify themes. A conceptual model was developed to describe the interaction of factors impacting the decision to seek and initiate WBA. Findings: Participants identified two main motivations when deciding to seek assessments: the need to fulfill program requirements and the desire to receive feedback for learning. Analysis suggested that these motivations are often at odds with each other. Participants also described several moderating factors that impact the decision to initiate assessments, irrespective of the primary underlying motivation. These included resident performance, assessor factors, training program expectations, and clinical context. A conceptual framework was developed to describe the factors that lead to strategic assessment-seeking behaviors. Insights: Faced with the dual purpose of WBA in CBME, resident behavior in initiating assessment is guided by specific assessment-seeking strategies. Strategies reflect individual underlying motivations, influenced by four moderating factors. These findings have broad implications for programmatic assessment in a CBME context including validity considerations for assessment data used in summative decision-making including readiness for unsupervised practice.

A scoping review of assessment methods of competence of general surgical trainees

BACKGROUND

Only rigorous evaluation of competence will result in the production of safe surgeons that are able to provide the best care for patients. The development of competency-based assessment should ultimately be evidence driven.

OBJECTIVES

Explore the volume of existing evidence pertaining to the different objective assessment methods reported in the literature.

ELIGIBILITY CRITERIA

Studies describing objective assessment of postgraduate general surgical trainees within the last 20 years.

SOURCES OF EVIDENCE

PubMed, Ovid Medline and Web of Sciences.

CHARTING METHODS

A data chart proforma was designed and data were extracted into tables. Basic numerical analysis of extracted data and narrative synthesis of charted data.

RESULTS

A total of 343 papers were reviewed. 26 were eligible for inclusion. 92% of articles were published from 2008 onwards. 50% have been published in the last five years. The articles originated from 6 different countries, predominantly the United Kingdom (42%), followed by the United States of America (38%). In addition, a small number were published from Canada (8%), Japan (4%), Germany (4%) and Australia (4%). UK publications were predominantly between 2008 and 2014 while the USA had a later predominance between 2015 and 2018. 42% were based on quantitative methodology, 27% had a qualitative approach while 31% had mixed analysis. There were sixteen assessment methods presented. The most common type of assessment was Objective Structured Assessments (27%), which included Objective Structured Assessment of Technical Skill (OSATS) (23%) and Objective Structured Assessment of Non-Technical Skill (4%). Procedure Based Assessment (PBA) (23%) and Entrustability Scales (23%) were also prevalent.

CONCLUSIONS

This scoping review has identified a range of different assessment methods. The assessment methods with a higher volume and level of supporting evidence were OSATS, PBAs and Entrustability Scales. There was a lower volume and level of supporting evidence found within this review for the remaining assessment methods.

Feedback-Seeking Behavior and Practice Readiness for General Surgery

OBJECTIVE

To analyze the relationship between feedback-seeking behavior, operationalized as the number of trainee-requested evaluations, with ratings of surgical trainees' operative autonomy and performance.

DESIGN

We analyzed operative assessment data using the System for Improving and Measuring Procedural Learning's smartphone-based assessment app called Society for Improving Medical Professional Learning (SIMPL) OR. Using cross-classified mixed effects models, we analyzed the association between trainee-requested SIMPL OR app evaluations and both trainee performance and autonomy ratings. Models included covariates for requested evaluations, PGY-year, month of the academic year, and patient-related case complexity. Random effects for program, procedure, rater, and trainee were also included to account for correlations among evaluations. Only ratings for procedures deemed Core to general surgery were included.

SETTING

Operative assessment data using the SIMPL OR app requested by categorical U.S. general surgery residents between September 2015 to April 2021.

PARTICIPANTS

A total of 61 general surgery residency programs, encompassing 2190 categorical general surgery residents.

RESULTS

A total of 58,104 SIMPL app operative assessments were analyzed. Autonomy scores were weakly but positively associated with number of trainee-requested evaluations (B = 0.002, p < 0.001). Trainee-requested evaluations were also statistically associated with operative performance scores ( B = 0.002, p < 0.001).

CONCLUSIONS

The propensity of a resident to seek feedback using the SIMPL app was weakly associated with higher operative autonomy ratings and higher operative performance ratings. While regular feedback is important for monitoring performance over time, more direct approaches related to the quality of feedback that trainees receive may be needed to better assess the relationships between feedback-seeking behavior and operative autonomy as well as performance.

Strategies for the assessment of competences during rheumatology training across Europe: results of a qualitative study

OBJECTIVES

To gain insight into current methods and practices for the assessment of competences during rheumatology training, and to explore the underlying priorities and rationales for competence assessment.

METHODS

We used a qualitative approach through online focus groups (FGs) of rheumatology trainers and trainees, separately. The study included five countries-Denmark, the Netherlands, Slovenia, Spain and the United Kingdom. A summary of current practices of assessment of competences was developed, modified and validated by the FGs based on an independent response to a questionnaire. A prioritising method (9 Diamond technique) was then used to identify and justify key assessment priorities.

RESULTS

Overall, 26 participants (12 trainers, 14 trainees) participated in nine online FGs (2 per country, Slovenia 1 joint), totalling 12 hours of online discussion. Strong nationally (the Netherlands, UK) or institutionally (Spain, Slovenia, Denmark) standardised approaches were described. Most groups identified providing frequent formative feedback to trainees for developmental purposes as the highest priority. Most discussions identified a need for improvement, particularly in developing streamlined approaches to portfolios that remain close to clinical practice, protecting time for quality observation and feedback, and adopting systematic approaches to incorporating teamwork and professionalism into assessment systems.

CONCLUSION

This paper presents a clearer picture of the current practice on the assessment of competences in rheumatology in five European countries and the underlying rationale of trainers' and trainees' priorities. This work will inform EULAR Points-to-Consider for the assessment of competences in rheumatology training across Europe.

Resident impression management within feedback conversations: A qualitative study

INTRODUCTION

Medical education is moving to conceptualise feedback as a bidirectional learning conversation. Within this conversation, learners experience a tension between assessment and feedback. That perceived tension affects learners' outward performances. In this study, we aimed to characterise residents' experiences with this tension and its effect on learner authenticity within feedback conversations.

METHODS

In this constructivist grounded theory study, the authors were informed by Goffman's theory of impression management. During data analysis, Dweck's theory of mindset was adopted. The authors conducted semi-structured interviews with 15 internal medicine residents. Data collection and analysis were conducted iteratively, using constant comparison to identify themes coinciding with impression management and mindset, ultimately developing a theoretical model to help explain residents' responses to tensions within feedback conversations.

RESULTS

Residents constantly felt 'scrutinized', and this affected their engagement in feedback conversations. They staged a performance within those conversations, linked to their underlying mindset: growth or fixed. Growth mindset was characterised by a focus on development as a physician and was associated with asking questions and seeking opportunities for growth. Fixed mindset was characterised by a focus on achieving a favourable evaluation and was associated with a hesitation to ask questions when faced with uncertainty and admit opportunities for growth, because they were concerned about impression management. Context influenced mindset and impression management. Residents adopted a fixed mindset and managed impressions when they perceived the permanence or consequences of evaluations within feedback. Residents adopted a growth mindset when they trusted the supervisor.

DISCUSSION

Residents assess the context of feedback conversations, altering the authenticity of their behaviours. Context, including the perceptions of formal assessment and relationships with supervisors, affected residents' mindset and impression management. Providing space for relationship-building and clarifying the purpose and structure of assessment may be helpful in supporting effective learning conversations in graduate medical education.