1
|
Costa CDDS, Silva GG, Santos ERD, Engel AMRPVTDC, Costa ACDS, da Silva TM, da Conceição WH, Cristóvão H, Lima ARDA, Brienze VM, Bizotto TSG, Oliani AH, André JC. Surgical Residents' Perception of Feedback on Their Education: Protocol for a Scoping Review. JMIR Res Protoc 2024; 13:e56727. [PMID: 39158942 PMCID: PMC11369536 DOI: 10.2196/56727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 04/22/2024] [Accepted: 07/11/2024] [Indexed: 08/20/2024] Open
Abstract
BACKGROUND Feedback is an essential tool for learning and improving performance in any sphere of education, including training of resident physicians. The learner's perception of the feedback they receive is extremely relevant to their learning progress, which must aim at providing qualified care for patients. Studies pertinent to the matter differ substantially with respect to methodology, population, context, and objective, which makes it even more difficult to achieve a clear understanding of the topic. A scoping review on this theme will unequivocally enhance and organize what is already known. OBJECTIVE The aim of this study is to identify and map out data from studies that report surgical residents' perception of the feedback received during their education. METHODS The review will consider studies on the feedback perception of resident physicians of any surgical specialty and age group, attending any year of residency, regardless of the type of feedback given and the way the perceptions were measured. Primary studies published in English, Spanish, and Portuguese since 2017 will be considered. The search will be carried out in 6 databases and reference lists will also be searched for additional studies. Duplicates will be removed, and 2 independent reviewers will screen the selected studies' titles, abstracts, and full texts. Data extraction will be performed through a tool developed by the researchers. Descriptive statistics and qualitative analysis (content analysis) will be used to analyze the data. A summary of the results will be presented in the form of diagrams, narratives, and tables. RESULTS The findings of this scoping review were submitted to an indexed journal in July 2024, currently awaiting reviewer approval. The search was executed on March 15, 2024, and resulted in 588 articles. After the exclusion of the duplicate articles and those that did not meet the eligibility criteria as well as the inclusion of articles through a manual search, 13 articles were included in the review. CONCLUSIONS Conducting a scoping review is the best way to map what is known about a subject. By focusing on the feedback perception more than the feedback itself, the results of this study will surely contribute to gaining a deeper understanding of how to proceed to enhance internal feedback and surgical residents' learning progress. TRIAL REGISTRATION Open Science Framework yexb; https://osf.io/yexkb. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) PRR1-10.2196/56727.
Collapse
Affiliation(s)
- Carlos Dario da Silva Costa
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| | - Gabriela Gouvea Silva
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| | - Emerson Roberto Dos Santos
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| | | | - Ana Caroline Dos Santos Costa
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| | - Taisa Morete da Silva
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| | - Washington Henrique da Conceição
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| | - Helena Cristóvão
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| | - Alba Regina de Abreu Lima
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| | - Vânia Ms Brienze
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| | - Thaís Santana Gastardelo Bizotto
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| | - Antonio Hélio Oliani
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
- University Hospital Center Cova da Beira, University of Beira Interior, Covilhã, Portugal
| | - Júlio César André
- Center for Studies and Development of Health Education, Faculty of Medicine of São José do Rio Preto, São José do Rio Preto, São Paulo, Brazil
| |
Collapse
|
2
|
Shavoun AH, Mirzazadeh A, Kashani H, Raeeskarami SR, Gandomkar R. Translation and psychometric evaluation of composite feedback-seeking behavior questionnaire among Iranian medical residents. BMC MEDICAL EDUCATION 2024; 24:594. [PMID: 38811982 PMCID: PMC11137997 DOI: 10.1186/s12909-024-05586-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Accepted: 05/22/2024] [Indexed: 05/31/2024]
Abstract
BACKGROUND Proactively seeking feedback from clinical supervisors, peers or other healthcare professionals is a valuable mechanism for residents to obtain useful information about and improve their performance in clinical settings. Given the scant studies investigating the limited aspects of psychometrics properties of the feedback-seeking instruments in medical education, this study aimed to translate the feedback-seeking behavior scales (frequency of feedback-seeking, motives of feedback-seeking, and promotion of feedback-seeking by supervisors) into Persian and evaluate the psychometric properties of the composite questionnaire among medical residents at Tehran University of Medical Sciences in Iran. METHODS In this cross-sectional study, feedback-seeking behavior scales were translated through the forward-backward method, and its face validity and content validity were assessed by 10 medical residents and 18 experts. The test-retest reliability was evaluated by administering the questionnaire to 20 medical residents on two testing occasions. A convenience sample of 548 residents completed the questionnaire. Construct validity was examined by exploratory factor analysis and confirmatory factor analysis and concurrent validity was determined by Pearson's correlation coefficient. RESULTS Content validity assessment showed that the CVR (0.66 to 0.99) and CVI (0.82 to 0.99) values for items and S-CVI values (0.88 to 0.99) for scales were satisfactory. The exploratory and confirmatory factor analysis demonstrated that the models were confirmed with eight items and two factors (explaining 70.98% of the total variance) for the frequency of feedback-seeking scale, with 16 items and four factors (explaining 73.22% of the total variance) for the motives of feedback seeking scale and with four items and one factor (explaining 69.46% of the total variance) for promotion of feedback-seeking by supervisors. AVE values greater than 0.5 and discriminant validity correlations significantly less than 1.0 demonstrated that the total scores of the composite feedback-seeking behavior questionnaire had a favorable fit and the questions could fit their respective factors, and the latent variables were distinct. We found positive and significant correlations between the three scales and their subscales. CONCLUSION The results of the present study supported the validity and reliability of the Persian composite feedback-seeking behavior questionnaire for assessing feedback-seeking behaviors in medical residents. Applying the questionnaire in residency programs may enhance the quality of clinical education.
Collapse
Affiliation(s)
- Amin Hoseini Shavoun
- Department of Medical Education, School of Medicine, Tehran University of Medical Sciences, No. 57, Hojatdoust St., Keshavarz Blvd, Tehran, Iran
| | - Azim Mirzazadeh
- Department of Internal Medicine, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - Homa Kashani
- Department of Research Methodology and Data Analysis, Institute for Environmental Research, Tehran University of Medical Sciences, Tehran, Iran
| | - Seyed Reza Raeeskarami
- Department of Pediatrics, Imam Khomeini Hospital Complex, Tehran University of Medical Sciences, Tehran, Iran
| | - Roghayeh Gandomkar
- Department of Medical Education, School of Medicine, Tehran University of Medical Sciences, No. 57, Hojatdoust St., Keshavarz Blvd, Tehran, Iran.
- Health Professions Education Research Center, Tehran University of Medical Sciences, Tehran, Iran.
| |
Collapse
|
3
|
Braund H, Dalgarno N, O’Dell R, Taylor DR. Making assessment a team sport: a qualitative study of facilitated group feedback in internal medicine residency. CANADIAN MEDICAL EDUCATION JOURNAL 2024; 15:14-26. [PMID: 38827914 PMCID: PMC11139793 DOI: 10.36834/cmej.75250] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/05/2024]
Abstract
Purpose Competency-based medical education relies on feedback from workplace-based assessment (WBA) to direct learning. Unfortunately, WBAs often lack rich narrative feedback and show bias towards Medical Expert aspects of care. Building on research examining interactive assessment approaches, the Queen's University Internal Medicine residency program introduced a facilitated, team-based assessment initiative ("Feedback Fridays") in July 2017, aimed at improving holistic assessment of resident performance on the inpatient medicine teaching units. In this study, we aim to explore how Feedback Fridays contributed to formative assessment of Internal Medicine residents within our current model of competency-based training. Method A total of 53 residents participated in facilitated, biweekly group assessment sessions during the 2017 and 2018 academic year. Each session was a 30-minute facilitated assessment discussion done with one inpatient team, which included medical students, residents, and their supervising attending. Feedback from the discussion was collected, summarized, and documented in narrative form in electronic WBA forms by the program's assessment officer for the residents. For research purposes, verbatim transcripts of feedback sessions were analyzed thematically. Results The researchers identified four major themes for feedback: communication, intra- and inter-personal awareness, leadership and teamwork, and learning opportunities. Although feedback related to a broad range of activities, it showed strong emphasis on competencies within the intrinsic CanMEDS roles. Additionally, a clear formative focus in the feedback was another important finding. Conclusions The introduction of facilitated team-based assessment in the Queen's Internal Medicine program filled an important gap in WBA by providing learners with detailed feedback across all CanMEDS roles and by providing constructive recommendations for identified areas for improvement.
Collapse
Affiliation(s)
- Heather Braund
- Office of Professional Development and Educational Scholarship, Ontario, Canada
- Faculty of Education, Queen’s University, Ontario, Canada
| | - Nancy Dalgarno
- Office of Professional Development and Educational Scholarship, Ontario, Canada
- Department of Biomedical and Molecular Sciences, Faculty of Health Sciences, Queen’s University, Ontario, Canada
| | - Rachel O’Dell
- Department of Internal Medicine, Faculty of Health Sciences, Queen’s University, Ontario, Canada
| | - David R Taylor
- Academy for Teachers and Educators, Department of Medicine, Queen’s University, Ontario, Canada
| |
Collapse
|
4
|
Sturman N, Tapley A, Holliday E, Fielding A, Davey A, van Driel M, Ball J, FitzGerald K, Spike N, Magin P. Utilisation of in-consultation supervisor assistance in general practice training and personal cost to trainees: a cross-sectional study. J Prim Health Care 2024; 16:4-11. [PMID: 38546770 DOI: 10.1071/hc23044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Accepted: 06/18/2023] [Indexed: 04/02/2024] Open
Abstract
Aim The aim of the study was to establish whether two previously described barriers to effective in-consultation assistance-seeking by general practice (GP) vocational specialist trainees (ie concern about patient impressions of their competence, and discomfort presenting to supervisors in front of patients) influenced the frequency of trainee in-consultation assistance-seeking from their supervisor. Methods This was a cross-sectional study nested in the Registrar Clinical Encounters in Clinical Training ongoing cohort study of Australian GP trainees. Trainee participants completed contemporaneous records of 60 consecutive patient consultations, including whether supervisory assistance was sought. Trainees also completed a cross-sectional survey including items eliciting their beliefs about patient impressions and their own discomfort in seeking in-consultation supervisory assistance. These were factors of interest in multivariable logistic regression analyses; the outcome factor in both regression models was the seeking of in-consultation supervisory assistance. Results In 2018, 778 trainees (778/876, response rate 89%) completed the cross-sectional survey. No association was found between the odds of in-consultation help-seeking and perceived decrease in patient impressions of trainee competence (OR = 1.09; 95% CI: 0.91, 1.31; P = 0.36) or higher comfort presenting outside patients' hearing (OR = 0.9; 95% CI: 0.77, 1.05; P = 0.19). Discussion Contrary to expected utility models of help-seeking, trainees may not consider personal discomfort or impression management to be important enough, compared to patient safety and other considerations, to influence decisions regarding in-consultation help-seeking. Clinical supervisors should, nevertheless, consider the potential personal costs to trainees and maintain trainee self-esteem and confidence by providing in-consultation assistance in front of patients as comfortably and effectively as possible.
Collapse
Affiliation(s)
- Nancy Sturman
- General Practice Clinical Unit, Faculty of Medicine, University of Queensland, Level 8, Health Sciences Building, Royal Brisbane and Women's Hospital, Brisbane, Qld 4006, Australia
| | - Amanda Tapley
- School of Medicine and Public Health, University of Newcastle, University Drive, Callaghan, Newcastle, NSW 2308, Australia; and NSW and ACT Research and Evaluation Unit, GP Synergy, Regional Training Organisation, 20 McIntosh Drive, Mayfield West, NSW 2304, Australia
| | - Elizabeth Holliday
- School of Medicine and Public Health, University of Newcastle, University Drive, Callaghan, Newcastle, NSW 2308, Australia
| | - Alison Fielding
- School of Medicine and Public Health, University of Newcastle, University Drive, Callaghan, Newcastle, NSW 2308, Australia; and NSW and ACT Research and Evaluation Unit, GP Synergy, Regional Training Organisation, 20 McIntosh Drive, Mayfield West, NSW 2304, Australia
| | - Andrew Davey
- School of Medicine and Public Health, University of Newcastle, University Drive, Callaghan, Newcastle, NSW 2308, Australia; and NSW and ACT Research and Evaluation Unit, GP Synergy, Regional Training Organisation, 20 McIntosh Drive, Mayfield West, NSW 2304, Australia
| | - Mieke van Driel
- General Practice Clinical Unit, Faculty of Medicine, University of Queensland, Level 8, Health Sciences Building, Royal Brisbane and Women's Hospital, Brisbane, Qld 4006, Australia
| | - Jean Ball
- Clinical Research Design and Statistical Support Unit, Hunter Medical Research Institute, New Lambton Heights, NSW 2305, Australia
| | - Kristen FitzGerald
- School of Medicine, University of Tasmania, Level 1, Medical Science 1, 17 Liverpool Street, Hobart, Tas. 7000, Australia; and General Practice Training Tasmania, Regional Training Organisation, Level 3, RACT House, 179 Murray Street, Hobart, Tas. 7000, Australia
| | - Neil Spike
- Department of General Practice and Primary Health Care, University of Melbourne, 200 Berkeley Street, Carlton, Vic. 3053, Australia; and School of Rural Health, Faculty of Medicine, Nursing and Health Sciences, Monash University, Northways Road, Churchill, Vic. 3842, Australia; and Eastern Victoria General Practice Training, Regional Training Organisation, 15 Cato Street, Hawthorn, Vic. 3122, Australia
| | - Parker Magin
- School of Medicine and Public Health, University of Newcastle, University Drive, Callaghan, Newcastle, NSW 2308, Australia; and NSW and ACT Research and Evaluation Unit, GP Synergy, Regional Training Organisation, 20 McIntosh Drive, Mayfield West, NSW 2304, Australia
| |
Collapse
|
5
|
Song Z, Shen Y, Yao X, Wen S, Wang J, Chen Y, Zhang P, Huang X. "Discovering shine through feedback seeking"---feedback seeking among new graduate nurses: a qualitative study. BMC Nurs 2024; 23:10. [PMID: 38163860 PMCID: PMC10759581 DOI: 10.1186/s12912-023-01657-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 12/11/2023] [Indexed: 01/03/2024] Open
Abstract
BACKGROUND Feedback is critical to improving practitioners' clinical practice and professional growth. Although they are still considered junior practitioners, their feedback-seeking experiences have yet to be investigated. This study aimed to understand the fundamental thoughts and experiences of new graduate nurses regarding feedback-seeking and to identify the main factors that influence their feedback-seeking behaviors. METHODS Conducting a descriptive phenomenological study, semi-structured in-depth interviews with newly graduated nurses from four hospitals in Zhejiang Province, China, face-to-face or via video call in the hospital conference room through purposive and snowball sampling. Interview data were evaluated using Colaizzi's 7-step phenomenological data analysis. The COREQ checklist was followed. RESULTS A total of 15 new graduate nurses were interviewed as a sample, and 13 categories emerged from our data. They were categorized into four central elements: (1) perceptions and attitudes, (2) drivers, (3) dilemmas and needs, and (4) transformation and growth. CONCLUSIONS This study found that new graduate nurses have various needs but face dilemmas in the feedback-seeking process. Nursing managers should be proficient at providing positive leadership, collaborating with clinical mentors to foster an atmosphere where new graduate nurses may obtain honest, transparent, and fair feedback, and exercising caution when providing negative feedback.
Collapse
Affiliation(s)
- Ziling Song
- School of Ophthalmology and Optometry, Wenzhou Medical University, 270 West Xueyuan Road Wenzhou, Wenzhou, 325027, Zhejiang, China
| | - Yuanyuan Shen
- School of Ophthalmology and Optometry, Wenzhou Medical University, 270 West Xueyuan Road Wenzhou, Wenzhou, 325027, Zhejiang, China
| | - Xin Yao
- School of Ophthalmology and Optometry, Wenzhou Medical University, 270 West Xueyuan Road Wenzhou, Wenzhou, 325027, Zhejiang, China
| | - Siqi Wen
- School of Nursing, Wenzhou Medical University, Wenzhou, 325000, Zhejiang, China
| | - Jing Wang
- School of Ophthalmology and Optometry, Wenzhou Medical University, 270 West Xueyuan Road Wenzhou, Wenzhou, 325027, Zhejiang, China
| | - Yanyan Chen
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, 270 West Xueyuan Road, Wenzhou, 325027, Zhejiang, China
| | - Peihua Zhang
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, 270 West Xueyuan Road, Wenzhou, 325027, Zhejiang, China
| | - Xiaoqiong Huang
- National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, 270 West Xueyuan Road, Wenzhou, 325027, Zhejiang, China.
| |
Collapse
|
6
|
Tahim A, Gill D, Bezemer J. Workplace-based assessments-Articulating the playbook. MEDICAL EDUCATION 2023; 57:939-948. [PMID: 36924016 DOI: 10.1111/medu.15083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 02/13/2023] [Accepted: 03/10/2023] [Indexed: 06/18/2023]
Abstract
INTRODUCTION A workplace-based assessment (WBA) is a learning recording device that is widely used in medical education globally. Although entrenched in medical curricula, and despite a substantial body of literature exploring them, it is not yet fully understood how WBAs play out in practice. Adopting a constructivist standpoint, we examine these assessments, in the workplace, using principles based upon naturalist inquiry, drawing from a theoretical framework based on Goffman's dramaturgical analogy for the presentation of self, and using qualitative research methods to articulate what is happening as learners complete them. METHODS Learners were voluntarily recruited to participate in the study from a single teaching hospital. Data were generated, in-situ, through observations with field notes and audiovisual recording of WBAs, along with accompanying interviews with learners. RESULTS Data from six learners was analysed to reveal a set of general principles-the WBA playbook. These four principles were tacit, unwritten, unofficial and learners applied them to complete their WBA proformas: (1) maintain the impression of progression, (2) manage the authenticity of the individual proforma, (3) avoid losing face with the assessor and (4) complete the proforma in an effort-efficient way. By adhering to these principles, learners expressed their understanding of their social position in their world at that time the documents were created. DISCUSSION This paper recognises the value of the WBA as a lived experience, and of the WBA document as a social space, where learners engage in a social performance before the readers of the proforma. Such an interpretation better represents what happens as learners undergo and record WBAs in the real-world, recognising WBAs as learner-centred, learner-driven, meaning-making phenomena. In this way, as a record of interpretation and meanings, the subjective nature of the WBA process is a strength to be harnessed, rather than a weakness to be glossed over.
Collapse
Affiliation(s)
- Arpan Tahim
- Department of Culture, Communication and Media, UCL Institute of Education, London, UK
| | - Deborah Gill
- Faculty of Medicine, University of Southampton, Southampton, UK
| | - Jeff Bezemer
- Department of Culture, Communication and Media, UCL Institute of Education, London, UK
| |
Collapse
|
7
|
Johnson WR, Durning SJ, Allard RJ, Barelski AM, Artino AR. A scoping review of self-monitoring in graduate medical education. MEDICAL EDUCATION 2023; 57:795-806. [PMID: 36739527 DOI: 10.1111/medu.15023] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/13/2022] [Revised: 01/11/2023] [Accepted: 01/31/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND Physicians and physicians-in-training have repeatedly demonstrated poor accuracy of global self-assessments, which are assessments removed from the context of a specific task, regardless of any intervention. Self-monitoring, an in-the-moment self-awareness of one's performance, offers a promising alternative to global self-assessment. The purpose of this scoping review is to better understand the state of self-monitoring in graduate medical education. METHODS We performed a scoping review following Arksey and O'Malley's six steps: identifying a research question, identifying relevant studies, selecting included studies, charting the data, collating and summarising the results and consulting experts. Our search queried Ovid Medline, Web of Science, PsychINFO, Eric and EMBASE databases from 1 January 1999 to 12 October 2022. RESULTS The literature search yielded 5363 unique articles. The authors identified 77 articles for inclusion. The search process helped create a framework to identify self-monitoring based on time and context dependence. More than 20 different terms were used to describe self-monitoring, and only 13 studies (17%) provided a definition for the equivalent term. Most research focused on post-performance self-judgements of a procedural skill (n = 31, 42%). Regardless of task, studies focused on self-judgement (n = 66, 86%) and measured the accuracy or impact on performance of self-monitoring (n = 41, 71%). Most self-monitoring was conducted post-task (n = 65, 84%). CONCLUSION Self-monitoring is a time- and context-dependent phenomenon that seems promising as a research focus to improve clinical performance of trainees in graduate medical education and beyond. The landscape of current literature on self-monitoring is sparse and heterogeneous, suffering from a lack of theoretical underpinning, inconsistent terminology and insufficiently clear definitions.
Collapse
Affiliation(s)
- William Rainey Johnson
- Military and Emergency Medicine and Medicine, Uniformed Services University of Health Sciences, Bethesda, Maryland, USA
| | - Steven J Durning
- Center for Health Professions Education, Uniformed Services University of Health Sciences, Bethesda, Maryland, USA
| | - Rhonda J Allard
- James A. Zimble Learning Resource Center, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Adam M Barelski
- Department of Medicine, Uniformed Services University of Health Sciences, Bethesda, Maryland, USA
| | - Anthony R Artino
- School of Medicine and Health Sciences, George Washington University, Washington, District of Columbia, USA
| |
Collapse
|
8
|
Mughal Z, Patel S, Gupta KK, Metcalfe C, Beech T, Jennings C. Evaluating the perceptions of workplace-based assessments in surgical training: a systematic review. Ann R Coll Surg Engl 2023; 105:507-512. [PMID: 36374289 PMCID: PMC10313445 DOI: 10.1308/rcsann.2022.0113] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/01/2022] [Indexed: 11/24/2023] Open
Abstract
INTRODUCTION Workplace-based assessments (WBAs) are intended to maximise learning opportunities in surgical training. There is speculation as to whether mandatory assessments in this form contribute to a tick-box culture. The objective of this review was to investigate surgical trainees' attitudes towards WBAs. METHODS This systematic review of qualitative studies was performed in accordance with the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) statement. The literature was searched on the Medline®, Embase™, PubMed and Web of Science™ databases on 22 March 2022. RESULTS Sixteen studies were included in the review, mostly carried out on users of the Intercollegiate Surgical Curriculum Programme portfolio in the UK. Trainees felt that WBAs were educationally useful, providing opportunity for feedback, but this was overshadowed by a pressure to reach a set annual quota for WBAs and achieve high scores. Other themes included inaccurate recording of WBAs, the role of WBAs as formative or summative assessments, engagement and accessibility of trainers, and lack of time to complete WBAs. CONCLUSIONS Negative perceptions about WBAs were widespread among surgical trainees despite a recognition of their capacity to facilitate learning. This review supports the recent removal of the annual quota for WBAs in UK surgical training programmes.
Collapse
Affiliation(s)
- Z Mughal
- University Hospitals Birmingham NHS Foundation Trust, UK
| | - S Patel
- Shrewsbury and Telford Hospital NHS Foundation Trust, UK
| | - K Kumar Gupta
- University Hospitals Birmingham NHS Foundation Trust, UK
| | - C Metcalfe
- University Hospitals Birmingham NHS Foundation Trust, UK
| | - T Beech
- University Hospitals Birmingham NHS Foundation Trust, UK
| | - C Jennings
- University Hospitals Birmingham NHS Foundation Trust, UK
| |
Collapse
|
9
|
Gauthier S, Braund H, Dalgarno N, Taylor D. Assessment-Seeking Strategies: Navigating the Decision to Initiate Workplace-Based Assessment. TEACHING AND LEARNING IN MEDICINE 2023:1-10. [PMID: 37384570 DOI: 10.1080/10401334.2023.2229803] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Revised: 04/13/2023] [Accepted: 06/01/2023] [Indexed: 07/01/2023]
Abstract
Phenomenon: Competency-based medical education (CBME) relies on workplace-based assessment (WBA) to generate formative feedback (assessment for learning-AfL) and make inferences about competence (assessment of learning-AoL). When approaches to CBME rely on residents to initiate WBA, learners experience tension between seeking WBA for learning and for establishing competence. How learners resolve this tension may lead to unintended consequences for both AfL and AoL. We sought to explore the factors that impact both decisions to seek and not to seek WBA and use the findings to build a model of assessment-seeking strategy used by residents. In building this model we consider how the link between WBA and promotion or progression within a program impacts an individual's assessment-seeking strategy. Approach: We conducted 20 semi-structured interviews with internal medicine residents at Queen's University about the factors that influence their decision to seek or avoid WBA. Using grounded theory methodology, we applied a constant comparative analysis to collect data iteratively and identify themes. A conceptual model was developed to describe the interaction of factors impacting the decision to seek and initiate WBA. Findings: Participants identified two main motivations when deciding to seek assessments: the need to fulfill program requirements and the desire to receive feedback for learning. Analysis suggested that these motivations are often at odds with each other. Participants also described several moderating factors that impact the decision to initiate assessments, irrespective of the primary underlying motivation. These included resident performance, assessor factors, training program expectations, and clinical context. A conceptual framework was developed to describe the factors that lead to strategic assessment-seeking behaviors. Insights: Faced with the dual purpose of WBA in CBME, resident behavior in initiating assessment is guided by specific assessment-seeking strategies. Strategies reflect individual underlying motivations, influenced by four moderating factors. These findings have broad implications for programmatic assessment in a CBME context including validity considerations for assessment data used in summative decision-making including readiness for unsupervised practice.
Collapse
Affiliation(s)
- Stephen Gauthier
- Department of Medicine, Queen's University, Kingston, Ontario, Canada
| | - Heather Braund
- Office of Professional Development and Educational Scholarship, Faculty of Health Sciences, Queen's University, Kingston, Ontario, Canada
| | - Nancy Dalgarno
- Office of Professional Development and Educational Scholarship, Faculty of Health Sciences, Queen's University, Kingston, Ontario, Canada
| | - David Taylor
- Department of Medicine, Queen's University, Kingston, Ontario, Canada
| |
Collapse
|
10
|
Hackney L, O'Neill S, O'Donnell M, Spence R. A scoping review of assessment methods of competence of general surgical trainees. Surgeon 2023; 21:60-69. [PMID: 35300909 DOI: 10.1016/j.surge.2022.01.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2021] [Accepted: 01/19/2022] [Indexed: 02/01/2023]
Abstract
BACKGROUND Only rigorous evaluation of competence will result in the production of safe surgeons that are able to provide the best care for patients. The development of competency-based assessment should ultimately be evidence driven. OBJECTIVES Explore the volume of existing evidence pertaining to the different objective assessment methods reported in the literature. ELIGIBILITY CRITERIA Studies describing objective assessment of postgraduate general surgical trainees within the last 20 years. SOURCES OF EVIDENCE PubMed, Ovid Medline and Web of Sciences. CHARTING METHODS A data chart proforma was designed and data were extracted into tables. Basic numerical analysis of extracted data and narrative synthesis of charted data. RESULTS A total of 343 papers were reviewed. 26 were eligible for inclusion. 92% of articles were published from 2008 onwards. 50% have been published in the last five years. The articles originated from 6 different countries, predominantly the United Kingdom (42%), followed by the United States of America (38%). In addition, a small number were published from Canada (8%), Japan (4%), Germany (4%) and Australia (4%). UK publications were predominantly between 2008 and 2014 while the USA had a later predominance between 2015 and 2018. 42% were based on quantitative methodology, 27% had a qualitative approach while 31% had mixed analysis. There were sixteen assessment methods presented. The most common type of assessment was Objective Structured Assessments (27%), which included Objective Structured Assessment of Technical Skill (OSATS) (23%) and Objective Structured Assessment of Non-Technical Skill (4%). Procedure Based Assessment (PBA) (23%) and Entrustability Scales (23%) were also prevalent. CONCLUSIONS This scoping review has identified a range of different assessment methods. The assessment methods with a higher volume and level of supporting evidence were OSATS, PBAs and Entrustability Scales. There was a lower volume and level of supporting evidence found within this review for the remaining assessment methods.
Collapse
Affiliation(s)
| | | | | | - Roy Spence
- Queens University Belfast, United Kingdom
| |
Collapse
|
11
|
Luckoski J, Thelen A, Russell D, George B, Krumm A. Feedback-Seeking Behavior and Practice Readiness for General Surgery. JOURNAL OF SURGICAL EDUCATION 2022; 79:295-301. [PMID: 34838471 DOI: 10.1016/j.jsurg.2021.10.003] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/27/2021] [Revised: 06/11/2021] [Accepted: 10/02/2021] [Indexed: 06/13/2023]
Abstract
OBJECTIVE To analyze the relationship between feedback-seeking behavior, operationalized as the number of trainee-requested evaluations, with ratings of surgical trainees' operative autonomy and performance. DESIGN We analyzed operative assessment data using the System for Improving and Measuring Procedural Learning's smartphone-based assessment app called Society for Improving Medical Professional Learning (SIMPL) OR. Using cross-classified mixed effects models, we analyzed the association between trainee-requested SIMPL OR app evaluations and both trainee performance and autonomy ratings. Models included covariates for requested evaluations, PGY-year, month of the academic year, and patient-related case complexity. Random effects for program, procedure, rater, and trainee were also included to account for correlations among evaluations. Only ratings for procedures deemed Core to general surgery were included. SETTING Operative assessment data using the SIMPL OR app requested by categorical U.S. general surgery residents between September 2015 to April 2021. PARTICIPANTS A total of 61 general surgery residency programs, encompassing 2190 categorical general surgery residents. RESULTS A total of 58,104 SIMPL app operative assessments were analyzed. Autonomy scores were weakly but positively associated with number of trainee-requested evaluations (B = 0.002, p < 0.001). Trainee-requested evaluations were also statistically associated with operative performance scores ( B = 0.002, p < 0.001). CONCLUSIONS The propensity of a resident to seek feedback using the SIMPL app was weakly associated with higher operative autonomy ratings and higher operative performance ratings. While regular feedback is important for monitoring performance over time, more direct approaches related to the quality of feedback that trainees receive may be needed to better assess the relationships between feedback-seeking behavior and operative autonomy as well as performance.
Collapse
Affiliation(s)
- John Luckoski
- Department of Surgery, University of Michigan, Ann Arbor, Michigan.
| | - Angela Thelen
- Department of Surgery, University of Michigan, Ann Arbor, Michigan
| | - Dylan Russell
- Department of Surgery, Tripler Army Medical Center, Honolulu, Hawaii
| | - Brian George
- Department of Surgery, University of Michigan, Ann Arbor, Michigan
| | - Andrew Krumm
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, Michigan
| |
Collapse
|
12
|
Najm A, Alunno A, Sivera F, Ramiro S, Haines C. Strategies for the assessment of competences during rheumatology training across Europe: results of a qualitative study. RMD Open 2021; 6:rmdopen-2020-001183. [PMID: 32641449 PMCID: PMC7962674 DOI: 10.1136/rmdopen-2020-001183] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2020] [Revised: 03/05/2020] [Accepted: 03/29/2020] [Indexed: 12/18/2022] Open
Abstract
OBJECTIVES To gain insight into current methods and practices for the assessment of competences during rheumatology training, and to explore the underlying priorities and rationales for competence assessment. METHODS We used a qualitative approach through online focus groups (FGs) of rheumatology trainers and trainees, separately. The study included five countries-Denmark, the Netherlands, Slovenia, Spain and the United Kingdom. A summary of current practices of assessment of competences was developed, modified and validated by the FGs based on an independent response to a questionnaire. A prioritising method (9 Diamond technique) was then used to identify and justify key assessment priorities. RESULTS Overall, 26 participants (12 trainers, 14 trainees) participated in nine online FGs (2 per country, Slovenia 1 joint), totalling 12 hours of online discussion. Strong nationally (the Netherlands, UK) or institutionally (Spain, Slovenia, Denmark) standardised approaches were described. Most groups identified providing frequent formative feedback to trainees for developmental purposes as the highest priority. Most discussions identified a need for improvement, particularly in developing streamlined approaches to portfolios that remain close to clinical practice, protecting time for quality observation and feedback, and adopting systematic approaches to incorporating teamwork and professionalism into assessment systems. CONCLUSION This paper presents a clearer picture of the current practice on the assessment of competences in rheumatology in five European countries and the underlying rationale of trainers' and trainees' priorities. This work will inform EULAR Points-to-Consider for the assessment of competences in rheumatology training across Europe.
Collapse
Affiliation(s)
- Aurélie Najm
- Rheumatology, University of Glasgow Institute of Infection Immunity and Inflammation, Glasgow, UK
| | - Alessia Alunno
- Department of Medicine, Rheumatology Unit, University of Perugia, Perugia, Italy
| | - Francisca Sivera
- Department of Rheumatology, Hospital General Universitario Elda, Elda, Spain.,Department of Rheumatology, Universidad Miguel Hernandez De Elche, Elche, Spain
| | - Sofia Ramiro
- Leiden University Medical Center, Leiden, Netherlands.,Zuyderland Medical Centre Heerlen, Heerlen, Limburg, Netherlands
| | - Catherine Haines
- EULAR, Zurich, Switzerland.,Clinical Education, King's College London, London, UK
| | | |
Collapse
|
13
|
Huffman BM, Hafferty FW, Bhagra A, Leasure EL, Santivasi WL, Sawatsky AP. Resident impression management within feedback conversations: A qualitative study. MEDICAL EDUCATION 2021; 55:266-274. [PMID: 32815152 DOI: 10.1111/medu.14360] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/21/2020] [Revised: 08/10/2020] [Accepted: 08/17/2020] [Indexed: 05/26/2023]
Abstract
INTRODUCTION Medical education is moving to conceptualise feedback as a bidirectional learning conversation. Within this conversation, learners experience a tension between assessment and feedback. That perceived tension affects learners' outward performances. In this study, we aimed to characterise residents' experiences with this tension and its effect on learner authenticity within feedback conversations. METHODS In this constructivist grounded theory study, the authors were informed by Goffman's theory of impression management. During data analysis, Dweck's theory of mindset was adopted. The authors conducted semi-structured interviews with 15 internal medicine residents. Data collection and analysis were conducted iteratively, using constant comparison to identify themes coinciding with impression management and mindset, ultimately developing a theoretical model to help explain residents' responses to tensions within feedback conversations. RESULTS Residents constantly felt 'scrutinized', and this affected their engagement in feedback conversations. They staged a performance within those conversations, linked to their underlying mindset: growth or fixed. Growth mindset was characterised by a focus on development as a physician and was associated with asking questions and seeking opportunities for growth. Fixed mindset was characterised by a focus on achieving a favourable evaluation and was associated with a hesitation to ask questions when faced with uncertainty and admit opportunities for growth, because they were concerned about impression management. Context influenced mindset and impression management. Residents adopted a fixed mindset and managed impressions when they perceived the permanence or consequences of evaluations within feedback. Residents adopted a growth mindset when they trusted the supervisor. DISCUSSION Residents assess the context of feedback conversations, altering the authenticity of their behaviours. Context, including the perceptions of formal assessment and relationships with supervisors, affected residents' mindset and impression management. Providing space for relationship-building and clarifying the purpose and structure of assessment may be helpful in supporting effective learning conversations in graduate medical education.
Collapse
Affiliation(s)
| | | | - Anjali Bhagra
- Division of General Internal Medicine, Mayo Clinic, Rochester, MN, USA
| | - Emily L Leasure
- Division of Community Internal Medicine, Mayo Clinic, Rochester, MN, USA
| | - Wil L Santivasi
- Division of Hospice and Palliative Medicine, Mayo Clinic, Rochester, MN, USA
| | - Adam P Sawatsky
- Division of General Internal Medicine, Mayo Clinic, Rochester, MN, USA
| |
Collapse
|